2024-03-21T08:46:03.337 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T08:46:03.337 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163 --owner scheduled_vshankar@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/orig.config.yaml 2024-03-21T08:46:03.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:46:03.492 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163 branch: wip-vshankar-testing-20240320.053002 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} email: vshankar@redhat.com first_in_suite: false job_id: '7612163' kernel: client: branch: testing flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240320.053002 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240320.053002 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8073 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 sleep_before_teardown: 0 subset: 203/1024 suite: fs suite_branch: wip-vshankar-testing-20240320.053002 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFpGf8LP1jXp81CNatiBzOo4U2gTBprAuefXx8fImVU2Ss5b1rl6FZm+sbhRNHRZNI+cuqXHrIZFYaS04bfarqc= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhfYZAonPqb6ERwQS87seGftQ9BR7x+++8t5AMUqfN2+ubU4aiGl43nNY4ldEzJ1S7Lrdu+9ChnabsDyuNyICk= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4EKKGF8alrFf6eH5MlbjtZcPbUdjEVRgKmGpBTA6OCZy78yvnrFhy0RMSAuDjEOZEI+JFgCLYwN26fC/6FSOU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-20_11:21:18 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T08:46:03.493 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa; will attempt to use it 2024-03-21T08:46:03.493 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks 2024-03-21T08:46:03.494 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T08:46:03.496 INFO:teuthology.task.internal:Checking packages... 2024-03-21T08:46:03.521 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2edfee68ad7243bed422a049b6f5f58619300f11' 2024-03-21T08:46:03.521 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T08:46:03.521 INFO:teuthology.packaging:ref: None 2024-03-21T08:46:03.521 INFO:teuthology.packaging:tag: None 2024-03-21T08:46:03.521 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240320.053002 2024-03-21T08:46:03.521 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:03.521 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240320.053002 2024-03-21T08:46:03.661 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2231.g2edfee68 2024-03-21T08:46:03.663 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T08:46:03.669 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T08:46:03.669 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T08:46:03.704 INFO:teuthology.task.internal:Saving configuration 2024-03-21T08:46:03.729 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T08:46:03.735 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T08:46:03.758 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:39:53.941984', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFpGf8LP1jXp81CNatiBzOo4U2gTBprAuefXx8fImVU2Ss5b1rl6FZm+sbhRNHRZNI+cuqXHrIZFYaS04bfarqc='} 2024-03-21T08:46:03.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:39:53.940548', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhfYZAonPqb6ERwQS87seGftQ9BR7x+++8t5AMUqfN2+ubU4aiGl43nNY4ldEzJ1S7Lrdu+9ChnabsDyuNyICk='} 2024-03-21T08:46:03.803 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:39:53.943229', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4EKKGF8alrFf6eH5MlbjtZcPbUdjEVRgKmGpBTA6OCZy78yvnrFhy0RMSAuDjEOZEI+JFgCLYwN26fC/6FSOU='} 2024-03-21T08:46:03.803 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T08:46:03.809 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-21T08:46:03.809 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-21T08:46:03.809 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-21T08:46:03.809 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T08:46:03.938 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f835432f160>, signals=[15]) 2024-03-21T08:46:03.938 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T08:46:03.946 INFO:teuthology.task.internal:Opening connections... 2024-03-21T08:46:03.946 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T08:46:03.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:46:04.075 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-03-21T08:46:04.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:46:04.181 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-03-21T08:46:04.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:46:04.283 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T08:46:04.291 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-03-21T08:46:04.310 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-03-21T08:46:04.310 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-03-21T08:46:04.368 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-03-21T08:46:04.368 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-03-21T08:46:04.368 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-03-21T08:46:04.368 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:46:04.368 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:46:04.369 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:46:04.370 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-03-21T08:46:04.391 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-03-21T08:46:04.409 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-03-21T08:46:04.409 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-03-21T08:46:04.467 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:46:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:46:04.469 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-03-21T08:46:04.491 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-03-21T08:46:04.509 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-03-21T08:46:04.509 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-03-21T08:46:04.566 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:46:04.567 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:46:04.567 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:46:04.567 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:46:04.567 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:46:04.567 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:46:04.567 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-03-21T08:46:04.591 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T08:46:04.602 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T08:46:04.612 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T08:46:04.613 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:46:04.615 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:46:04.617 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:46:04.634 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T08:46:04.646 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T08:46:04.646 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:46:04.673 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:46:04.676 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:46:04.777 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T08:46:04.849 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T08:46:04.864 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-21T08:46:04.864 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-21T08:46:04.864 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-21T08:46:04.865 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:46:04.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:46:04.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-21T08:46:05.186 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:46:05.186 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T08:46:05.186 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-03-21T08:46:05.206 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-361.el9.x86_64 2024-03-21T08:46:05.206 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:46:05.206 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T08:46:05.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum upgrade -y linux-firmware 2024-03-21T08:46:07.969 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - AppStream 6.8 MB/s | 6.7 MB 00:00 2024-03-21T08:46:09.585 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - BaseOS 4.4 MB/s | 2.0 MB 00:00 2024-03-21T08:46:10.801 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-03-21T08:46:12.685 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-03-21T08:46:17.736 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-21T08:46:19.478 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-21T08:46:20.949 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-21T08:46:25.963 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-21T08:46:28.263 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 1 Package 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:46:28.278 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 375 M 2024-03-21T08:46:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-21T08:46:36.522 INFO:teuthology.orchestra.run.smithi037.stdout:linux-firmware-20240219-143.el9.noarch.rpm 45 MB/s | 375 MB 00:08 2024-03-21T08:46:36.523 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:46:36.524 INFO:teuthology.orchestra.run.smithi037.stdout:Total 45 MB/s | 375 MB 00:08 2024-03-21T08:46:36.524 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-21T08:46:36.604 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-21T08:46:36.604 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-21T08:46:39.102 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-21T08:46:39.102 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-21T08:46:45.566 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-21T08:46:45.846 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:46:46.049 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:46:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:46:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T08:46:47.727 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T08:46:47.727 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:46:47.727 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-03-21T08:46:47.727 INFO:teuthology.orchestra.run.smithi037.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T08:46:47.727 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:46:47.727 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-21T08:46:48.050 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-21T08:46:48.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T08:46:48.196 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T08:46:48.196 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T08:46:48.200 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi037.front.sepia.ceph.com to complete... 2024-03-21T08:46:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21 08:46:59 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T08:46:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:FINISHED --2024-03-21 08:46:59-- 2024-03-21T08:46:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:Total wall clock time: 11s 2024-03-21T08:46:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-21T08:46:59.598 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-21T08:46:59.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T08:47:01.637 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying... ######################################## 2024-03-21T08:47:01.701 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing... ######################################## 2024-03-21T08:47:01.785 INFO:teuthology.orchestra.run.smithi037.stdout:Updating / installing... 2024-03-21T08:47:09.616 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T08:47:25.355 INFO:teuthology.orchestra.run.smithi037.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T08:47:48.481 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T08:47:48.482 DEBUG:teuthology.orchestra.run.smithi037:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T08:47:48.563 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T08:47:48.563 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T08:47:48.564 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T08:47:48.565 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T08:47:48.566 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T08:47:48.567 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T08:47:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T08:47:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T08:47:48.570 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T08:47:48.571 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T08:47:48.572 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T08:47:48.573 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T08:47:48.574 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T08:47:48.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T08:47:48.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T08:47:48.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T08:47:48.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T08:47:48.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T08:47:48.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T08:47:48.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T08:47:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T08:47:48.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T08:47:48.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T08:47:48.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T08:47:48.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T08:47:48.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T08:47:48.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T08:47:48.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T08:47:48.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T08:47:48.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T08:47:48.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T08:47:48.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T08:47:48.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T08:47:48.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T08:47:48.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T08:47:48.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T08:47:48.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T08:47:48.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T08:47:48.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T08:47:48.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T08:47:48.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T08:47:48.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T08:47:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T08:47:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T08:47:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T08:47:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T08:47:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T08:47:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T08:47:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T08:47:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T08:47:48.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T08:47:48.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T08:47:48.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T08:47:48.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T08:47:48.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T08:47:48.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T08:47:48.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T08:47:48.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T08:47:48.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T08:47:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T08:47:48.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T08:47:48.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T08:47:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T08:47:48.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T08:47:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T08:47:48.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T08:47:48.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T08:47:48.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T08:47:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T08:47:48.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T08:47:48.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T08:47:48.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T08:47:48.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T08:47:48.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T08:47:48.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T08:47:48.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T08:47:48.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T08:47:48.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T08:47:48.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T08:47:48.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T08:47:48.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T08:47:48.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T08:47:48.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T08:47:48.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T08:47:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T08:47:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T08:47:48.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T08:47:48.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T08:47:48.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T08:47:48.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T08:47:48.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T08:47:48.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T08:47:48.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T08:47:48.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T08:47:48.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T08:47:48.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T08:47:48.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T08:47:48.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T08:47:48.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T08:47:48.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T08:47:48.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T08:47:48.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T08:47:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T08:47:48.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T08:47:48.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T08:47:48.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T08:47:48.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T08:47:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T08:47:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T08:47:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T08:47:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T08:47:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T08:47:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T08:47:48.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T08:47:48.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T08:47:48.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T08:47:48.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T08:47:48.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T08:47:48.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T08:47:48.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T08:47:48.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T08:47:48.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T08:47:48.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T08:47:48.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T08:47:48.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T08:47:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T08:47:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T08:47:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T08:47:48.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T08:47:48.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T08:47:48.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T08:47:48.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T08:47:48.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T08:47:48.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T08:47:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T08:47:48.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T08:47:48.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T08:47:48.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T08:47:48.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T08:47:48.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T08:47:48.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T08:47:48.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T08:47:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T08:47:48.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T08:47:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T08:47:48.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T08:47:48.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T08:47:48.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T08:47:48.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T08:47:48.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T08:47:48.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T08:47:48.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T08:47:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T08:47:48.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T08:47:48.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T08:47:48.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T08:47:48.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T08:47:48.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T08:47:48.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T08:47:48.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T08:47:48.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T08:47:48.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T08:47:48.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T08:47:48.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T08:47:48.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T08:47:48.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T08:47:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T08:47:48.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T08:47:48.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T08:47:48.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T08:47:48.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T08:47:48.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T08:47:48.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T08:47:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T08:47:48.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T08:47:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T08:47:48.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T08:47:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T08:47:48.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T08:47:48.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T08:47:48.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T08:47:48.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T08:47:48.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T08:47:48.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T08:47:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T08:47:48.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T08:47:48.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T08:47:48.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T08:47:48.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T08:47:48.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T08:47:48.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T08:47:48.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T08:47:48.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T08:47:48.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T08:47:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T08:47:48.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T08:47:48.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T08:47:48.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T08:47:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T08:47:48.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T08:47:48.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T08:47:48.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T08:47:48.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T08:47:48.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T08:47:48.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:47:48.737 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T08:47:48.737 DEBUG:teuthology.orchestra.run.smithi037:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout:for file in vmlinuz System.map config; do 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T08:47:48.761 INFO:teuthology.orchestra.run.smithi037.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout: fi 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:done 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:47:48.762 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-03-21T08:47:48.763 INFO:teuthology.orchestra.run.smithi037.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T08:47:48.763 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T08:47:48.763 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T08:47:48.763 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-03-21T08:47:48.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-03-21T08:47:48.844 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-03-21T08:47:48.844 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-03-21T08:47:48.844 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 68.el9 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8264254 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-03-21T08:47:48.845 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:47:48.846 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T08:47:48.847 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T08:47:48.847 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T08:47:48.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T08:47:49.464 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-03-21T08:47:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T08:47:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-03-21T08:47:50.941 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-03-21T08:47:50.957 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.F16l7Z80E7 2024-03-21T08:47:50.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.F16l7Z80E7 2024-03-21T08:47:51.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.F16l7Z80E7 2024-03-21T08:47:51.169 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.F16l7Z80E7 is 6KB 2024-03-21T08:47:51.218 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.F16l7Z80E7 2024-03-21T08:47:51.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T08:47:51.301 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T08:47:51.301 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T08:47:51.301 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T08:47:51.301 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T08:47:51.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T08:47:51.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-03-21T08:48:21.494 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-21T08:48:21.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-21T08:48:21.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:48:39.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-21T08:48:48.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-21T08:48:48.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:49:48.975 DEBUG:teuthology.orchestra.remote:timed out 2024-03-21T08:50:00.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-21T08:50:00.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:50:01.377 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-21T08:50:02.085 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-21T08:50:02.086 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T08:50:02.086 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-03-21T08:50:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T08:50:02.141 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T08:50:02.141 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T08:50:02.141 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T08:50:03.142 DEBUG:teuthology.parallel:result is None 2024-03-21T08:50:03.143 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T08:50:03.150 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T08:50:03.150 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:50:03.158 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:50:03.160 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:50:03.206 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T08:50:03.213 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T08:50:03.241 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T08:50:03.242 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:50:03.249 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:50:03.252 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:50:03.319 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T08:50:03.326 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T08:50:03.326 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:50:03.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:50:03.344 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:50:03.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:50:03.360 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:50:03.367 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:50:03.387 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:50:03.426 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:50:03.455 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:50:03.456 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:50:03.474 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:50:03.502 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:50:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:50:03.589 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T08:50:03.597 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T08:50:03.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:50:03.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:50:03.605 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:50:03.700 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T08:50:03.714 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T08:50:03.714 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:50:03.720 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:50:03.722 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:50:03.771 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:50:03.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:50:03.969 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:50:04.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:50:04.151 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:50:04.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:50:04.252 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:50:04.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:50:04.363 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:50:04.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:50:04.456 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T08:50:04.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:50:04.528 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:50:04.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:50:04.640 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:50:04.684 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:50:04.728 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T08:50:04.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:50:04.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-03-21T08:50:04.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-03-21T08:50:04.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-03-21T08:50:04.848 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:50:04.879 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:50:04.931 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:50:05.246 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T08:50:05.254 INFO:teuthology.task.internal:Starting timer... 2024-03-21T08:50:05.255 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T08:50:05.281 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T08:50:05.310 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-21T08:50:05.310 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-03-21T08:50:05.437 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-03-21T08:50:05.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-03-21T08:50:05.505 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-03-21T08:50:05.507 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-03-21T08:50:05.582 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-03-21T08:50:05.584 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T08:50:05.584 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-03-21T08:50:05.643 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-03-21T08:50:05.643 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-03-21T08:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-03-21T08:50:05.671 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-03-21T08:50:05.693 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-03-21T08:50:05.693 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive'} 2024-03-21T08:50:05.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:50:05.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:50:05.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:50:05.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:50:05.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:50:05.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:50:05.838 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T08:50:05.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:50:05.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:50:05.989 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:50:06.028 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T08:50:06.038 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T08:50:06.058 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T08:50:06.059 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi071.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T08:56:16.752 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com')] 2024-03-21T08:56:16.753 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-21T08:56:16.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:56:16.909 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-21T08:56:17.058 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-21T08:56:17.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-21T08:56:17.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:56:17.164 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-03-21T08:56:17.234 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-21T08:56:17.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-21T08:56:17.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:56:17.326 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-03-21T08:56:17.402 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-21T08:56:17.403 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T08:56:17.415 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T08:56:17.416 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:56:17.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:56:17.423 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:56:17.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:56:17.426 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:56:17.426 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:56:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:56:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:56:17.486 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:56:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:56:17.520 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:56:17.524 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-03-21T08:56:17.525 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-03-21T08:56:17.539 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-03-21T08:56:17.542 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-03-21T08:56:17.556 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:56:17.562 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:56:17.573 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:56:17.574 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:56:17.583 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:56:17.617 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:56:17.620 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:56:17.620 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-03-21T08:56:17.620 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.620 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.621 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.621 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.631 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:56:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:56:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-03-21T08:56:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-03-21T08:56:17.717 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-03-21T08:56:17.775 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:56:17.834 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:56:17.970 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:56:17.979 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:56:17.979 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-03-21T08:56:17.979 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.979 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.979 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.979 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:56:17.981 INFO:teuthology.run_tasks:Running task install... 2024-03-21T08:56:17.990 DEBUG:teuthology.task.install:project ceph 2024-03-21T08:56:17.990 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'}} 2024-03-21T08:56:17.991 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T08:56:17.991 INFO:teuthology.task.install:Using flavor: default 2024-03-21T08:56:17.997 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T08:56:17.997 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-21T08:56:17.997 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:56:17.998 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:56:18.000 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:56:18.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:56:18.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:56:18.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:56:18.151 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:56:18.151 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:56:18.167 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:56:18.168 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:56:18.183 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:56:18.183 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:56:18.258 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:56:18.258 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T08:56:18.259 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:56:18.274 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:56:18.274 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T08:56:18.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:56:18.289 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:56:18.290 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:56:18.301 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:56:18.301 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:56:18.301 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:56:18.310 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:56:18.310 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:56:18.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:56:18.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:56:18.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:56:18.410 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:56:18.437 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:56:18.475 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:56:18.502 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-03-21T08:56:18.504 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-03-21T08:56:18.506 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-03-21T08:56:18.508 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-03-21T08:56:18.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:56:18.697 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:56:18.737 INFO:teuthology.orchestra.run.smithi071.stdout:74 files removed 2024-03-21T08:56:18.757 INFO:teuthology.orchestra.run.smithi053.stdout:74 files removed 2024-03-21T08:56:18.764 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:56:18.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:56:18.790 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-03-21T08:56:18.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-03-21T08:56:19.149 INFO:teuthology.orchestra.run.smithi037.stdout:74 files removed 2024-03-21T08:56:19.189 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:56:19.382 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 251 kB/s | 83 kB 00:00 2024-03-21T08:56:19.457 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 228 kB/s | 83 kB 00:00 2024-03-21T08:56:19.716 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 34 kB/s | 10 kB 00:00 2024-03-21T08:56:19.815 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-21T08:56:20.067 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-21T08:56:20.123 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 206 kB/s | 83 kB 00:00 2024-03-21T08:56:20.167 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-21T08:56:20.452 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-03-21T08:56:20.585 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-21T08:56:20.632 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-21T08:56:21.141 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-03-21T08:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-21T08:56:22.002 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2024-03-21T08:56:22.061 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-21T08:56:22.843 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-21T08:56:23.360 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-03-21T08:56:23.428 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-21T08:56:24.744 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-21T08:56:25.422 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-21T08:56:25.610 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-03-21T08:56:27.842 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-21T08:56:28.753 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-21T08:56:30.245 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-03-21T08:56:30.570 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.1 MB 00:01 2024-03-21T08:56:31.988 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-03-21T08:56:35.109 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 21 MB 00:03 2024-03-21T08:56:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 797 kB/s | 6.1 MB 00:07 2024-03-21T08:56:40.331 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-21T08:56:41.809 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-03-21T08:56:42.796 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:56:42.797 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:56:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:56:42.812 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:56:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:56:42.814 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:56:42.817 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:56:42.820 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:56:42.821 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:56:42.911 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:56:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:56:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:56:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:56:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:56:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:56:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:56:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:56:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:56:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:56:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:56:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:56:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:56:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:Install 150 Packages 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 24 Packages 2024-03-21T08:56:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:56:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 224 M 2024-03-21T08:56:42.938 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-03-21T08:56:43.683 INFO:teuthology.orchestra.run.smithi053.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-03-21T08:56:44.025 INFO:teuthology.orchestra.run.smithi053.stdout:(2/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 9.8 MB/s | 5.4 MB 00:00 2024-03-21T08:56:44.066 INFO:teuthology.orchestra.run.smithi053.stdout:(3/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 2.3 MB/s | 890 kB 00:00 2024-03-21T08:56:44.092 INFO:teuthology.orchestra.run.smithi053.stdout:(4/174): ceph-immutable-object-cache-19.0.0-223 2.1 MB/s | 145 kB 00:00 2024-03-21T08:56:44.209 INFO:teuthology.orchestra.run.smithi053.stdout:(5/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-21T08:56:44.567 INFO:teuthology.orchestra.run.smithi053.stdout:(6/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-21T08:56:44.976 INFO:teuthology.orchestra.run.smithi053.stdout:(7/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-03-21T08:56:45.810 INFO:teuthology.orchestra.run.smithi053.stdout:(8/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x86 14 MB/s | 17 MB 00:01 2024-03-21T08:56:45.851 INFO:teuthology.orchestra.run.smithi053.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 610 kB/s | 25 kB 00:00 2024-03-21T08:56:46.911 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 21 MB 00:04 2024-03-21T08:56:47.811 INFO:teuthology.orchestra.run.smithi053.stdout:(10/174): ceph-common-19.0.0-2231.g2edfee68.el9 4.7 MB/s | 21 MB 00:04 2024-03-21T08:56:47.860 INFO:teuthology.orchestra.run.smithi053.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 4.3 MB/s | 220 kB 00:00 2024-03-21T08:56:47.995 INFO:teuthology.orchestra.run.smithi053.stdout:(12/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 3.3 MB/s | 9.9 MB 00:03 2024-03-21T08:56:48.011 INFO:teuthology.orchestra.run.smithi053.stdout:(13/174): cephfs-shell-19.0.0-2231.g2edfee68.el 180 kB/s | 27 kB 00:00 2024-03-21T08:56:48.044 INFO:teuthology.orchestra.run.smithi053.stdout:(14/174): libcephfs-devel-19.0.0-2231.g2edfee68 626 kB/s | 31 kB 00:00 2024-03-21T08:56:48.120 INFO:teuthology.orchestra.run.smithi053.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.1 MB/s | 163 kB 00:00 2024-03-21T08:56:48.162 INFO:teuthology.orchestra.run.smithi053.stdout:(16/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 4.8 MB/s | 747 kB 00:00 2024-03-21T08:56:48.187 INFO:teuthology.orchestra.run.smithi053.stdout:(17/174): librados-devel-19.0.0-2231.g2edfee68. 1.9 MB/s | 127 kB 00:00 2024-03-21T08:56:48.695 INFO:teuthology.orchestra.run.smithi053.stdout:(18/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 17 MB/s | 48 MB 00:02 2024-03-21T08:56:48.720 INFO:teuthology.orchestra.run.smithi053.stdout:(19/174): libradosstriper1-19.0.0-2231.g2edfee6 897 kB/s | 501 kB 00:00 2024-03-21T08:56:48.746 INFO:teuthology.orchestra.run.smithi053.stdout:(20/174): python3-ceph-argparse-19.0.0-2231.g2e 902 kB/s | 45 kB 00:00 2024-03-21T08:56:48.771 INFO:teuthology.orchestra.run.smithi053.stdout:(21/174): python3-ceph-common-19.0.0-2231.g2edf 2.6 MB/s | 131 kB 00:00 2024-03-21T08:56:48.797 INFO:teuthology.orchestra.run.smithi053.stdout:(22/174): python3-cephfs-19.0.0-2231.g2edfee68. 3.1 MB/s | 161 kB 00:00 2024-03-21T08:56:48.847 INFO:teuthology.orchestra.run.smithi053.stdout:(23/174): python3-rados-19.0.0-2231.g2edfee68.e 4.2 MB/s | 320 kB 00:00 2024-03-21T08:56:48.880 INFO:teuthology.orchestra.run.smithi053.stdout:(24/174): python3-rbd-19.0.0-2231.g2edfee68.el9 3.5 MB/s | 297 kB 00:00 2024-03-21T08:56:48.906 INFO:teuthology.orchestra.run.smithi053.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.7 MB/s | 99 kB 00:00 2024-03-21T08:56:48.931 INFO:teuthology.orchestra.run.smithi053.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-21T08:56:48.981 INFO:teuthology.orchestra.run.smithi053.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 3.3 MB/s | 171 kB 00:00 2024-03-21T08:56:49.040 INFO:teuthology.orchestra.run.smithi053.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 584 kB/s | 34 kB 00:00 2024-03-21T08:56:49.099 INFO:teuthology.orchestra.run.smithi053.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 2.4 MB/s | 140 kB 00:00 2024-03-21T08:56:49.266 INFO:teuthology.orchestra.run.smithi053.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 11 MB/s | 1.8 MB 00:00 2024-03-21T08:56:49.333 INFO:teuthology.orchestra.run.smithi053.stdout:(31/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-03-21T08:56:49.391 INFO:teuthology.orchestra.run.smithi053.stdout:(32/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 6.4 MB/s | 3.1 MB 00:00 2024-03-21T08:56:49.450 INFO:teuthology.orchestra.run.smithi053.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.1 MB/s | 248 kB 00:00 2024-03-21T08:56:49.467 INFO:teuthology.orchestra.run.smithi053.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 658 kB/s | 49 kB 00:00 2024-03-21T08:56:49.508 INFO:teuthology.orchestra.run.smithi053.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 282 kB/s | 16 kB 00:00 2024-03-21T08:56:49.534 INFO:teuthology.orchestra.run.smithi053.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 3.9 MB/s | 264 kB 00:00 2024-03-21T08:56:49.592 INFO:teuthology.orchestra.run.smithi053.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 312 kB/s | 18 kB 00:00 2024-03-21T08:56:49.709 INFO:teuthology.orchestra.run.smithi053.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-03-21T08:56:49.751 INFO:teuthology.orchestra.run.smithi053.stdout:(39/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 3.0 MB/s | 750 kB 00:00 2024-03-21T08:56:49.836 INFO:teuthology.orchestra.run.smithi053.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-03-21T08:56:49.861 INFO:teuthology.orchestra.run.smithi053.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 969 kB/s | 106 kB 00:00 2024-03-21T08:56:49.937 INFO:teuthology.orchestra.run.smithi053.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 920 kB 00:00 2024-03-21T08:56:49.962 INFO:teuthology.orchestra.run.smithi053.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 307 kB 00:00 2024-03-21T08:56:49.979 INFO:teuthology.orchestra.run.smithi053.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 273 kB/s | 32 kB 00:00 2024-03-21T08:56:50.013 INFO:teuthology.orchestra.run.smithi053.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 342 kB/s | 17 kB 00:00 2024-03-21T08:56:50.063 INFO:teuthology.orchestra.run.smithi053.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-21T08:56:50.079 INFO:teuthology.orchestra.run.smithi053.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 10 kB 00:00 2024-03-21T08:56:50.105 INFO:teuthology.orchestra.run.smithi053.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-03-21T08:56:50.130 INFO:teuthology.orchestra.run.smithi053.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-21T08:56:50.155 INFO:teuthology.orchestra.run.smithi053.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-21T08:56:50.189 INFO:teuthology.orchestra.run.smithi053.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 578 kB 00:00 2024-03-21T08:56:50.214 INFO:teuthology.orchestra.run.smithi053.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 46 kB 00:00 2024-03-21T08:56:50.239 INFO:teuthology.orchestra.run.smithi053.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 282 kB 00:00 2024-03-21T08:56:50.265 INFO:teuthology.orchestra.run.smithi053.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-21T08:56:50.290 INFO:teuthology.orchestra.run.smithi053.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-21T08:56:50.315 INFO:teuthology.orchestra.run.smithi053.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 458 kB/s | 34 kB 00:00 2024-03-21T08:56:50.399 INFO:teuthology.orchestra.run.smithi053.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-21T08:56:50.441 INFO:teuthology.orchestra.run.smithi053.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-21T08:56:50.466 INFO:teuthology.orchestra.run.smithi053.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.4 MB/s | 811 kB 00:00 2024-03-21T08:56:50.499 INFO:teuthology.orchestra.run.smithi053.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-21T08:56:50.566 INFO:teuthology.orchestra.run.smithi053.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-03-21T08:56:50.650 INFO:teuthology.orchestra.run.smithi053.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-21T08:56:50.675 INFO:teuthology.orchestra.run.smithi053.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 454 kB/s | 78 kB 00:00 2024-03-21T08:56:50.767 INFO:teuthology.orchestra.run.smithi053.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-21T08:56:50.784 INFO:teuthology.orchestra.run.smithi053.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-21T08:56:50.801 INFO:teuthology.orchestra.run.smithi053.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 148 kB/s | 18 kB 00:00 2024-03-21T08:56:50.835 INFO:teuthology.orchestra.run.smithi053.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 4.3 MB/s | 144 kB 00:00 2024-03-21T08:56:50.860 INFO:teuthology.orchestra.run.smithi053.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 543 kB/s | 32 kB 00:00 2024-03-21T08:56:50.879 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-21T08:56:50.943 INFO:teuthology.orchestra.run.smithi053.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-21T08:56:50.969 INFO:teuthology.orchestra.run.smithi053.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-21T08:56:50.994 INFO:teuthology.orchestra.run.smithi053.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-21T08:56:51.019 INFO:teuthology.orchestra.run.smithi053.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-21T08:56:51.036 INFO:teuthology.orchestra.run.smithi053.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 989 kB/s | 41 kB 00:00 2024-03-21T08:56:51.061 INFO:teuthology.orchestra.run.smithi053.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-21T08:56:51.086 INFO:teuthology.orchestra.run.smithi053.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-21T08:56:51.112 INFO:teuthology.orchestra.run.smithi053.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-21T08:56:51.129 INFO:teuthology.orchestra.run.smithi053.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 711 kB/s | 47 kB 00:00 2024-03-21T08:56:51.146 INFO:teuthology.orchestra.run.smithi053.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 733 kB/s | 43 kB 00:00 2024-03-21T08:56:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 515 kB/s | 39 kB 00:00 2024-03-21T08:56:51.313 INFO:teuthology.orchestra.run.smithi053.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 247 kB/s | 41 kB 00:00 2024-03-21T08:56:51.530 INFO:teuthology.orchestra.run.smithi053.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-03-21T08:56:51.555 INFO:teuthology.orchestra.run.smithi053.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 863 kB/s | 300 kB 00:00 2024-03-21T08:56:51.572 INFO:teuthology.orchestra.run.smithi053.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 220 kB/s | 56 kB 00:00 2024-03-21T08:56:51.589 INFO:teuthology.orchestra.run.smithi053.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-21T08:56:51.606 INFO:teuthology.orchestra.run.smithi053.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T08:56:51.631 INFO:teuthology.orchestra.run.smithi053.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-21T08:56:51.656 INFO:teuthology.orchestra.run.smithi053.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-21T08:56:51.698 INFO:teuthology.orchestra.run.smithi053.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-21T08:56:51.723 INFO:teuthology.orchestra.run.smithi053.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-21T08:56:51.749 INFO:teuthology.orchestra.run.smithi053.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-21T08:56:51.774 INFO:teuthology.orchestra.run.smithi053.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-21T08:56:52.058 INFO:teuthology.orchestra.run.smithi053.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 689 kB/s | 195 kB 00:00 2024-03-21T08:56:52.083 INFO:teuthology.orchestra.run.smithi053.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 522 kB/s | 187 kB 00:00 2024-03-21T08:56:52.125 INFO:teuthology.orchestra.run.smithi053.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-21T08:56:52.250 INFO:teuthology.orchestra.run.smithi053.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-03-21T08:56:52.276 INFO:teuthology.orchestra.run.smithi053.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 112 kB/s | 21 kB 00:00 2024-03-21T08:56:52.434 INFO:teuthology.orchestra.run.smithi053.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-21T08:56:52.493 INFO:teuthology.orchestra.run.smithi053.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-03-21T08:56:52.551 INFO:teuthology.orchestra.run.smithi053.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 216 kB 00:00 2024-03-21T08:56:52.604 INFO:teuthology.orchestra.run.smithi053.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 663 kB/s | 35 kB 00:00 2024-03-21T08:56:52.655 INFO:teuthology.orchestra.run.smithi053.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 891 kB/s | 44 kB 00:00 2024-03-21T08:56:52.722 INFO:teuthology.orchestra.run.smithi053.stdout:(102/174): openssl-devel-3.0.7-27.el9.x86_64.rp 8.8 MB/s | 4.1 MB 00:00 2024-03-21T08:56:52.747 INFO:teuthology.orchestra.run.smithi053.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-21T08:56:52.822 INFO:teuthology.orchestra.run.smithi053.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-03-21T08:56:52.886 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-21T08:56:52.931 INFO:teuthology.orchestra.run.smithi053.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-03-21T08:56:53.040 INFO:teuthology.orchestra.run.smithi053.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-03-21T08:56:53.074 INFO:teuthology.orchestra.run.smithi053.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 589 kB/s | 19 kB 00:00 2024-03-21T08:56:53.098 INFO:teuthology.orchestra.run.smithi053.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-03-21T08:56:53.132 INFO:teuthology.orchestra.run.smithi053.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 780 kB/s | 26 kB 00:00 2024-03-21T08:56:53.174 INFO:teuthology.orchestra.run.smithi053.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-03-21T08:56:53.233 INFO:teuthology.orchestra.run.smithi053.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-03-21T08:56:53.258 INFO:teuthology.orchestra.run.smithi053.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 804 kB/s | 67 kB 00:00 2024-03-21T08:56:53.283 INFO:teuthology.orchestra.run.smithi053.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-03-21T08:56:53.333 INFO:teuthology.orchestra.run.smithi053.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-03-21T08:56:53.367 INFO:teuthology.orchestra.run.smithi053.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-03-21T08:56:53.392 INFO:teuthology.orchestra.run.smithi053.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-03-21T08:56:53.419 INFO:teuthology.orchestra.run.smithi053.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 636 kB/s | 32 kB 00:00 2024-03-21T08:56:53.443 INFO:teuthology.orchestra.run.smithi053.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-03-21T08:56:53.469 INFO:teuthology.orchestra.run.smithi053.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-03-21T08:56:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-03-21T08:56:53.527 INFO:teuthology.orchestra.run.smithi053.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-03-21T08:56:53.552 INFO:teuthology.orchestra.run.smithi053.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-03-21T08:56:53.578 INFO:teuthology.orchestra.run.smithi053.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-03-21T08:56:53.620 INFO:teuthology.orchestra.run.smithi053.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-03-21T08:56:53.670 INFO:teuthology.orchestra.run.smithi053.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-03-21T08:56:53.695 INFO:teuthology.orchestra.run.smithi053.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-21T08:56:53.721 INFO:teuthology.orchestra.run.smithi053.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-03-21T08:56:53.746 INFO:teuthology.orchestra.run.smithi053.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-03-21T08:56:53.771 INFO:teuthology.orchestra.run.smithi053.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-21T08:56:53.797 INFO:teuthology.orchestra.run.smithi053.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-21T08:56:53.830 INFO:teuthology.orchestra.run.smithi053.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-03-21T08:56:53.847 INFO:teuthology.orchestra.run.smithi053.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 925 kB/s | 46 kB 00:00 2024-03-21T08:56:53.872 INFO:teuthology.orchestra.run.smithi053.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-03-21T08:56:53.890 INFO:teuthology.orchestra.run.smithi053.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-03-21T08:56:53.915 INFO:teuthology.orchestra.run.smithi053.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-03-21T08:56:53.932 INFO:teuthology.orchestra.run.smithi053.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 390 kB/s | 16 kB 00:00 2024-03-21T08:56:53.957 INFO:teuthology.orchestra.run.smithi053.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-03-21T08:56:53.974 INFO:teuthology.orchestra.run.smithi053.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-03-21T08:56:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 916 kB/s | 31 kB 00:00 2024-03-21T08:56:54.024 INFO:teuthology.orchestra.run.smithi053.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-21T08:56:54.041 INFO:teuthology.orchestra.run.smithi053.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-21T08:56:54.058 INFO:teuthology.orchestra.run.smithi053.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-03-21T08:56:54.075 INFO:teuthology.orchestra.run.smithi053.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-03-21T08:56:54.109 INFO:teuthology.orchestra.run.smithi053.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-21T08:56:54.145 INFO:teuthology.orchestra.run.smithi053.stdout:(145/174): libgfortran-11.4.1-3.el9.x86_64.rpm 335 kB/s | 803 kB 00:02 2024-03-21T08:56:54.170 INFO:teuthology.orchestra.run.smithi053.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 951 kB/s | 90 kB 00:00 2024-03-21T08:56:54.195 INFO:teuthology.orchestra.run.smithi053.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.8 MB/s | 427 kB 00:00 2024-03-21T08:56:54.221 INFO:teuthology.orchestra.run.smithi053.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-03-21T08:56:54.246 INFO:teuthology.orchestra.run.smithi053.stdout:(149/174): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-03-21T08:56:54.288 INFO:teuthology.orchestra.run.smithi053.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-21T08:56:54.446 INFO:teuthology.orchestra.run.smithi053.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.4 MB/s | 321 kB 00:00 2024-03-21T08:56:54.697 INFO:teuthology.orchestra.run.smithi053.stdout:(152/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-03-21T08:56:54.855 INFO:teuthology.orchestra.run.smithi053.stdout:(153/174): librados2-19.0.0-2231.g2edfee68.el9. 5.3 MB/s | 3.4 MB 00:00 2024-03-21T08:56:54.989 INFO:teuthology.orchestra.run.smithi053.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 316 kB/s | 42 kB 00:00 2024-03-21T08:56:55.072 INFO:teuthology.orchestra.run.smithi053.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-03-21T08:56:55.139 INFO:teuthology.orchestra.run.smithi053.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 71 kB 00:00 2024-03-21T08:56:55.323 INFO:teuthology.orchestra.run.smithi053.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 199 kB/s | 36 kB 00:00 2024-03-21T08:56:55.540 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:56:55.541 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:56:55.561 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:56:55.562 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:56:55.563 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:56:55.565 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:56:55.567 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:56:55.570 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:56:55.571 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:56:55.665 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-21T08:56:55.683 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-21T08:56:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-21T08:56:55.683 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-21T08:56:55.683 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-21T08:56:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:56:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:56:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:56:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:56:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:56:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:56:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:56:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:56:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:56:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:56:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:56:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:56:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:56:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:56:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-21T08:56:55.697 INFO:teuthology.orchestra.run.smithi037.stdout:Install 150 Packages 2024-03-21T08:56:55.698 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 24 Packages 2024-03-21T08:56:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:56:55.698 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 224 M 2024-03-21T08:56:55.698 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-21T08:56:55.736 INFO:teuthology.orchestra.run.smithi053.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 465 kB/s | 192 kB 00:00 2024-03-21T08:56:55.926 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 7.9 kB/s | 1.7 kB 00:00 2024-03-21T08:56:56.507 INFO:teuthology.orchestra.run.smithi053.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 332 kB/s | 256 kB 00:00 2024-03-21T08:56:56.557 INFO:teuthology.orchestra.run.smithi053.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 463 kB/s | 23 kB 00:00 2024-03-21T08:56:56.796 INFO:teuthology.orchestra.run.smithi037.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-21T08:56:57.015 INFO:teuthology.orchestra.run.smithi053.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 236 kB/s | 108 kB 00:00 2024-03-21T08:56:57.096 INFO:teuthology.orchestra.run.smithi037.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 2.9 MB/s | 890 kB 00:00 2024-03-21T08:56:57.140 INFO:teuthology.orchestra.run.smithi053.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-03-21T08:56:57.171 INFO:teuthology.orchestra.run.smithi037.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 1.9 MB/s | 145 kB 00:00 2024-03-21T08:56:57.296 INFO:teuthology.orchestra.run.smithi037.stdout:(4/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 6.8 MB/s | 5.4 MB 00:00 2024-03-21T08:56:57.316 INFO:teuthology.orchestra.run.smithi053.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 776 kB/s | 136 kB 00:00 2024-03-21T08:56:57.497 INFO:teuthology.orchestra.run.smithi037.stdout:(5/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-03-21T08:56:57.680 INFO:teuthology.orchestra.run.smithi037.stdout:(6/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 4.6 MB/s | 2.3 MB 00:00 2024-03-21T08:56:57.972 INFO:teuthology.orchestra.run.smithi037.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 9.9 MB/s | 4.7 MB 00:00 2024-03-21T08:56:58.116 INFO:teuthology.orchestra.run.smithi053.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 80 kB/s | 64 kB 00:00 2024-03-21T08:56:58.236 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:56:58.237 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:56:58.248 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:56:58.249 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:56:58.250 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:56:58.251 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:56:58.253 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:56:58.255 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:56:58.256 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:56:58.333 INFO:teuthology.orchestra.run.smithi053.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 132 kB/s | 29 kB 00:00 2024-03-21T08:56:58.335 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-03-21T08:56:58.358 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-03-21T08:56:58.358 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-03-21T08:56:58.358 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-03-21T08:56:58.358 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:56:58.359 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:56:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:56:58.361 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:56:58.362 INFO:teuthology.orchestra.run.smithi071.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:56:58.363 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:56:58.364 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:56:58.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:56:58.366 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:56:58.367 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:56:58.368 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:56:58.369 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:56:58.370 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:56:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:56:58.372 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:56:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:56:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-03-21T08:56:58.375 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-03-21T08:56:58.376 INFO:teuthology.orchestra.run.smithi071.stdout:Install 150 Packages 2024-03-21T08:56:58.376 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 24 Packages 2024-03-21T08:56:58.376 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:56:58.376 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 224 M 2024-03-21T08:56:58.376 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-03-21T08:56:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:(8/174): ceph-common-19.0.0-2231.g2edfee68.el9. 10 MB/s | 21 MB 00:02 2024-03-21T08:56:58.631 INFO:teuthology.orchestra.run.smithi037.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 383 kB/s | 25 kB 00:00 2024-03-21T08:56:58.931 INFO:teuthology.orchestra.run.smithi037.stdout:(10/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 10 MB/s | 9.9 MB 00:00 2024-03-21T08:56:59.023 INFO:teuthology.orchestra.run.smithi037.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 2.4 MB/s | 220 kB 00:00 2024-03-21T08:56:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 472 kB/s | 27 kB 00:00 2024-03-21T08:56:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-03-21T08:56:59.140 INFO:teuthology.orchestra.run.smithi037.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 545 kB/s | 31 kB 00:00 2024-03-21T08:56:59.291 INFO:teuthology.orchestra.run.smithi037.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 4.9 MB/s | 747 kB 00:00 2024-03-21T08:56:59.416 INFO:teuthology.orchestra.run.smithi037.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.3 MB/s | 163 kB 00:00 2024-03-21T08:56:59.499 INFO:teuthology.orchestra.run.smithi037.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.5 MB/s | 127 kB 00:00 2024-03-21T08:56:59.608 INFO:teuthology.orchestra.run.smithi037.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 4.6 MB/s | 501 kB 00:00 2024-03-21T08:56:59.668 INFO:teuthology.orchestra.run.smithi071.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 1.5 MB/s | 890 kB 00:00 2024-03-21T08:57:00.052 INFO:teuthology.orchestra.run.smithi071.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 379 kB/s | 145 kB 00:00 2024-03-21T08:57:00.417 INFO:teuthology.orchestra.run.smithi037.stdout:(18/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 6.4 MB/s | 5.1 MB 00:00 2024-03-21T08:57:00.517 INFO:teuthology.orchestra.run.smithi037.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 456 kB/s | 45 kB 00:00 2024-03-21T08:57:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.4 MB/s | 131 kB 00:00 2024-03-21T08:57:00.692 INFO:teuthology.orchestra.run.smithi037.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.9 MB/s | 161 kB 00:00 2024-03-21T08:57:00.866 INFO:teuthology.orchestra.run.smithi053.stdout:(166/174): NetworkManager-libnm-1.47.2-1.el9.x8 304 kB/s | 1.8 MB 00:06 2024-03-21T08:57:00.918 INFO:teuthology.orchestra.run.smithi037.stdout:(22/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 5.2 MB/s | 17 MB 00:03 2024-03-21T08:57:00.959 INFO:teuthology.orchestra.run.smithi037.stdout:(23/174): python3-rados-19.0.0-2231.g2edfee68.e 1.2 MB/s | 320 kB 00:00 2024-03-21T08:57:01.018 INFO:teuthology.orchestra.run.smithi037.stdout:(24/174): python3-rbd-19.0.0-2231.g2edfee68.el9 2.9 MB/s | 297 kB 00:00 2024-03-21T08:57:01.052 INFO:teuthology.orchestra.run.smithi037.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.1 MB/s | 99 kB 00:00 2024-03-21T08:57:01.110 INFO:teuthology.orchestra.run.smithi037.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 938 kB/s | 85 kB 00:00 2024-03-21T08:57:01.560 INFO:teuthology.orchestra.run.smithi071.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 1.5 MB/s | 2.3 MB 00:01 2024-03-21T08:57:01.625 INFO:teuthology.orchestra.run.smithi053.stdout:(167/174): NetworkManager-1.47.2-1.el9.x86_64.r 329 kB/s | 2.3 MB 00:07 2024-03-21T08:57:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:(27/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 5.1 MB/s | 3.1 MB 00:00 2024-03-21T08:57:01.686 INFO:teuthology.orchestra.run.smithi037.stdout:(28/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 297 kB/s | 171 kB 00:00 2024-03-21T08:57:01.728 INFO:teuthology.orchestra.run.smithi037.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2231.g 457 kB/s | 34 kB 00:00 2024-03-21T08:57:01.770 INFO:teuthology.orchestra.run.smithi037.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.6 MB/s | 140 kB 00:00 2024-03-21T08:57:02.100 INFO:teuthology.orchestra.run.smithi053.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 329 kB/s | 1.2 MB 00:03 2024-03-21T08:57:02.269 INFO:teuthology.orchestra.run.smithi071.stdout:(5/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 2.1 MB/s | 1.5 MB 00:00 2024-03-21T08:57:02.420 INFO:teuthology.orchestra.run.smithi037.stdout:(31/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 13 MB/s | 48 MB 00:03 2024-03-21T08:57:02.452 INFO:teuthology.orchestra.run.smithi071.stdout:(6/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-03-21T08:57:02.496 INFO:teuthology.orchestra.run.smithi037.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 2.3 MB/s | 1.8 MB 00:00 2024-03-21T08:57:02.529 INFO:teuthology.orchestra.run.smithi037.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.3 MB/s | 248 kB 00:00 2024-03-21T08:57:02.554 INFO:teuthology.orchestra.run.smithi037.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 857 kB/s | 49 kB 00:00 2024-03-21T08:57:02.588 INFO:teuthology.orchestra.run.smithi037.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 284 kB/s | 16 kB 00:00 2024-03-21T08:57:02.638 INFO:teuthology.orchestra.run.smithi037.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 3.1 MB/s | 264 kB 00:00 2024-03-21T08:57:02.679 INFO:teuthology.orchestra.run.smithi071.stdout:(7/174): ceph-common-19.0.0-2231.g2edfee68.el9. 5.4 MB/s | 21 MB 00:03 2024-03-21T08:57:02.688 INFO:teuthology.orchestra.run.smithi037.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 372 kB/s | 18 kB 00:00 2024-03-21T08:57:02.730 INFO:teuthology.orchestra.run.smithi037.stdout:(38/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 5.2 MB/s | 750 kB 00:00 2024-03-21T08:57:02.956 INFO:teuthology.orchestra.run.smithi037.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-21T08:57:03.006 INFO:teuthology.orchestra.run.smithi037.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2024-03-21T08:57:03.048 INFO:teuthology.orchestra.run.smithi037.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2024-03-21T08:57:03.081 INFO:teuthology.orchestra.run.smithi037.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 307 kB 00:00 2024-03-21T08:57:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 576 kB/s | 32 kB 00:00 2024-03-21T08:57:03.207 INFO:teuthology.orchestra.run.smithi037.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-21T08:57:03.224 INFO:teuthology.orchestra.run.smithi037.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 170 kB/s | 17 kB 00:00 2024-03-21T08:57:03.257 INFO:teuthology.orchestra.run.smithi037.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 314 kB/s | 10 kB 00:00 2024-03-21T08:57:03.282 INFO:teuthology.orchestra.run.smithi037.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-21T08:57:03.316 INFO:teuthology.orchestra.run.smithi037.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T08:57:03.350 INFO:teuthology.orchestra.run.smithi037.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-21T08:57:03.400 INFO:teuthology.orchestra.run.smithi037.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.0 MB/s | 578 kB 00:00 2024-03-21T08:57:03.411 INFO:teuthology.orchestra.run.smithi071.stdout:(8/174): ceph-radosgw-19.0.0-2231.g2edfee68.el9 13 MB/s | 9.9 MB 00:00 2024-03-21T08:57:03.417 INFO:teuthology.orchestra.run.smithi037.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 760 kB/s | 46 kB 00:00 2024-03-21T08:57:03.451 INFO:teuthology.orchestra.run.smithi037.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-03-21T08:57:03.453 INFO:teuthology.orchestra.run.smithi071.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 607 kB/s | 25 kB 00:00 2024-03-21T08:57:03.484 INFO:teuthology.orchestra.run.smithi037.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-21T08:57:03.518 INFO:teuthology.orchestra.run.smithi037.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-21T08:57:03.543 INFO:teuthology.orchestra.run.smithi037.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 613 kB/s | 34 kB 00:00 2024-03-21T08:57:03.617 INFO:teuthology.orchestra.run.smithi053.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 307 kB/s | 465 kB 00:01 2024-03-21T08:57:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-21T08:57:03.695 INFO:teuthology.orchestra.run.smithi071.stdout:(10/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x8 3.3 MB/s | 4.7 MB 00:01 2024-03-21T08:57:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 8.2 MB/s | 1.5 MB 00:00 2024-03-21T08:57:03.779 INFO:teuthology.orchestra.run.smithi071.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 2.6 MB/s | 220 kB 00:00 2024-03-21T08:57:03.786 INFO:teuthology.orchestra.run.smithi037.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2024-03-21T08:57:03.811 INFO:teuthology.orchestra.run.smithi037.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-21T08:57:03.817 INFO:teuthology.orchestra.run.smithi053.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 156 kB/s | 31 kB 00:00 2024-03-21T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 464 kB/s | 27 kB 00:00 2024-03-21T08:57:03.888 INFO:teuthology.orchestra.run.smithi071.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 628 kB/s | 31 kB 00:00 2024-03-21T08:57:03.978 INFO:teuthology.orchestra.run.smithi037.stdout:(60/174): ceph-mgr-diskprediction-local-19.0.0- 3.3 MB/s | 7.4 MB 00:02 2024-03-21T08:57:04.080 INFO:teuthology.orchestra.run.smithi071.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 3.8 MB/s | 747 kB 00:00 2024-03-21T08:57:04.095 INFO:teuthology.orchestra.run.smithi037.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 19 MB/s | 5.7 MB 00:00 2024-03-21T08:57:04.137 INFO:teuthology.orchestra.run.smithi037.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-03-21T08:57:04.154 INFO:teuthology.orchestra.run.smithi037.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 471 kB/s | 78 kB 00:00 2024-03-21T08:57:04.155 INFO:teuthology.orchestra.run.smithi071.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.1 MB/s | 163 kB 00:00 2024-03-21T08:57:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 272 kB/s | 24 kB 00:00 2024-03-21T08:57:04.230 INFO:teuthology.orchestra.run.smithi071.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.7 MB/s | 127 kB 00:00 2024-03-21T08:57:04.355 INFO:teuthology.orchestra.run.smithi071.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 3.9 MB/s | 501 kB 00:00 2024-03-21T08:57:04.363 INFO:teuthology.orchestra.run.smithi037.stdout:(65/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-21T08:57:04.388 INFO:teuthology.orchestra.run.smithi037.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 80 kB/s | 18 kB 00:00 2024-03-21T08:57:04.413 INFO:teuthology.orchestra.run.smithi037.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 864 kB/s | 144 kB 00:00 2024-03-21T08:57:04.455 INFO:teuthology.orchestra.run.smithi037.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 490 kB/s | 32 kB 00:00 2024-03-21T08:57:04.656 INFO:teuthology.orchestra.run.smithi037.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-03-21T08:57:04.680 INFO:teuthology.orchestra.run.smithi037.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 867 kB/s | 217 kB 00:00 2024-03-21T08:57:04.706 INFO:teuthology.orchestra.run.smithi037.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 743 kB/s | 174 kB 00:00 2024-03-21T08:57:04.773 INFO:teuthology.orchestra.run.smithi037.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-21T08:57:04.798 INFO:teuthology.orchestra.run.smithi037.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 456 kB/s | 41 kB 00:00 2024-03-21T08:57:04.815 INFO:teuthology.orchestra.run.smithi037.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 238 kB 00:00 2024-03-21T08:57:04.857 INFO:teuthology.orchestra.run.smithi037.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2024-03-21T08:57:04.882 INFO:teuthology.orchestra.run.smithi037.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 211 kB 00:00 2024-03-21T08:57:04.907 INFO:teuthology.orchestra.run.smithi037.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 524 kB/s | 47 kB 00:00 2024-03-21T08:57:04.940 INFO:teuthology.orchestra.run.smithi037.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-21T08:57:04.999 INFO:teuthology.orchestra.run.smithi037.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 428 kB/s | 39 kB 00:00 2024-03-21T08:57:05.345 INFO:teuthology.orchestra.run.smithi037.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-03-21T08:57:05.372 INFO:teuthology.orchestra.run.smithi071.stdout:(18/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 5.1 MB/s | 5.1 MB 00:01 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi037.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 98 kB/s | 41 kB 00:00 2024-03-21T08:57:05.400 INFO:teuthology.orchestra.run.smithi037.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 780 kB/s | 300 kB 00:00 2024-03-21T08:57:05.430 INFO:teuthology.orchestra.run.smithi071.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 777 kB/s | 45 kB 00:00 2024-03-21T08:57:05.441 INFO:teuthology.orchestra.run.smithi037.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-21T08:57:05.467 INFO:teuthology.orchestra.run.smithi037.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 963 kB/s | 63 kB 00:00 2024-03-21T08:57:05.489 INFO:teuthology.orchestra.run.smithi071.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 2.2 MB/s | 131 kB 00:00 2024-03-21T08:57:05.491 INFO:teuthology.orchestra.run.smithi037.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 490 kB/s | 44 kB 00:00 2024-03-21T08:57:05.534 INFO:teuthology.orchestra.run.smithi037.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-03-21T08:57:05.573 INFO:teuthology.orchestra.run.smithi071.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.9 MB/s | 161 kB 00:00 2024-03-21T08:57:05.576 INFO:teuthology.orchestra.run.smithi037.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-03-21T08:57:05.617 INFO:teuthology.orchestra.run.smithi037.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.1 MB 00:00 2024-03-21T08:57:05.643 INFO:teuthology.orchestra.run.smithi037.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-21T08:57:05.676 INFO:teuthology.orchestra.run.smithi037.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-21T08:57:05.690 INFO:teuthology.orchestra.run.smithi071.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 2.7 MB/s | 320 kB 00:00 2024-03-21T08:57:05.702 INFO:teuthology.orchestra.run.smithi037.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-21T08:57:05.782 INFO:teuthology.orchestra.run.smithi071.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 3.2 MB/s | 297 kB 00:00 2024-03-21T08:57:05.836 INFO:teuthology.orchestra.run.smithi037.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 992 kB/s | 187 kB 00:00 2024-03-21T08:57:05.840 INFO:teuthology.orchestra.run.smithi071.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.7 MB/s | 99 kB 00:00 2024-03-21T08:57:05.861 INFO:teuthology.orchestra.run.smithi037.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-21T08:57:05.899 INFO:teuthology.orchestra.run.smithi071.stdout:(25/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-21T08:57:05.923 INFO:teuthology.orchestra.run.smithi037.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.4 MB/s | 125 kB 00:00 2024-03-21T08:57:05.946 INFO:teuthology.orchestra.run.smithi037.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 260 kB/s | 21 kB 00:00 2024-03-21T08:57:05.970 INFO:teuthology.orchestra.run.smithi037.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 553 kB/s | 25 kB 00:00 2024-03-21T08:57:06.012 INFO:teuthology.orchestra.run.smithi037.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-03-21T08:57:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-03-21T08:57:06.079 INFO:teuthology.orchestra.run.smithi037.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-03-21T08:57:06.112 INFO:teuthology.orchestra.run.smithi037.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-21T08:57:06.138 INFO:teuthology.orchestra.run.smithi037.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 604 kB/s | 35 kB 00:00 2024-03-21T08:57:06.163 INFO:teuthology.orchestra.run.smithi037.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 897 kB/s | 44 kB 00:00 2024-03-21T08:57:06.188 INFO:teuthology.orchestra.run.smithi037.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-03-21T08:57:06.322 INFO:teuthology.orchestra.run.smithi037.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-03-21T08:57:06.347 INFO:teuthology.orchestra.run.smithi037.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-03-21T08:57:06.448 INFO:teuthology.orchestra.run.smithi037.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-03-21T08:57:06.515 INFO:teuthology.orchestra.run.smithi037.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-03-21T08:57:06.540 INFO:teuthology.orchestra.run.smithi037.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 213 kB/s | 19 kB 00:00 2024-03-21T08:57:06.582 INFO:teuthology.orchestra.run.smithi037.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 638 kB/s | 26 kB 00:00 2024-03-21T08:57:06.624 INFO:teuthology.orchestra.run.smithi071.stdout:(26/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 15 MB/s | 48 MB 00:03 2024-03-21T08:57:06.632 INFO:teuthology.orchestra.run.smithi037.stdout:(110/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-03-21T08:57:06.657 INFO:teuthology.orchestra.run.smithi037.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 652 kB/s | 49 kB 00:00 2024-03-21T08:57:06.691 INFO:teuthology.orchestra.run.smithi071.stdout:(27/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 3.9 MB/s | 3.1 MB 00:00 2024-03-21T08:57:06.699 INFO:teuthology.orchestra.run.smithi037.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 3.7 MB/s | 151 kB 00:00 2024-03-21T08:57:06.716 INFO:teuthology.orchestra.run.smithi037.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 813 kB/s | 67 kB 00:00 2024-03-21T08:57:06.716 INFO:teuthology.orchestra.run.smithi071.stdout:(28/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 1.8 MB/s | 171 kB 00:00 2024-03-21T08:57:06.741 INFO:teuthology.orchestra.run.smithi071.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2231.g 678 kB/s | 34 kB 00:00 2024-03-21T08:57:06.775 INFO:teuthology.orchestra.run.smithi037.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-03-21T08:57:06.817 INFO:teuthology.orchestra.run.smithi037.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-21T08:57:06.850 INFO:teuthology.orchestra.run.smithi037.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-03-21T08:57:06.875 INFO:teuthology.orchestra.run.smithi037.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 557 kB/s | 32 kB 00:00 2024-03-21T08:57:06.900 INFO:teuthology.orchestra.run.smithi071.stdout:(30/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 3.8 MB/s | 17 MB 00:04 2024-03-21T08:57:06.901 INFO:teuthology.orchestra.run.smithi037.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-03-21T08:57:06.925 INFO:teuthology.orchestra.run.smithi071.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 671 kB/s | 140 kB 00:00 2024-03-21T08:57:06.934 INFO:teuthology.orchestra.run.smithi037.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-21T08:57:06.976 INFO:teuthology.orchestra.run.smithi071.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 4.9 MB/s | 248 kB 00:00 2024-03-21T08:57:06.976 INFO:teuthology.orchestra.run.smithi037.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-21T08:57:07.010 INFO:teuthology.orchestra.run.smithi037.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-21T08:57:07.018 INFO:teuthology.orchestra.run.smithi071.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 1.2 MB/s | 49 kB 00:00 2024-03-21T08:57:07.035 INFO:teuthology.orchestra.run.smithi037.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 611 kB/s | 35 kB 00:00 2024-03-21T08:57:07.060 INFO:teuthology.orchestra.run.smithi071.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 395 kB/s | 16 kB 00:00 2024-03-21T08:57:07.126 INFO:teuthology.orchestra.run.smithi071.stdout:(35/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 4.7 MB/s | 1.8 MB 00:00 2024-03-21T08:57:07.135 INFO:teuthology.orchestra.run.smithi037.stdout:(123/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-03-21T08:57:07.152 INFO:teuthology.orchestra.run.smithi071.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.8 MB/s | 264 kB 00:00 2024-03-21T08:57:07.169 INFO:teuthology.orchestra.run.smithi037.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.3 MB/s | 203 kB 00:00 2024-03-21T08:57:07.194 INFO:teuthology.orchestra.run.smithi071.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 437 kB/s | 18 kB 00:00 2024-03-21T08:57:07.203 INFO:teuthology.orchestra.run.smithi037.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-21T08:57:07.228 INFO:teuthology.orchestra.run.smithi037.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-03-21T08:57:07.253 INFO:teuthology.orchestra.run.smithi037.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2024-03-21T08:57:07.261 INFO:teuthology.orchestra.run.smithi071.stdout:(38/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 5.5 MB/s | 750 kB 00:00 2024-03-21T08:57:07.279 INFO:teuthology.orchestra.run.smithi037.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 469 kB/s | 23 kB 00:00 2024-03-21T08:57:07.304 INFO:teuthology.orchestra.run.smithi037.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-03-21T08:57:07.329 INFO:teuthology.orchestra.run.smithi037.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 398 kB/s | 20 kB 00:00 2024-03-21T08:57:07.371 INFO:teuthology.orchestra.run.smithi037.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-21T08:57:07.421 INFO:teuthology.orchestra.run.smithi037.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-03-21T08:57:07.447 INFO:teuthology.orchestra.run.smithi037.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-21T08:57:07.472 INFO:teuthology.orchestra.run.smithi037.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-21T08:57:07.506 INFO:teuthology.orchestra.run.smithi037.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-03-21T08:57:07.517 INFO:teuthology.orchestra.run.smithi053.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 297 kB/s | 1.1 MB 00:03 2024-03-21T08:57:07.531 INFO:teuthology.orchestra.run.smithi037.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-03-21T08:57:07.544 INFO:teuthology.orchestra.run.smithi071.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-03-21T08:57:07.556 INFO:teuthology.orchestra.run.smithi037.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-21T08:57:07.581 INFO:teuthology.orchestra.run.smithi037.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 572 kB/s | 28 kB 00:00 2024-03-21T08:57:07.607 INFO:teuthology.orchestra.run.smithi037.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 623 kB/s | 31 kB 00:00 2024-03-21T08:57:07.653 INFO:teuthology.orchestra.run.smithi071.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2024-03-21T08:57:07.674 INFO:teuthology.orchestra.run.smithi037.stdout:(140/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-03-21T08:57:07.699 INFO:teuthology.orchestra.run.smithi037.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 648 kB/s | 59 kB 00:00 2024-03-21T08:57:07.703 INFO:teuthology.orchestra.run.smithi071.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 668 kB/s | 106 kB 00:00 2024-03-21T08:57:07.724 INFO:teuthology.orchestra.run.smithi037.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 724 kB/s | 36 kB 00:00 2024-03-21T08:57:07.725 INFO:teuthology.orchestra.run.smithi053.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 357 kB/s | 74 kB 00:00 2024-03-21T08:57:07.750 INFO:teuthology.orchestra.run.smithi037.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-03-21T08:57:07.787 INFO:teuthology.orchestra.run.smithi071.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 307 kB 00:00 2024-03-21T08:57:07.812 INFO:teuthology.orchestra.run.smithi071.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 292 kB/s | 32 kB 00:00 2024-03-21T08:57:07.891 INFO:teuthology.orchestra.run.smithi037.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 266 kB/s | 51 kB 00:00 2024-03-21T08:57:07.917 INFO:teuthology.orchestra.run.smithi037.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 541 kB/s | 90 kB 00:00 2024-03-21T08:57:07.942 INFO:teuthology.orchestra.run.smithi037.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-03-21T08:57:07.962 INFO:teuthology.orchestra.run.smithi071.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 114 kB/s | 17 kB 00:00 2024-03-21T08:57:07.967 INFO:teuthology.orchestra.run.smithi037.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 405 kB/s | 20 kB 00:00 2024-03-21T08:57:08.001 INFO:teuthology.orchestra.run.smithi037.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-03-21T08:57:08.034 INFO:teuthology.orchestra.run.smithi037.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-03-21T08:57:08.066 INFO:teuthology.orchestra.run.smithi071.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-03-21T08:57:08.091 INFO:teuthology.orchestra.run.smithi071.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 79 kB/s | 10 kB 00:00 2024-03-21T08:57:08.101 INFO:teuthology.orchestra.run.smithi037.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-21T08:57:08.141 INFO:teuthology.orchestra.run.smithi071.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-21T08:57:08.160 INFO:teuthology.orchestra.run.smithi037.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2024-03-21T08:57:08.167 INFO:teuthology.orchestra.run.smithi071.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-21T08:57:08.226 INFO:teuthology.orchestra.run.smithi071.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-21T08:57:08.251 INFO:teuthology.orchestra.run.smithi071.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-21T08:57:08.276 INFO:teuthology.orchestra.run.smithi071.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 46 kB 00:00 2024-03-21T08:57:08.318 INFO:teuthology.orchestra.run.smithi071.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-03-21T08:57:08.343 INFO:teuthology.orchestra.run.smithi071.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-03-21T08:57:08.385 INFO:teuthology.orchestra.run.smithi071.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-21T08:57:08.409 INFO:teuthology.orchestra.run.smithi053.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 293 kB/s | 2.2 MB 00:07 2024-03-21T08:57:08.419 INFO:teuthology.orchestra.run.smithi071.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 454 kB/s | 34 kB 00:00 2024-03-21T08:57:08.502 INFO:teuthology.orchestra.run.smithi037.stdout:(152/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-03-21T08:57:08.617 INFO:teuthology.orchestra.run.smithi053.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 334 kB/s | 2.3 MB 00:06 2024-03-21T08:57:08.622 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:57:08.622 INFO:teuthology.orchestra.run.smithi053.stdout:Total 8.7 MB/s | 224 MB 00:25 2024-03-21T08:57:08.636 INFO:teuthology.orchestra.run.smithi071.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-03-21T08:57:08.678 INFO:teuthology.orchestra.run.smithi071.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 5.9 MB/s | 1.5 MB 00:00 2024-03-21T08:57:08.694 INFO:teuthology.orchestra.run.smithi037.stdout:(153/174): librados2-19.0.0-2231.g2edfee68.el9. 4.9 MB/s | 3.4 MB 00:00 2024-03-21T08:57:08.728 INFO:teuthology.orchestra.run.smithi071.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2024-03-21T08:57:08.753 INFO:teuthology.orchestra.run.smithi071.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.7 MB/s | 134 kB 00:00 2024-03-21T08:57:08.769 INFO:teuthology.orchestra.run.smithi037.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 6.9 MB/s | 1.8 MB 00:00 2024-03-21T08:57:08.861 INFO:teuthology.orchestra.run.smithi037.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.7 MB/s | 248 kB 00:00 2024-03-21T08:57:08.886 INFO:teuthology.orchestra.run.smithi037.stdout:(156/174): NetworkManager-team-1.47.2-1.el9.x86 220 kB/s | 42 kB 00:00 2024-03-21T08:57:08.945 INFO:teuthology.orchestra.run.smithi037.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 857 kB/s | 71 kB 00:00 2024-03-21T08:57:08.962 INFO:teuthology.orchestra.run.smithi037.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 488 kB/s | 36 kB 00:00 2024-03-21T08:57:09.003 INFO:teuthology.orchestra.run.smithi071.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-21T08:57:09.015 INFO:teuthology.orchestra.run.smithi037.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.7 MB/s | 192 kB 00:00 2024-03-21T08:57:09.037 INFO:teuthology.orchestra.run.smithi071.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-03-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi071.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-21T08:57:09.082 INFO:teuthology.orchestra.run.smithi037.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 350 kB/s | 23 kB 00:00 2024-03-21T08:57:09.108 INFO:teuthology.orchestra.run.smithi037.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.7 MB/s | 256 kB 00:00 2024-03-21T08:57:09.109 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-03-21T08:57:09.166 INFO:teuthology.orchestra.run.smithi037.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-03-21T08:57:09.192 INFO:teuthology.orchestra.run.smithi037.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-21T08:57:09.221 INFO:teuthology.orchestra.run.smithi071.stdout:(63/174): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-03-21T08:57:09.251 INFO:teuthology.orchestra.run.smithi037.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-21T08:57:09.284 INFO:teuthology.orchestra.run.smithi037.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 699 kB/s | 64 kB 00:00 2024-03-21T08:57:09.309 INFO:teuthology.orchestra.run.smithi037.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 495 kB/s | 29 kB 00:00 2024-03-21T08:57:09.321 INFO:teuthology.orchestra.run.smithi071.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-21T08:57:09.346 INFO:teuthology.orchestra.run.smithi071.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 89 kB/s | 24 kB 00:00 2024-03-21T08:57:09.380 INFO:teuthology.orchestra.run.smithi071.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 115 kB/s | 18 kB 00:00 2024-03-21T08:57:09.405 INFO:teuthology.orchestra.run.smithi071.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-21T08:57:09.409 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-03-21T08:57:09.409 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-03-21T08:57:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:(167/174): NetworkManager-1.47.2-1.el9.x86_64.r 1.8 MB/s | 2.3 MB 00:01 2024-03-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi071.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-03-21T08:57:09.485 INFO:teuthology.orchestra.run.smithi037.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.1 MB/s | 1.2 MB 00:00 2024-03-21T08:57:09.552 INFO:teuthology.orchestra.run.smithi037.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.9 MB/s | 2.2 MB 00:00 2024-03-21T08:57:09.594 INFO:teuthology.orchestra.run.smithi037.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-03-21T08:57:09.597 INFO:teuthology.orchestra.run.smithi071.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-03-21T08:57:09.619 INFO:teuthology.orchestra.run.smithi037.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 472 kB/s | 31 kB 00:00 2024-03-21T08:57:09.623 INFO:teuthology.orchestra.run.smithi071.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-21T08:57:09.648 INFO:teuthology.orchestra.run.smithi071.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 777 kB/s | 174 kB 00:00 2024-03-21T08:57:09.673 INFO:teuthology.orchestra.run.smithi071.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 918 kB/s | 69 kB 00:00 2024-03-21T08:57:09.678 INFO:teuthology.orchestra.run.smithi037.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-03-21T08:57:09.698 INFO:teuthology.orchestra.run.smithi071.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 825 kB/s | 41 kB 00:00 2024-03-21T08:57:09.703 INFO:teuthology.orchestra.run.smithi037.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 899 kB/s | 74 kB 00:00 2024-03-21T08:57:09.723 INFO:teuthology.orchestra.run.smithi071.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-21T08:57:09.749 INFO:teuthology.orchestra.run.smithi071.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-21T08:57:09.774 INFO:teuthology.orchestra.run.smithi071.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-21T08:57:09.799 INFO:teuthology.orchestra.run.smithi071.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-21T08:57:09.826 INFO:teuthology.orchestra.run.smithi071.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 568 kB/s | 43 kB 00:00 2024-03-21T08:57:09.920 INFO:teuthology.orchestra.run.smithi037.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 2.3 MB 00:00 2024-03-21T08:57:09.929 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:57:09.929 INFO:teuthology.orchestra.run.smithi037.stdout:Total 16 MB/s | 224 MB 00:14 2024-03-21T08:57:09.991 INFO:teuthology.orchestra.run.smithi071.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 235 kB/s | 39 kB 00:00 2024-03-21T08:57:10.017 INFO:teuthology.orchestra.run.smithi071.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 216 kB/s | 41 kB 00:00 2024-03-21T08:57:10.217 INFO:teuthology.orchestra.run.smithi071.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-21T08:57:10.242 INFO:teuthology.orchestra.run.smithi071.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-21T08:57:10.267 INFO:teuthology.orchestra.run.smithi071.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 227 kB/s | 56 kB 00:00 2024-03-21T08:57:10.293 INFO:teuthology.orchestra.run.smithi071.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 841 kB/s | 63 kB 00:00 2024-03-21T08:57:10.318 INFO:teuthology.orchestra.run.smithi071.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-21T08:57:10.343 INFO:teuthology.orchestra.run.smithi071.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-21T08:57:10.368 INFO:teuthology.orchestra.run.smithi071.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-21T08:57:10.394 INFO:teuthology.orchestra.run.smithi071.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-21T08:57:10.436 INFO:teuthology.orchestra.run.smithi071.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-21T08:57:10.461 INFO:teuthology.orchestra.run.smithi071.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-21T08:57:10.486 INFO:teuthology.orchestra.run.smithi071.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-21T08:57:10.686 INFO:teuthology.orchestra.run.smithi071.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 834 kB/s | 187 kB 00:00 2024-03-21T08:57:10.711 INFO:teuthology.orchestra.run.smithi071.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 868 kB/s | 195 kB 00:00 2024-03-21T08:57:10.745 INFO:teuthology.orchestra.run.smithi071.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-21T08:57:10.770 INFO:teuthology.orchestra.run.smithi071.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-03-21T08:57:10.812 INFO:teuthology.orchestra.run.smithi071.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-03-21T08:57:10.837 INFO:teuthology.orchestra.run.smithi071.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-03-21T08:57:10.938 INFO:teuthology.orchestra.run.smithi071.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-03-21T08:57:10.963 INFO:teuthology.orchestra.run.smithi071.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-21T08:57:11.005 INFO:teuthology.orchestra.run.smithi071.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-21T08:57:11.030 INFO:teuthology.orchestra.run.smithi071.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 519 kB/s | 35 kB 00:00 2024-03-21T08:57:11.081 INFO:teuthology.orchestra.run.smithi071.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-21T08:57:11.115 INFO:teuthology.orchestra.run.smithi071.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-21T08:57:11.257 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-03-21T08:57:11.257 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-03-21T08:57:11.292 INFO:teuthology.orchestra.run.smithi071.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 106 kB/s | 22 kB 00:00 2024-03-21T08:57:11.415 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-21T08:57:11.415 INFO:teuthology.orchestra.run.smithi071.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 6.4 MB/s | 4.1 MB 00:00 2024-03-21T08:57:11.449 INFO:teuthology.orchestra.run.smithi071.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-21T08:57:11.699 INFO:teuthology.orchestra.run.smithi071.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-03-21T08:57:11.814 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-21T08:57:11.814 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-21T08:57:11.875 INFO:teuthology.orchestra.run.smithi071.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 673 kB/s | 309 kB 00:00 2024-03-21T08:57:11.900 INFO:teuthology.orchestra.run.smithi071.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 902 kB/s | 548 kB 00:00 2024-03-21T08:57:11.975 INFO:teuthology.orchestra.run.smithi071.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-03-21T08:57:12.000 INFO:teuthology.orchestra.run.smithi071.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-03-21T08:57:12.085 INFO:teuthology.orchestra.run.smithi071.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-03-21T08:57:12.151 INFO:teuthology.orchestra.run.smithi071.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-21T08:57:12.301 INFO:teuthology.orchestra.run.smithi071.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-03-21T08:57:12.335 INFO:teuthology.orchestra.run.smithi071.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-03-21T08:57:12.401 INFO:teuthology.orchestra.run.smithi071.stdout:(116/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-03-21T08:57:12.427 INFO:teuthology.orchestra.run.smithi071.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-03-21T08:57:12.452 INFO:teuthology.orchestra.run.smithi071.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-03-21T08:57:12.510 INFO:teuthology.orchestra.run.smithi071.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-21T08:57:12.561 INFO:teuthology.orchestra.run.smithi071.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-21T08:57:12.586 INFO:teuthology.orchestra.run.smithi071.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-21T08:57:12.636 INFO:teuthology.orchestra.run.smithi071.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-21T08:57:12.661 INFO:teuthology.orchestra.run.smithi071.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-03-21T08:57:12.704 INFO:teuthology.orchestra.run.smithi071.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-03-21T08:57:12.821 INFO:teuthology.orchestra.run.smithi071.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 910 kB/s | 144 kB 00:00 2024-03-21T08:57:12.846 INFO:teuthology.orchestra.run.smithi071.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-03-21T08:57:12.888 INFO:teuthology.orchestra.run.smithi071.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-03-21T08:57:12.913 INFO:teuthology.orchestra.run.smithi071.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-03-21T08:57:12.947 INFO:teuthology.orchestra.run.smithi071.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-03-21T08:57:12.980 INFO:teuthology.orchestra.run.smithi071.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 210 kB/s | 19 kB 00:00 2024-03-21T08:57:13.005 INFO:teuthology.orchestra.run.smithi071.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-03-21T08:57:13.097 INFO:teuthology.orchestra.run.smithi071.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-03-21T08:57:13.123 INFO:teuthology.orchestra.run.smithi071.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-03-21T08:57:13.165 INFO:teuthology.orchestra.run.smithi071.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-03-21T08:57:13.207 INFO:teuthology.orchestra.run.smithi071.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 528 kB/s | 58 kB 00:00 2024-03-21T08:57:13.265 INFO:teuthology.orchestra.run.smithi071.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-03-21T08:57:13.290 INFO:teuthology.orchestra.run.smithi071.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-03-21T08:57:13.316 INFO:teuthology.orchestra.run.smithi071.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-03-21T08:57:13.374 INFO:teuthology.orchestra.run.smithi071.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-03-21T08:57:13.425 INFO:teuthology.orchestra.run.smithi071.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-21T08:57:13.450 INFO:teuthology.orchestra.run.smithi071.stdout:(141/174): python3-repoze-lru-0.7-16.el9.noarch 193 kB/s | 31 kB 00:00 2024-03-21T08:57:13.492 INFO:teuthology.orchestra.run.smithi071.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-03-21T08:57:13.517 INFO:teuthology.orchestra.run.smithi071.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-03-21T08:57:13.559 INFO:teuthology.orchestra.run.smithi071.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-03-21T08:57:13.618 INFO:teuthology.orchestra.run.smithi071.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-21T08:57:13.643 INFO:teuthology.orchestra.run.smithi071.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-03-21T08:57:13.685 INFO:teuthology.orchestra.run.smithi071.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-21T08:57:13.710 INFO:teuthology.orchestra.run.smithi071.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-03-21T08:57:13.743 INFO:teuthology.orchestra.run.smithi071.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-21T08:57:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-03-21T08:57:13.977 INFO:teuthology.orchestra.run.smithi071.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 321 kB 00:00 2024-03-21T08:57:14.144 INFO:teuthology.orchestra.run.smithi071.stdout:(152/174): librados2-19.0.0-2231.g2edfee68.el9. 7.8 MB/s | 3.4 MB 00:00 2024-03-21T08:57:14.280 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-21T08:57:14.280 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-21T08:57:14.328 INFO:teuthology.orchestra.run.smithi071.stdout:(153/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-03-21T08:57:14.403 INFO:teuthology.orchestra.run.smithi071.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2024-03-21T08:57:14.462 INFO:teuthology.orchestra.run.smithi071.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-21T08:57:14.487 INFO:teuthology.orchestra.run.smithi071.stdout:(156/174): NetworkManager-team-1.47.2-1.el9.x86 265 kB/s | 42 kB 00:00 2024-03-21T08:57:14.546 INFO:teuthology.orchestra.run.smithi071.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 624 kB/s | 36 kB 00:00 2024-03-21T08:57:14.587 INFO:teuthology.orchestra.run.smithi071.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 569 kB/s | 71 kB 00:00 2024-03-21T08:57:14.613 INFO:teuthology.orchestra.run.smithi071.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-21T08:57:14.647 INFO:teuthology.orchestra.run.smithi071.stdout:(160/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.7 MB/s | 1.8 MB 00:00 2024-03-21T08:57:14.672 INFO:teuthology.orchestra.run.smithi071.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 392 kB/s | 23 kB 00:00 2024-03-21T08:57:14.705 INFO:teuthology.orchestra.run.smithi071.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-21T08:57:14.730 INFO:teuthology.orchestra.run.smithi071.stdout:(163/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.7 MB/s | 256 kB 00:00 2024-03-21T08:57:14.756 INFO:teuthology.orchestra.run.smithi071.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-21T08:57:14.781 INFO:teuthology.orchestra.run.smithi071.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-21T08:57:14.806 INFO:teuthology.orchestra.run.smithi071.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 842 kB/s | 64 kB 00:00 2024-03-21T08:57:14.831 INFO:teuthology.orchestra.run.smithi071.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 378 kB/s | 29 kB 00:00 2024-03-21T08:57:14.907 INFO:teuthology.orchestra.run.smithi071.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-03-21T08:57:14.982 INFO:teuthology.orchestra.run.smithi071.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-03-21T08:57:15.024 INFO:teuthology.orchestra.run.smithi071.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-03-21T08:57:15.049 INFO:teuthology.orchestra.run.smithi071.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 464 kB/s | 31 kB 00:00 2024-03-21T08:57:15.091 INFO:teuthology.orchestra.run.smithi071.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 2.2 MB 00:00 2024-03-21T08:57:15.141 INFO:teuthology.orchestra.run.smithi071.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 809 kB/s | 74 kB 00:00 2024-03-21T08:57:15.175 INFO:teuthology.orchestra.run.smithi071.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-03-21T08:57:15.178 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:57:15.178 INFO:teuthology.orchestra.run.smithi071.stdout:Total 13 MB/s | 224 MB 00:16 2024-03-21T08:57:15.621 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-03-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-03-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-03-21T08:57:17.644 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-03-21T08:57:17.644 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-03-21T08:57:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-03-21T08:57:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:57:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:57:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:57:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:57:18.803 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:57:18.857 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:57:18.891 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:57:18.915 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:57:18.958 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:57:19.361 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:57:19.391 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:57:19.507 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:57:19.542 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:57:19.618 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:57:19.646 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:57:19.685 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:57:19.865 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:57:19.920 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:57:19.947 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:57:20.036 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:57:20.065 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:57:20.129 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:57:20.168 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:57:20.191 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:57:20.208 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:57:20.358 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:57:20.388 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:57:20.412 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:57:20.511 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:57:20.589 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:57:20.633 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:57:20.707 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:20.758 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:20.991 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:21.100 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:57:21.197 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:57:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:57:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:57:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:57:21.369 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:57:21.422 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:57:21.495 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:57:21.559 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:57:21.599 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:57:21.643 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:57:21.675 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:57:21.703 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:57:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:57:21.792 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-21T08:57:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:57:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:57:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:57:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:57:21.954 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:57:21.968 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:57:21.995 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:57:22.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:57:22.091 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:57:22.188 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:57:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:57:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:57:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:57:22.369 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:57:22.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:57:22.683 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:57:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:57:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:57:22.980 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:57:23.037 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:57:23.045 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:57:23.077 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:57:23.116 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:57:23.126 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:57:23.165 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:57:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:57:23.210 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:57:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:57:23.420 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:57:23.428 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:57:23.547 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:57:23.583 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:57:23.601 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:57:23.663 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:57:23.672 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:57:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:57:23.851 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:57:23.903 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:57:23.963 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:57:23.997 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:57:24.051 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:57:24.053 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:57:24.076 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:57:24.091 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:57:24.128 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:57:24.175 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:57:24.210 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:57:24.228 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:57:24.260 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:57:24.263 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:57:24.304 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:57:24.371 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:57:24.422 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:57:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:57:24.453 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:57:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:57:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:57:24.523 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:57:24.565 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:57:24.566 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-03-21T08:57:24.653 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:57:24.664 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:57:24.696 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:57:24.697 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:57:24.729 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:57:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:57:24.743 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:57:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:57:24.754 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:57:24.779 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:57:24.792 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:57:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:57:24.846 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:57:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:57:24.855 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:57:24.879 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:57:24.906 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:57:24.947 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:57:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:57:25.096 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:57:25.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:57:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:57:25.220 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:57:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:57:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:57:25.323 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:57:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:57:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:57:25.444 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:57:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:57:25.468 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:57:25.517 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:57:25.557 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:57:25.586 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:57:25.628 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:57:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:57:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:57:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:57:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:57:25.789 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:57:25.814 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:57:25.831 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:57:25.859 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:57:25.870 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:57:25.890 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:57:25.902 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:57:25.932 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:57:25.985 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:57:26.007 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:57:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:57:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:57:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:57:26.071 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:57:26.103 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:57:26.109 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:57:26.125 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:57:26.141 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:57:26.143 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:57:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:57:26.266 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:57:26.282 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:57:26.314 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:57:26.325 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:57:26.346 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:57:26.366 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:57:26.393 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:57:26.397 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:57:26.438 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:57:26.445 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:57:26.451 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:57:26.475 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:57:26.537 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:57:26.546 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:57:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:57:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:57:26.599 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:57:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:57:26.734 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:57:26.734 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:26.782 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:57:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:57:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:57:26.932 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:57:26.981 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:57:27.032 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:57:27.038 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:57:27.072 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:57:27.142 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:57:27.181 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:57:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:57:27.208 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:57:27.259 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:57:27.260 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:57:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:57:27.334 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:57:27.358 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:57:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:57:27.387 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:57:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:57:27.426 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:57:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:57:27.484 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:57:27.490 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:57:27.495 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:57:27.533 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:57:27.543 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:57:27.577 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:57:27.586 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:57:27.609 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:57:27.622 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:57:27.637 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:57:27.707 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:57:27.738 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:57:27.742 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:57:27.788 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:57:27.806 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:57:27.815 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:57:27.835 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:57:27.883 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:57:27.915 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:57:27.926 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:57:27.929 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:57:27.997 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:57:28.002 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:57:28.136 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:57:28.206 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:57:28.592 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:57:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:57:28.727 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:57:28.730 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:57:28.809 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:57:28.838 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:57:28.892 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:57:28.937 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:57:28.945 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:57:28.949 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:57:28.977 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:57:29.010 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:57:29.026 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:57:29.036 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:57:29.064 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:57:29.082 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:57:29.094 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:57:29.145 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:57:29.197 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:57:29.243 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:57:29.324 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:57:29.488 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:57:29.554 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:57:29.599 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:57:29.644 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:57:29.747 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:57:29.801 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:57:29.856 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:57:29.865 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:57:29.928 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:57:29.949 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:57:29.960 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:57:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:57:29.995 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:57:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:57:30.042 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:57:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:57:30.078 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:57:30.108 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:57:30.121 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:57:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:57:30.163 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:57:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:57:30.225 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:57:30.296 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:57:30.302 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:57:30.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:57:30.427 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:57:30.457 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:57:30.478 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:57:30.495 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:57:30.499 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:57:30.519 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:57:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:57:30.558 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:57:30.593 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:57:30.604 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:57:30.663 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:57:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:57:30.689 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:57:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:57:30.714 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:57:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:57:30.786 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:57:30.833 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:57:30.922 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:57:31.022 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:57:31.037 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:57:31.063 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:57:31.097 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:57:31.114 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:57:31.134 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:57:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:57:31.167 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:57:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:57:31.193 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:57:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:57:31.284 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:57:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:57:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:57:31.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:57:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:57:31.430 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:57:31.466 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:57:31.490 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:57:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:57:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:57:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:57:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:57:31.652 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:57:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:57:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:57:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:57:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:57:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:57:31.830 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:57:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:57:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:57:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:57:31.968 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:57:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:57:32.017 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:57:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:57:32.080 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:57:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:57:32.155 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:57:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:57:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:57:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:57:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:57:32.408 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:57:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:57:32.471 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:57:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:57:32.647 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:57:32.693 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:57:32.733 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:57:32.771 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:57:32.812 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:57:32.859 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:57:32.943 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:57:33.028 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:57:33.078 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:57:33.220 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:57:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:57:33.263 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:57:33.297 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:57:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:57:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:57:33.362 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:57:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:57:33.400 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:57:33.464 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:57:33.514 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:57:33.548 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:57:33.645 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:57:33.680 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:33.689 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:57:33.720 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:57:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:57:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:57:33.797 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:57:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:57:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:57:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:57:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:57:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:57:34.349 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:57:34.349 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:57:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:57:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:57:34.541 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:57:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:57:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:57:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:57:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:57:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:57:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:57:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:57:35.609 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:57:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:57:35.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:57:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:57:35.850 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:57:35.881 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:57:35.909 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:57:36.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:57:36.247 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:57:36.329 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:57:36.360 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:57:36.396 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:57:36.537 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:57:36.675 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:57:36.854 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:57:36.883 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:57:36.946 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:57:36.977 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:57:38.301 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:57:38.388 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:57:38.426 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:57:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:57:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:57:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:57:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:57:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:57:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:57:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:57:39.237 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:39.273 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:57:39.706 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:39.824 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:57:39.851 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:57:39.915 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:57:39.915 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:57:39.915 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:57:39.915 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:39.943 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:57:42.493 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:42.529 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:43.185 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:57:43.224 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:57:43.410 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:57:43.410 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:57:43.410 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:57:43.410 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:57:43.450 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:57:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:57:46.450 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-03-21T08:57:46.450 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-03-21T08:57:46.451 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-03-21T08:57:46.451 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-03-21T08:57:46.451 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-03-21T08:57:46.451 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-03-21T08:57:46.451 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-03-21T08:57:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:57:46.641 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:57:46.641 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:46.641 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:57:46.641 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:57:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:57:48.202 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:48.202 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:57:48.202 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:57:48.202 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:57:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:48.433 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:57:48.457 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:57:48.457 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:48.457 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:57:48.457 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:57:48.457 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:57:48.458 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:48.917 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:57:48.941 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:57:48.941 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:48.941 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:57:48.942 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:57:48.942 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:57:48.942 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:49.036 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:57:49.405 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:57:49.434 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:57:49.467 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:57:49.521 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:57:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:57:49.796 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:57:49.824 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:57:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:57:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:57:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:57:51.137 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:57:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:57:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:57:51.184 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:51.184 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:57:51.184 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:57:51.184 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:57:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:51.236 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:57:51.247 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:57:51.284 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:57:51.587 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:57:51.587 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-03-21T08:57:51.587 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-03-21T08:57:51.587 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-03-21T08:57:51.588 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-03-21T08:57:51.588 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-03-21T08:57:51.588 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-03-21T08:57:51.588 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-03-21T08:57:51.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:51.753 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:57:51.768 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:57:51.768 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:51.768 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:57:51.768 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:52.207 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:57:52.238 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:57:52.262 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:57:52.262 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:52.262 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:57:52.262 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:57:52.262 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:57:52.262 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:52.305 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:57:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:57:52.329 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:52.329 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:57:52.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:52.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:57:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:57:52.397 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:52.397 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:57:52.397 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:57:52.397 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:57:52.397 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:52.723 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:57:52.747 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:57:52.747 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:52.747 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:57:52.747 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:57:52.748 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:57:52.748 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:57:53.266 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:57:53.290 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:57:53.290 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:53.290 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:57:53.290 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:57:53.290 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:57:53.290 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:57:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:57:53.529 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:53.530 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:57:53.530 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:57:53.530 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:57:53.530 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:53.967 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:57:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:57:53.989 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:53.989 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:57:53.989 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:57:53.989 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:57:53.989 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:54.085 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:57:54.427 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:57:54.458 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:57:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:57:54.550 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:57:54.573 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:57:54.805 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:57:54.840 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:57:55.802 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:57:55.824 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:57:55.925 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:57:56.095 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:57:56.126 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:57:56.148 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:57:56.148 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:56.148 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:57:56.148 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:57:56.148 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:57:56.148 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:56.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:57:56.204 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:57:56.249 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:57:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:57:56.522 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-03-21T08:57:56.522 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-03-21T08:57:56.522 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-03-21T08:57:56.522 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-03-21T08:57:56.523 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-03-21T08:57:56.523 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-03-21T08:57:56.523 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-03-21T08:57:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:57:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:57:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:57:56.813 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:56.813 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:57:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:57:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:57:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:57:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:57:57.176 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:57.176 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:57:57.176 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:57:57.176 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:57:57.176 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:57.212 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:57:57.233 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:57:57.234 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:57.234 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:57:57.234 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:57.280 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:57:57.301 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:57:57.301 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:57.302 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:57:57.302 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:57:57.302 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:57:57.302 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:57:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:57:57.526 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:57:57.553 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:57:57.605 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:57:57.627 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:57:57.627 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:57.627 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:57:57.627 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:57:57.627 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:57:57.627 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:57:57.745 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:57:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:57:57.918 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:57:57.957 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:57:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:57:58.109 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:57:58.139 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:57:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:57:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:57:58.286 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:57:58.482 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:57:58.566 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:57:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:57:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:57:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:57:58.566 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:57:58.832 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:57:58.855 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:57:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:57:58.904 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:57:58.904 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:58.904 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:57:58.905 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:57:58.905 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:57:58.905 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:57:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:57:59.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:57:59.490 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:57:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:57:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:57:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:57:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:57:59.490 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:57:59.610 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:57:59.806 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:57:59.938 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:58:00.020 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:58:00.057 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:58:00.082 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:58:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:58:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:58:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:58:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:58:00.238 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:58:00.243 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:58:00.243 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:58:00.269 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:58:00.322 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:58:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:58:00.354 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:58:00.387 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:58:00.412 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:58:00.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:58:00.437 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:58:00.456 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:58:00.471 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:58:00.519 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:58:00.559 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:58:00.584 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:58:00.609 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:58:00.634 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:58:00.660 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:58:00.667 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:00.667 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:00.802 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:58:00.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:58:00.854 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:00.854 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:58:00.937 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:58:00.962 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:58:00.994 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:58:01.021 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:58:01.046 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:58:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:58:02.022 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:58:02.057 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:58:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:58:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:58:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:58:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:58:02.207 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:58:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:58:02.387 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:58:02.417 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:58:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:58:02.499 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:58:02.499 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:58:02.499 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:58:02.499 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:58:02.499 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:58:02.499 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:58:02.551 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:58:02.552 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:58:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:58:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:58:02.632 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:58:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:58:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:58:02.688 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:58:02.718 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:58:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:58:02.844 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:58:02.877 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:58:03.443 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:58:03.483 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:58:03.502 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:58:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:58:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:58:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:58:03.791 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:58:03.791 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:58:03.791 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:58:03.791 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:58:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:58:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:58:03.930 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:58:03.930 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:58:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:04.005 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:58:04.110 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:58:04.110 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:58:04.110 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:58:04.110 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:58:04.110 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:58:04.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:58:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:58:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:58:04.497 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:58:04.515 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:58:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:58:04.578 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:58:04.578 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:58:04.578 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:58:04.578 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:58:04.578 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:58:04.578 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:04.597 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:58:04.623 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:58:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:58:04.649 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:58:04.737 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:58:04.767 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:58:04.767 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:58:04.860 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:58:04.886 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:58:04.919 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:58:04.944 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:58:04.969 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:58:05.002 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:58:05.050 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:58:05.091 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:58:05.116 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:58:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:58:05.166 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:58:05.191 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:58:05.198 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:05.198 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:05.370 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:05.370 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:58:05.452 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:58:05.477 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:58:05.509 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:58:05.537 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:58:05.562 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:58:05.626 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:58:05.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:58:05.909 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:58:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:58:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:58:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:58:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:58:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:58:05.910 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:58:05.931 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:58:05.932 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:58:05.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:58:05.934 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:58:05.935 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:58:05.936 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:58:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:58:05.938 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:58:05.939 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:58:05.940 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:58:05.941 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:58:05.942 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:58:05.943 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:58:05.944 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:58:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:58:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:58:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:58:05.945 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:09.015 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.016 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.017 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:58:09.019 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:58:09.020 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:58:09.021 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:58:09.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:58:09.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:58:09.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:58:09.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.026 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:58:09.027 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:58:09.028 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:58:09.029 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-21T08:58:09.029 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-03-21T08:58:09.517 DEBUG:teuthology.parallel:result is None 2024-03-21T08:58:09.678 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:58:09.724 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:58:09.760 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:58:09.799 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:58:10.117 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:58:10.242 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:58:10.287 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:58:10.343 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:58:10.435 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:58:10.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:58:10.447 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:58:10.448 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:58:10.448 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:58:10.448 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:58:10.448 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:58:10.448 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:58:10.448 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:58:10.464 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:58:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:58:10.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:58:10.467 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:58:10.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:58:10.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:58:10.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:58:10.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:58:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:58:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:58:10.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:58:10.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:58:10.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:58:10.480 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:58:10.516 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:58:10.561 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:58:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:58:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:58:10.813 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:58:10.852 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:58:11.801 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:58:11.851 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:58:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:58:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:58:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:58:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:58:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:58:13.617 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.618 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.619 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.620 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:58:13.621 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:58:13.622 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:58:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:58:13.624 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:58:13.625 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:58:13.626 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:58:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:58:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:58:13.629 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:58:13.630 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:58:13.631 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:58:13.631 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T08:58:13.631 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-03-21T08:58:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:58:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:58:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:58:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:58:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:58:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:58:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:58:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:58:14.065 DEBUG:teuthology.parallel:result is None 2024-03-21T08:58:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:58:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:58:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:58:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:58:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:58:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:58:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:58:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:58:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:58:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:58:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:58:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:58:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:58:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:58:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:58:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:58:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:58:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:58:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:58:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:58:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:58:15.700 INFO:teuthology.orchestra.run.smithi037.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T08:58:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:20.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:58:20.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:58:20.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:58:20.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:58:20.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:58:20.654 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:58:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:58:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:58:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:58:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:58:20.671 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:58:20.672 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:58:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:58:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:58:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:58:20.676 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:58:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:58:20.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:58:20.679 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:58:20.680 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:58:20.681 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:58:23.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:58:23.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:23.311 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-03-21T08:58:23.311 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:23.311 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.312 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:58:23.313 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.314 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:58:23.315 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.316 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:58:23.317 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:58:23.318 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:58:23.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:58:23.320 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:58:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:58:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:58:23.323 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:58:23.324 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:58:23.325 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:58:23.325 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:58:23.325 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:58:23.325 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:58:23.325 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:58:23.325 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-21T08:58:23.917 DEBUG:teuthology.parallel:result is None 2024-03-21T08:58:23.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:58:24.130 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:58:24.190 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:58:24.191 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:58:24.191 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:58:24.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:58:24.368 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:58:24.394 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:58:24.394 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:58:24.394 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:58:24.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:58:24.540 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:58:24.565 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:58:24.565 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:58:24.565 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:58:24.567 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T08:58:24.568 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:58:24.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:58:24.652 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T08:58:24.652 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:58:24.692 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T08:58:24.692 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:58:24.727 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:58:24.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:58:24.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:58:24.946 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T08:58:24.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:58:24.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:58:25.056 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T08:58:25.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:58:25.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:58:25.158 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T08:58:25.158 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:58:25.158 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:58:25.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:58:25.358 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T08:58:25.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:58:25.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:58:25.462 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T08:58:25.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:58:25.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:58:25.557 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T08:58:25.557 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:58:25.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:58:25.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:58:25.746 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T08:58:25.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:58:25.777 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:58:25.849 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T08:58:25.849 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:58:25.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:58:25.941 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T08:58:26.046 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-21T08:58:26.046 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:58:26.047 INFO:tasks.cephadm:Cluster fsid is 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:58:26.047 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T08:58:26.047 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.53', 'mon.c': '172.21.15.71'} 2024-03-21T08:58:26.047 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-03-21T08:58:26.048 INFO:tasks.cephadm:First mgr is x 2024-03-21T08:58:26.048 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T08:58:26.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-03-21T08:58:26.125 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-03-21T08:58:26.157 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-03-21T08:58:26.183 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-21T08:58:26.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:58:26.335 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77791/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2231-g2edfee68', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2231.g2edfee68'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 07:25:08.579979', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240320.053002', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-21T08:58:26.487 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240320.053002, sha1 2edfee68ad7243bed422a049b6f5f58619300f11 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:58:26.488 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm 2024-03-21T08:58:26.488 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:26.946 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:58 /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:26.946 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:27.370 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:58 /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:27.371 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:27.971 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:58 /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:27.971 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:28.023 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:28.042 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:58:28.079 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 on all hosts... 2024-03-21T08:58:28.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:58:28.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:58:28.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:58:28.886 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:58:28.953 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:58:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:59:01.501 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-03-21T08:59:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:59:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:59:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-03-21T08:59:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:59:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-03-21T08:59:01.502 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-03-21T08:59:02.788 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-21T08:59:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:59:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:59:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-03-21T08:59:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:59:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-21T08:59:02.789 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-21T08:59:16.215 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T08:59:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:59:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:59:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-03-21T08:59:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:59:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T08:59:16.216 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T08:59:16.256 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-03-21T08:59:16.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-03-21T08:59:16.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-03-21T08:59:16.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-03-21T08:59:16.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-03-21T08:59:16.536 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-03-21T08:59:16.567 INFO:tasks.cephadm:Writing seed config... 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-21T08:59:16.569 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T08:59:16.570 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-21T08:59:16.571 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-21T08:59:16.572 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-21T08:59:16.572 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-21T08:59:16.572 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-21T08:59:16.572 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-21T08:59:16.572 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T08:59:16.572 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-21T08:59:16.573 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T08:59:16.573 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T08:59:16.625 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2dc01c4e-e761-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-21T08:59:16.625 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T08:59:16.670 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T08:59:16.716 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T08:59:16.716 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 -v bootstrap --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:59:16.986 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:59:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11', '-v', 'bootstrap', '--fsid', '2dc01c4e-e761-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-03-21T08:59:16.987 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T08:59:16.987 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-03-21T08:59:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T08:59:17.040 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-03-21T08:59:17.041 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-03-21T08:59:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T08:59:17.055 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T08:59:17.070 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T08:59:17.070 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-21T08:59:17.087 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-03-21T08:59:17.102 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-03-21T08:59:17.102 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-03-21T08:59:17.102 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-03-21T08:59:17.159 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T08:59:17.159 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T08:59:17.160 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-03-21T08:59:17.160 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-03-21T08:59:17.181 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T08:59:17.181 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T08:59:17.203 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T08:59:17.203 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-21T08:59:17.229 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-03-21T08:59:17.250 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-03-21T08:59:17.251 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-03-21T08:59:17.251 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-03-21T08:59:17.251 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:17.251 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140191692833744 on /run/cephadm/2dc01c4e-e761-11ee-95cd-87774f69a715.lock 2024-03-21T08:59:17.251 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140191692833744 acquired on /run/cephadm/2dc01c4e-e761-11ee-95cd-87774f69a715.lock 2024-03-21T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-03-21T08:59:17.254 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-03-21T08:59:17.254 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-03-21T08:59:17.269 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-03-21T08:59:17.269 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-03-21T08:59:17.283 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T08:59:17.283 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-21T08:59:17.283 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-21T08:59:17.294 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T08:59:17.295 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T08:59:17.295 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T08:59:17.295 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-21T08:59:17.295 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-03-21T08:59:17.295 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T08:59:17.298 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-03-21T08:59:17.298 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-03-21T08:59:17.298 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T08:59:17.301 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T08:59:17.304 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:96a207f569d15e36f8e9e3b2feafd8af7b93861f24c2b60ada6c9667dd5405be 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T08:59:17.987 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T08:59:19.257 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T08:59:19.257 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T08:59:19.257 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T08:59:20.540 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-03-21T08:59:20.540 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-03-21T08:59:21.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDp9vtlq5wWABAAb9oX50w/o/ag0XWamfJy+Q== 2024-03-21T08:59:23.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDq9vtlLvfVFRAAZJZpq+rPuqQCUv2+7zhfMQ== 2024-03-21T08:59:24.274 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDr9vtlOB3hIRAAfkaLfgbD52qNwkzlQ7uDew== 2024-03-21T08:59:24.276 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-03-21T08:59:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T08:59:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T08:59:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-03-21T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-03-21T08:59:26.982 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-03-21T08:59:27.336 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T08:59:27.589 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T08:59:27.882 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715.target. 2024-03-21T08:59:27.883 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715.target. 2024-03-21T08:59:28.304 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a 2024-03-21T08:59:28.304 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T08:59:28.603 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715.target.wants/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@.service. 2024-03-21T08:59:29.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 podman[31448]: 2024-03-21 08:59:29.025999823 +0000 UTC m=+0.155012966 container create 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:59:29.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 podman[31448]: 2024-03-21 08:59:29.271167658 +0000 UTC m=+0.400180808 container init 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:59:29.399 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:59:29.399 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-21T08:59:29.416 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:59:29.416 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-21T08:59:29.416 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-21T08:59:29.416 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T08:59:29.417 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-03-21T08:59:29.417 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-03-21T08:59:29.567 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 podman[31448]: 2024-03-21 08:59:29.282126923 +0000 UTC m=+0.411140037 container start 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T08:59:29.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 ceph-mon[31466]: mkfs 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:29.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 ceph-mon[31466]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:59:29.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 bash[31448]: 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd 2024-03-21T08:59:29.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:29 smithi037 systemd[1]: Started Ceph mon.a for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T08:59:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:30 smithi037 ceph-mon[31466]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:59:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:30 smithi037 ceph-mon[31466]: monmap e1: 1 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:59:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:30 smithi037 ceph-mon[31466]: fsmap 2024-03-21T08:59:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:30 smithi037 ceph-mon[31466]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T08:59:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:30 smithi037 ceph-mon[31466]: mgrmap e1: no daemons active 2024-03-21T08:59:30.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:30 smithi037 ceph-mon[31466]: from='client.? 172.21.15.37:0/551413219' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T08:59:30.776 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T08:59:30.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.733661s) 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T08:59:30.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.000+0000 7f5c49200700 1 Processor -- start 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.000+0000 7f5c49200700 1 -- start start 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.001+0000 7f5c49200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4410a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.001+0000 7f5c49200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4410a7d0 con 0x7f5c44109de0 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.002+0000 7f5c41e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4410a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.002+0000 7f5c41e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4410a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56880/0 (socket says 172.21.15.37:56880) 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.002+0000 7f5c41e00700 1 -- 172.21.15.37:0/3368899626 learned_addr learned my addr 172.21.15.37:0/3368899626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.003+0000 7f5c41e00700 1 -- 172.21.15.37:0/3368899626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4410af90 con 0x7f5c44109de0 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.003+0000 7f5c41e00700 1 --2- 172.21.15.37:0/3368899626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4410a200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c340098a0 tx=0x7f5c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111907e940491930 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.004+0000 7f5c40a00700 1 -- 172.21.15.37:0/3368899626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c34004160 con 0x7f5c44109de0 2024-03-21T08:59:30.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.004+0000 7f5c40a00700 1 -- 172.21.15.37:0/3368899626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34004300 con 0x7f5c44109de0 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.004+0000 7f5c40a00700 1 -- 172.21.15.37:0/3368899626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c340036a0 con 0x7f5c44109de0 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 -- 172.21.15.37:0/3368899626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 msgr2=0x7f5c4410a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 --2- 172.21.15.37:0/3368899626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4410a200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c340098a0 tx=0x7f5c34009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 -- 172.21.15.37:0/3368899626 shutdown_connections 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 --2- 172.21.15.37:0/3368899626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4410a200 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 -- 172.21.15.37:0/3368899626 >> 172.21.15.37:0/3368899626 conn(0x7f5c4407d220 msgr2=0x7f5c4407d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 -- 172.21.15.37:0/3368899626 shutdown_connections 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.005+0000 7f5c49200700 1 -- 172.21.15.37:0/3368899626 wait complete. 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.006+0000 7f5c49200700 1 Processor -- start 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.007+0000 7f5c49200700 1 -- start start 2024-03-21T08:59:30.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.007+0000 7f5c49200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.007+0000 7f5c49200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4410b4c0 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.007+0000 7f5c41e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.007+0000 7f5c41e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56896/0 (socket says 172.21.15.37:56896) 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.007+0000 7f5c41e00700 1 -- 172.21.15.37:0/551413219 learned_addr learned my addr 172.21.15.37:0/551413219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c41e00700 1 -- 172.21.15.37:0/551413219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4419da60 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c41e00700 1 --2- 172.21.15.37:0/551413219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4419d520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c4410aa70 tx=0x7f5c34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c34003db0 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34011630 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c34011930 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4419dcf0 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.008+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c441a09e0 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.009+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5c34024480 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.009+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c3401a7a0 con 0x7f5c44109de0 2024-03-21T08:59:30.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.010+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c440637c0 con 0x7f5c44109de0 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.013+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5c3401aa40 con 0x7f5c44109de0 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.092+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5c4402dc20 con 0x7f5c44109de0 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.093+0000 7f5c3aa00700 1 -- 172.21.15.37:0/551413219 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f5c3401ac20 con 0x7f5c44109de0 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 msgr2=0x7f5c4419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 --2- 172.21.15.37:0/551413219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4419d520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c4410aa70 tx=0x7f5c34004750 comp rx=0 tx=0).stop 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 shutdown_connections 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 --2- 172.21.15.37:0/551413219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c44109de0 0x7f5c4419d520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 >> 172.21.15.37:0/551413219 conn(0x7f5c4407d220 msgr2=0x7f5c44194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 shutdown_connections 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:30.095+0000 7f5c49200700 1 -- 172.21.15.37:0/551413219 wait complete. 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-03-21T08:59:30.788 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T08:59:31.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:31 smithi037 ceph-mon[31466]: from='client.? 172.21.15.37:0/39697307' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T08:59:31.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:31 smithi037 ceph-mon[31466]: from='client.? 172.21.15.37:0/39697307' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T08:59:32.146 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T08:59:32.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.369+0000 7fdc19a00700 1 Processor -- start 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.370+0000 7fdc19a00700 1 -- start start 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.370+0000 7fdc19a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.370+0000 7fdc19a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1407c7e0 con 0x7fdc1407dda0 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.371+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:32.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.371+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56908/0 (socket says 172.21.15.37:56908) 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.371+0000 7fdc12a00700 1 -- 172.21.15.37:0/3883110780 learned_addr learned my addr 172.21.15.37:0/3883110780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.371+0000 7fdc12a00700 1 -- 172.21.15.37:0/3883110780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1407c960 con 0x7fdc1407dda0 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.371+0000 7fdc12a00700 1 --2- 172.21.15.37:0/3883110780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1407c2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098a0 tx=0x7fdc04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425cd481af94fc9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc12000700 1 -- 172.21.15.37:0/3883110780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc04004160 con 0x7fdc1407dda0 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc12000700 1 -- 172.21.15.37:0/3883110780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04004300 con 0x7fdc1407dda0 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc12000700 1 -- 172.21.15.37:0/3883110780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc040036a0 con 0x7fdc1407dda0 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc19a00700 1 -- 172.21.15.37:0/3883110780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 msgr2=0x7fdc1407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc19a00700 1 --2- 172.21.15.37:0/3883110780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1407c2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098a0 tx=0x7fdc04009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc19a00700 1 -- 172.21.15.37:0/3883110780 shutdown_connections 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc19a00700 1 --2- 172.21.15.37:0/3883110780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1407c2a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.372+0000 7fdc19a00700 1 -- 172.21.15.37:0/3883110780 >> 172.21.15.37:0/3883110780 conn(0x7fdc141031d0 msgr2=0x7fdc141055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.373+0000 7fdc19a00700 1 -- 172.21.15.37:0/3883110780 shutdown_connections 2024-03-21T08:59:32.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.373+0000 7fdc19a00700 1 -- 172.21.15.37:0/3883110780 wait complete. 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.374+0000 7fdc19a00700 1 Processor -- start 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.374+0000 7fdc19a00700 1 -- start start 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.375+0000 7fdc19a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.375+0000 7fdc19a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1407ce30 con 0x7fdc1407dda0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.375+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.375+0000 7fdc12a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56912/0 (socket says 172.21.15.37:56912) 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.375+0000 7fdc12a00700 1 -- 172.21.15.37:0/39697307 learned_addr learned my addr 172.21.15.37:0/39697307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.375+0000 7fdc12a00700 1 -- 172.21.15.37:0/39697307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1419d960 con 0x7fdc1407dda0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc12a00700 1 --2- 172.21.15.37:0/39697307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1419d420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdc04000c00 tx=0x7fdc04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc04003db0 con 0x7fdc1407dda0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04011630 con 0x7fdc1407dda0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1419dbf0 con 0x7fdc1407dda0 2024-03-21T08:59:32.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc04011950 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc141a08e0 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.376+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdc04011af0 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.377+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdc0401c450 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.377+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc140637c0 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.380+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdc04032020 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.459+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdc1402dbb0 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.464+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fdc04011de0 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.465+0000 7fdc0b400700 1 -- 172.21.15.37:0/39697307 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc04004260 con 0x7fdc1407dda0 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.466+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 msgr2=0x7fdc1419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.466+0000 7fdc19a00700 1 --2- 172.21.15.37:0/39697307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1419d420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdc04000c00 tx=0x7fdc04004750 comp rx=0 tx=0).stop 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.466+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 shutdown_connections 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.467+0000 7fdc19a00700 1 --2- 172.21.15.37:0/39697307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc1407dda0 0x7fdc1419d420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.467+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 >> 172.21.15.37:0/39697307 conn(0x7fdc141031d0 msgr2=0x7fdc14104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:32.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.467+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 shutdown_connections 2024-03-21T08:59:32.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:31.467+0000 7fdc19a00700 1 -- 172.21.15.37:0/39697307 wait complete. 2024-03-21T08:59:32.160 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-03-21T08:59:33.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:32 smithi037 ceph-mon[31466]: from='client.? 172.21.15.37:0/308771943' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:59:33.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.792+0000 7f7ec9800700 1 Processor -- start 2024-03-21T08:59:33.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.792+0000 7f7ec9800700 1 -- start start 2024-03-21T08:59:33.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.792+0000 7f7ec9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec4111080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:33.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.792+0000 7f7ec9800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40823f0 con 0x7f7ec410ec90 2024-03-21T08:59:33.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.793+0000 7f7ec2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec4111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:33.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.793+0000 7f7ec2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec4111080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56924/0 (socket says 172.21.15.37:56924) 2024-03-21T08:59:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.793+0000 7f7ec2a00700 1 -- 172.21.15.37:0/1977333299 learned_addr learned my addr 172.21.15.37:0/1977333299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.793+0000 7f7ec2a00700 1 -- 172.21.15.37:0/1977333299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec41115c0 con 0x7f7ec410ec90 2024-03-21T08:59:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.793+0000 7f7ec2a00700 1 --2- 172.21.15.37:0/1977333299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec4111080 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40098a0 tx=0x7f7eb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656b5e66c2596d67 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.794+0000 7f7ec1600700 1 -- 172.21.15.37:0/1977333299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb4004160 con 0x7f7ec410ec90 2024-03-21T08:59:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.794+0000 7f7ec1600700 1 -- 172.21.15.37:0/1977333299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eb4004300 con 0x7f7ec410ec90 2024-03-21T08:59:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.794+0000 7f7ec1600700 1 -- 172.21.15.37:0/1977333299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb4011b50 con 0x7f7ec410ec90 2024-03-21T08:59:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.794+0000 7f7ec9800700 1 -- 172.21.15.37:0/1977333299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 msgr2=0x7f7ec4111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.794+0000 7f7ec9800700 1 --2- 172.21.15.37:0/1977333299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec4111080 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40098a0 tx=0x7f7eb4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.795+0000 7f7ec9800700 1 -- 172.21.15.37:0/1977333299 shutdown_connections 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.795+0000 7f7ec9800700 1 --2- 172.21.15.37:0/1977333299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec4111080 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.795+0000 7f7ec9800700 1 -- 172.21.15.37:0/1977333299 >> 172.21.15.37:0/1977333299 conn(0x7f7ec407d1c0 msgr2=0x7f7ec407f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.795+0000 7f7ec9800700 1 -- 172.21.15.37:0/1977333299 shutdown_connections 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.795+0000 7f7ec9800700 1 -- 172.21.15.37:0/1977333299 wait complete. 2024-03-21T08:59:33.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.796+0000 7f7ec9800700 1 Processor -- start 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.796+0000 7f7ec9800700 1 -- start start 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.796+0000 7f7ec9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec41a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.796+0000 7f7ec9800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec4112580 con 0x7f7ec410ec90 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.797+0000 7f7ec2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec41a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.797+0000 7f7ec2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec41a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56928/0 (socket says 172.21.15.37:56928) 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.797+0000 7f7ec2a00700 1 -- 172.21.15.37:0/308771943 learned_addr learned my addr 172.21.15.37:0/308771943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.797+0000 7f7ec2a00700 1 -- 172.21.15.37:0/308771943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec41a2690 con 0x7f7ec410ec90 2024-03-21T08:59:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.797+0000 7f7ec2a00700 1 --2- 172.21.15.37:0/308771943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec41a2150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4011b60 tx=0x7f7eb4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:33.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb4003970 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eb4003b10 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec41a2920 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb4024610 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41a5610 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7eb4024ca0 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.798+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7eb4011e00 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.799+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec40637c0 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.802+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7eb4043dd0 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.881+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7ec41a57e0 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.882+0000 7f7ebb400700 1 -- 172.21.15.37:0/308771943 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f7eb4032030 con 0x7f7ec410ec90 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.883+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 msgr2=0x7f7ec41a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:33.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.883+0000 7f7ec9800700 1 --2- 172.21.15.37:0/308771943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec41a2150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4011b60 tx=0x7f7eb4011c40 comp rx=0 tx=0).stop 2024-03-21T08:59:33.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.884+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 shutdown_connections 2024-03-21T08:59:33.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.884+0000 7f7ec9800700 1 --2- 172.21.15.37:0/308771943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec410ec90 0x7f7ec41a2150 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:33.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.884+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 >> 172.21.15.37:0/308771943 conn(0x7f7ec407d1c0 msgr2=0x7f7ec407dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:33.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.884+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 shutdown_connections 2024-03-21T08:59:33.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:32.884+0000 7f7ec9800700 1 -- 172.21.15.37:0/308771943 wait complete. 2024-03-21T08:59:33.497 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-03-21T08:59:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:33 smithi037 systemd[1]: Stopping Ceph mon.a for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T08:59:34.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a[31462]: 2024-03-21T08:59:34.063+0000 7fe47a400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:59:34.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a[31462]: 2024-03-21T08:59:34.063+0000 7fe47a400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T08:59:34.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:34 smithi037 podman[31709]: 2024-03-21 08:59:34.105979982 +0000 UTC m=+0.216456683 container died 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T08:59:43.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 podman[31709]: 2024-03-21 08:59:43.020604845 +0000 UTC m=+9.131081578 container cleanup 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:59:43.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 bash[31709]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a 2024-03-21T08:59:43.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 podman[31723]: 2024-03-21 08:59:43.213153099 +0000 UTC m=+9.090052956 container remove 3cad30e57387738ad8a31fc080591fc394448d738e910fd740d875027e9381fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:59:43.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 systemd[1]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T08:59:43.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 systemd[1]: Stopped Ceph mon.a for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T08:59:43.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 systemd[1]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service: Consumed 1.956s CPU time. 2024-03-21T08:59:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:43 smithi037 systemd[1]: Starting Ceph mon.a for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T08:59:44.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 podman[31812]: 2024-03-21 08:59:43.96544214 +0000 UTC m=+0.039596718 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:59:44.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 podman[31812]: 2024-03-21 08:59:44.063691823 +0000 UTC m=+0.137846427 container create 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T08:59:44.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 podman[31812]: 2024-03-21 08:59:44.342946377 +0000 UTC m=+0.417100988 container init 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:59:44.443 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T08:59:44.593 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 podman[31812]: 2024-03-21 08:59:44.353928527 +0000 UTC m=+0.428083104 container start 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T08:59:44.593 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: pidfile_write: ignore empty --pid-file 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: load: jerasure load: lrc 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: RocksDB version: 7.9.2 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Git sha 0 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: DB SUMMARY 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: DB Session ID: 3XH0QWBGM84PH343FPIZ 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: CURRENT file: CURRENT 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T08:59:44.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101194 ; 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.error_if_exists: 0 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.create_if_missing: 0 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.paranoid_checks: 1 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.env: 0x55d2d91b8b20 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.info_log: 0x55d2db373960 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.statistics: (nil) 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.use_fsync: 0 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_log_file_size: 0 2024-03-21T08:59:44.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.allow_fallocate: 1 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T08:59:44.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.use_direct_reads: 0 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.db_log_dir: 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.wal_dir: 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T08:59:44.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.write_buffer_manager: 0x55d2db3461e0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.unordered_write: 0 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T08:59:44.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.row_cache: None 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.wal_filter: None 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.two_write_queues: 0 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T08:59:44.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.wal_compression: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.atomic_flush: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.log_readahead_size: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_background_jobs: 2 2024-03-21T08:59:44.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_background_compactions: -1 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_subcompactions: 1 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_open_files: -1 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T08:59:44.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_background_flushes: -1 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Compression algorithms supported: 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kZSTD supported: 0 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kXpressCompression supported: 0 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kZlibCompression supported: 1 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kSnappyCompression supported: 1 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kLZ4Compression supported: 1 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: kBZip2Compression supported: 0 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T08:59:44.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T08:59:44.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T08:59:44.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T08:59:44.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T08:59:44.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.merge_operator: 2024-03-21T08:59:44.604 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_filter: None 2024-03-21T08:59:44.604 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T08:59:44.604 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T08:59:44.604 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d2db373a60) 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x55d2db379090 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-03-21T08:59:44.605 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-03-21T08:59:44.606 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T08:59:44.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression: NoCompression 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.num_levels: 7 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T08:59:44.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T08:59:44.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T08:59:44.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T08:59:44.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.table_properties_collectors: 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.inplace_update_support: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.bloom_locality: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.max_successive_merges: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.ttl: 2592000 2024-03-21T08:59:44.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.enable_blob_files: false 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.min_blob_size: 0 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T08:59:44.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7313b82f-a7a7-4bba-93ea-3b08420b1a6f 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011584412996, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011584414851, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96275, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 94264, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14045, "raw_average_key_size": 50, "raw_value_size": 86630, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711011584, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7313b82f-a7a7-4bba-93ea-3b08420b1a6f", "db_session_id": "3XH0QWBGM84PH343FPIZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011584415028, "job": 1, "event": "recovery_finished"} 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d2db46c000 2024-03-21T08:59:44.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: DB pointer 0x55d2db456000 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-21T08:59:44.615 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 95.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 95.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T08:59:44.616 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 14.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 14.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x55d2db379090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T08:59:44.617 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???) e1 preinit fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).mds e1 new map 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).mds e1 print_map 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:59:44.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 bash[31812]: 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 systemd[1]: Started Ceph mon.a for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: monmap e1: 1 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: fsmap 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T08:59:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:44 smithi037 ceph-mon[31830]: mgrmap e1: no daemons active 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.162+0000 7f57a0c00700 1 Processor -- start 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.162+0000 7f57a0c00700 1 -- start start 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.162+0000 7f57a0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c10a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.162+0000 7f57a0c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c10a740 con 0x7f579c109de0 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.163+0000 7f5799e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c10a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.163+0000 7f5799e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c10a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33036/0 (socket says 172.21.15.37:33036) 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.163+0000 7f5799e00700 1 -- 172.21.15.37:0/4065163615 learned_addr learned my addr 172.21.15.37:0/4065163615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.164+0000 7f5799e00700 1 -- 172.21.15.37:0/4065163615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c10a8c0 con 0x7f579c109de0 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.164+0000 7f5799e00700 1 --2- 172.21.15.37:0/4065163615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c10a200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f578c0098a0 tx=0x7f578c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00a461824db59f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:45.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.165+0000 7f5798a00700 1 -- 172.21.15.37:0/4065163615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f578c004160 con 0x7f579c109de0 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.165+0000 7f5798a00700 1 -- 172.21.15.37:0/4065163615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f578c004300 con 0x7f579c109de0 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.165+0000 7f5798a00700 1 -- 172.21.15.37:0/4065163615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f578c011b50 con 0x7f579c109de0 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 -- 172.21.15.37:0/4065163615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 msgr2=0x7f579c10a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 --2- 172.21.15.37:0/4065163615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c10a200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f578c0098a0 tx=0x7f578c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 -- 172.21.15.37:0/4065163615 shutdown_connections 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 --2- 172.21.15.37:0/4065163615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c10a200 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 -- 172.21.15.37:0/4065163615 >> 172.21.15.37:0/4065163615 conn(0x7f579c07d250 msgr2=0x7f579c07d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 -- 172.21.15.37:0/4065163615 shutdown_connections 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.166+0000 7f57a0c00700 1 -- 172.21.15.37:0/4065163615 wait complete. 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.167+0000 7f57a0c00700 1 Processor -- start 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.167+0000 7f57a0c00700 1 -- start start 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.168+0000 7f57a0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:45.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.168+0000 7f57a0c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c10e250 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.168+0000 7f5799e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.168+0000 7f5799e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33044/0 (socket says 172.21.15.37:33044) 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.168+0000 7f5799e00700 1 -- 172.21.15.37:0/3861956768 learned_addr learned my addr 172.21.15.37:0/3861956768 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.169+0000 7f5799e00700 1 -- 172.21.15.37:0/3861956768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c19e290 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.169+0000 7f5799e00700 1 --2- 172.21.15.37:0/3861956768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c19dd50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f578c011b60 tx=0x7f578c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.169+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f578c003970 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.169+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f578c003b10 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.169+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f578c024610 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.170+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c19e520 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.170+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c1a1210 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.170+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f578c024880 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.170+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f578c011e00 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.171+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c0637c0 con 0x7f579c109de0 2024-03-21T08:59:45.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.174+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f578c01a7c0 con 0x7f579c109de0 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.252+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f579c1a13e0 con 0x7f579c109de0 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.255+0000 7f5792a00700 1 -- 172.21.15.37:0/3861956768 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f578c024cb0 con 0x7f579c109de0 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 msgr2=0x7f579c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 --2- 172.21.15.37:0/3861956768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c19dd50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f578c011b60 tx=0x7f578c011c40 comp rx=0 tx=0).stop 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 shutdown_connections 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 --2- 172.21.15.37:0/3861956768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f579c109de0 0x7f579c19dd50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 >> 172.21.15.37:0/3861956768 conn(0x7f579c07d250 msgr2=0x7f579c107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 shutdown_connections 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:45.256+0000 7f57a0c00700 1 -- 172.21.15.37:0/3861956768 wait complete. 2024-03-21T08:59:45.825 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T08:59:45.874 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:59:45.875 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-03-21T08:59:45.876 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T08:59:45.877 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T08:59:46.316 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x 2024-03-21T08:59:46.316 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-21T08:59:46.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:46 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3861956768' entity='client.admin' 2024-03-21T08:59:46.597 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715.target.wants/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@.service. 2024-03-21T08:59:46.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:46 smithi037 systemd[1]: Starting Ceph mgr.x for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T08:59:47.298 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:46 smithi037 podman[32057]: 2024-03-21 08:59:46.903008988 +0000 UTC m=+0.034478140 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:59:47.298 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:47 smithi037 podman[32057]: 2024-03-21 08:59:47.041025091 +0000 UTC m=+0.172494255 container create 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T08:59:47.298 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:47 smithi037 podman[32057]: 2024-03-21 08:59:47.302650352 +0000 UTC m=+0.434119523 container init 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-21T08:59:47.440 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:59:47.441 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-21T08:59:47.459 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:59:47.459 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-21T08:59:47.459 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-21T08:59:47.459 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T08:59:47.480 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:59:47.481 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-21T08:59:47.501 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:59:47.501 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-21T08:59:47.501 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-21T08:59:47.501 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T08:59:47.502 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-03-21T08:59:47.502 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-03-21T08:59:47.659 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:47 smithi037 podman[32057]: 2024-03-21 08:59:47.31294346 +0000 UTC m=+0.444412627 container start 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=) 2024-03-21T08:59:47.659 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:47 smithi037 bash[32057]: 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 2024-03-21T08:59:47.659 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:47 smithi037 systemd[1]: Started Ceph mgr.x for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T08:59:48.417 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:48.153+0000 7f4bc8a2d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:59:48.417 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:48.280+0000 7f4bc8a2d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:59:48.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:48 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3056160038' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:59:49.386 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:49.151+0000 7f4bc8a2d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:59:49.386 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:49.277+0000 7f4bc8a2d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:59:49.395 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:59:49.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:59:49.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:59:49.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:59:29.363310+0000", 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.579+0000 7f0e3c000700 1 Processor -- start 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.579+0000 7f0e3c000700 1 -- start start 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.579+0000 7f0e3c000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3410aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.579+0000 7f0e3c000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3407be70 con 0x7f0e34108620 2024-03-21T08:59:49.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.580+0000 7f0e39600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3410aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.580+0000 7f0e39600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3410aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33066/0 (socket says 172.21.15.37:33066) 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.580+0000 7f0e39600700 1 -- 172.21.15.37:0/3546109579 learned_addr learned my addr 172.21.15.37:0/3546109579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.580+0000 7f0e39600700 1 -- 172.21.15.37:0/3546109579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3410af50 con 0x7f0e34108620 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.581+0000 7f0e39600700 1 --2- 172.21.15.37:0/3546109579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3410aa10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e280098a0 tx=0x7f0e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b5de190d39551f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.581+0000 7f0e33e00700 1 -- 172.21.15.37:0/3546109579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e28004160 con 0x7f0e34108620 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.581+0000 7f0e33e00700 1 -- 172.21.15.37:0/3546109579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e28004300 con 0x7f0e34108620 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.581+0000 7f0e33e00700 1 -- 172.21.15.37:0/3546109579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e28011b50 con 0x7f0e34108620 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 -- 172.21.15.37:0/3546109579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 msgr2=0x7f0e3410aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 --2- 172.21.15.37:0/3546109579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3410aa10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e280098a0 tx=0x7f0e28009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 -- 172.21.15.37:0/3546109579 shutdown_connections 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 --2- 172.21.15.37:0/3546109579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3410aa10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 -- 172.21.15.37:0/3546109579 >> 172.21.15.37:0/3546109579 conn(0x7f0e34102050 msgr2=0x7f0e34104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 -- 172.21.15.37:0/3546109579 shutdown_connections 2024-03-21T08:59:49.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.582+0000 7f0e3c000700 1 -- 172.21.15.37:0/3546109579 wait complete. 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.583+0000 7f0e3c000700 1 Processor -- start 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.584+0000 7f0e3c000700 1 -- start start 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.584+0000 7f0e3c000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.584+0000 7f0e3c000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3410bf80 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.584+0000 7f0e39600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.584+0000 7f0e39600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33070/0 (socket says 172.21.15.37:33070) 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.584+0000 7f0e39600700 1 -- 172.21.15.37:0/3056160038 learned_addr learned my addr 172.21.15.37:0/3056160038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e39600700 1 -- 172.21.15.37:0/3056160038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3419e2f0 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e39600700 1 --2- 172.21.15.37:0/3056160038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3419ddb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e28011b60 tx=0x7f0e28011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e28003970 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e28003b10 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e28024610 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3419e580 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.585+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e341a1270 con 0x7f0e34108620 2024-03-21T08:59:49.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.586+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0e28024c50 con 0x7f0e34108620 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.586+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e28011e00 con 0x7f0e34108620 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.587+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e340637c0 con 0x7f0e34108620 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.589+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0e28043dd0 con 0x7f0e34108620 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.669+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0e341a1440 con 0x7f0e34108620 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.669+0000 7f0e32000700 1 -- 172.21.15.37:0/3056160038 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f0e28032030 con 0x7f0e34108620 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 msgr2=0x7f0e3419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 --2- 172.21.15.37:0/3056160038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3419ddb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e28011b60 tx=0x7f0e28011c40 comp rx=0 tx=0).stop 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 shutdown_connections 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 --2- 172.21.15.37:0/3056160038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e34108620 0x7f0e3419ddb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 >> 172.21.15.37:0/3056160038 conn(0x7f0e34102050 msgr2=0x7f0e34102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 shutdown_connections 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:48.671+0000 7f0e3c000700 1 -- 172.21.15.37:0/3056160038 wait complete. 2024-03-21T08:59:49.413 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-03-21T08:59:49.674 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:49.460+0000 7f4bc8a2d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:59:49.674 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:49.588+0000 7f4bc8a2d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:59:50.038 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:49.701+0000 7f4bc8a2d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:59:50.299 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:50.043+0000 7f4bc8a2d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:59:50.299 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:50.188+0000 7f4bc8a2d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:59:50.647 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:50.303+0000 7f4bc8a2d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:59:50.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:50.431+0000 7f4bc8a2d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:59:50.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:50.652+0000 7f4bc8a2d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:59:52.070 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:52.034+0000 7f4bc8a2d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:59:52.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:52 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/955747173' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:59:52.424 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:52.223+0000 7f4bc8a2d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:59:52.772 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:52.654+0000 7f4bc8a2d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:59:52.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:52.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:59:52.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T08:59:52.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:59:52.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:59:52.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:59:52.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:59:52.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:59:52.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:59:52.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:59:52.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:52.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:59:52.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:59:52.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:52.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-21T08:59:52.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:59:52.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:52.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:59:52.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:59:52.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:59:52.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:59:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:59:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:59:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:59:52.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:52.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:59:29.363310+0000", 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.982+0000 7f122be00700 1 Processor -- start 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.982+0000 7f122be00700 1 -- start start 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.983+0000 7f122be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.983+0000 7f122be00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c07c7c0 con 0x7f122c07ddd0 2024-03-21T08:59:52.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.983+0000 7f122aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:52.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.983+0000 7f122aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33076/0 (socket says 172.21.15.37:33076) 2024-03-21T08:59:52.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.983+0000 7f122aa00700 1 -- 172.21.15.37:0/138797142 learned_addr learned my addr 172.21.15.37:0/138797142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:52.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.984+0000 7f122aa00700 1 -- 172.21.15.37:0/138797142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c07c940 con 0x7f122c07ddd0 2024-03-21T08:59:52.799 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.984+0000 7f122aa00700 1 --2- 172.21.15.37:0/138797142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c07c280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f121c0098a0 tx=0x7f121c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5143de3b3c4dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:52.799 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.984+0000 7f1229600700 1 -- 172.21.15.37:0/138797142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f121c004160 con 0x7f122c07ddd0 2024-03-21T08:59:52.799 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.984+0000 7f1229600700 1 -- 172.21.15.37:0/138797142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f121c004300 con 0x7f122c07ddd0 2024-03-21T08:59:52.799 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.984+0000 7f1229600700 1 -- 172.21.15.37:0/138797142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f121c011b50 con 0x7f122c07ddd0 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 -- 172.21.15.37:0/138797142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 msgr2=0x7f122c07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 --2- 172.21.15.37:0/138797142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c07c280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f121c0098a0 tx=0x7f121c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 -- 172.21.15.37:0/138797142 shutdown_connections 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 --2- 172.21.15.37:0/138797142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c07c280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 -- 172.21.15.37:0/138797142 >> 172.21.15.37:0/138797142 conn(0x7f122c102fd0 msgr2=0x7f122c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 -- 172.21.15.37:0/138797142 shutdown_connections 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.985+0000 7f122be00700 1 -- 172.21.15.37:0/138797142 wait complete. 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.987+0000 7f122be00700 1 Processor -- start 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.987+0000 7f122be00700 1 -- start start 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.987+0000 7f122be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.987+0000 7f122be00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c10e000 con 0x7f122c07ddd0 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.988+0000 7f122aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:52.800 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.988+0000 7f122aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33078/0 (socket says 172.21.15.37:33078) 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.988+0000 7f122aa00700 1 -- 172.21.15.37:0/955747173 learned_addr learned my addr 172.21.15.37:0/955747173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.988+0000 7f122aa00700 1 -- 172.21.15.37:0/955747173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c19e0a0 con 0x7f122c07ddd0 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.988+0000 7f122aa00700 1 --2- 172.21.15.37:0/955747173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c19db60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f121c011b60 tx=0x7f121c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f121c003b20 con 0x7f122c07ddd0 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f121c003cc0 con 0x7f122c07ddd0 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c19e330 con 0x7f122c07ddd0 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f121c024840 con 0x7f122c07ddd0 2024-03-21T08:59:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c1a1020 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f121c022020 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.989+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f121c011e00 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.990+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c0637c0 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:51.992+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f121c043dd0 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.075+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f122c1a11f0 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.076+0000 7f1223400700 1 -- 172.21.15.37:0/955747173 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f121c032020 con 0x7f122c07ddd0 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.077+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 msgr2=0x7f122c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.077+0000 7f122be00700 1 --2- 172.21.15.37:0/955747173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c19db60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f121c011b60 tx=0x7f121c011c40 comp rx=0 tx=0).stop 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.077+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 shutdown_connections 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.077+0000 7f122be00700 1 --2- 172.21.15.37:0/955747173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122c07ddd0 0x7f122c19db60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.077+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 >> 172.21.15.37:0/955747173 conn(0x7f122c102fd0 msgr2=0x7f122c103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.078+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 shutdown_connections 2024-03-21T08:59:52.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:52.078+0000 7f122be00700 1 -- 172.21.15.37:0/955747173 wait complete. 2024-03-21T08:59:52.804 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-03-21T08:59:53.037 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:52.776+0000 7f4bc8a2d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:59:53.288 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:53.041+0000 7f4bc8a2d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:59:53.288 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:53.292+0000 7f4bc8a2d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:59:53.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:53.673+0000 7f4bc8a2d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:59:54.300 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:53.994+0000 7f4bc8a2d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:59:54.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:54.305+0000 7f4bc8a2d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:59:55.509 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:55.386+0000 7f4bc8a2d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:59:55.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:55 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3376275343' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:59:56.222 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:56.120+0000 7f4bc8a2d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:59:56.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:56.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:59:56.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T08:59:56.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:59:56.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:56.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:59:56.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:59:56.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:59:56.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:56.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:59:56.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:59:56.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:56.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:59:29.363310+0000", 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.387+0000 7f5609400700 1 Processor -- start 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.387+0000 7f5609400700 1 -- start start 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.387+0000 7f5609400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f560410aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.387+0000 7f5609400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560407bdb0 con 0x7f56041086e0 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.388+0000 7f5602800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f560410aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.388+0000 7f5602800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f560410aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44270/0 (socket says 172.21.15.37:44270) 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.388+0000 7f5602800700 1 -- 172.21.15.37:0/1086490325 learned_addr learned my addr 172.21.15.37:0/1086490325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.388+0000 7f5602800700 1 -- 172.21.15.37:0/1086490325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560410b010 con 0x7f56041086e0 2024-03-21T08:59:56.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.388+0000 7f5602800700 1 --2- 172.21.15.37:0/1086490325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f560410aad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098a0 tx=0x7f55f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2551b19f9782a02f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.389+0000 7f5601400700 1 -- 172.21.15.37:0/1086490325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55f4004160 con 0x7f56041086e0 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.389+0000 7f5601400700 1 -- 172.21.15.37:0/1086490325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f4004300 con 0x7f56041086e0 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.389+0000 7f5601400700 1 -- 172.21.15.37:0/1086490325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55f4011b50 con 0x7f56041086e0 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.389+0000 7f5609400700 1 -- 172.21.15.37:0/1086490325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 msgr2=0x7f560410aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.389+0000 7f5609400700 1 --2- 172.21.15.37:0/1086490325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f560410aad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098a0 tx=0x7f55f4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.390+0000 7f5609400700 1 -- 172.21.15.37:0/1086490325 shutdown_connections 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.390+0000 7f5609400700 1 --2- 172.21.15.37:0/1086490325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f560410aad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.390+0000 7f5609400700 1 -- 172.21.15.37:0/1086490325 >> 172.21.15.37:0/1086490325 conn(0x7f5604102050 msgr2=0x7f5604104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.390+0000 7f5609400700 1 -- 172.21.15.37:0/1086490325 shutdown_connections 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.390+0000 7f5609400700 1 -- 172.21.15.37:0/1086490325 wait complete. 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.391+0000 7f5609400700 1 Processor -- start 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.391+0000 7f5609400700 1 -- start start 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.391+0000 7f5609400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f5604107f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.391+0000 7f5609400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560410bf80 con 0x7f56041086e0 2024-03-21T08:59:56.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f5602800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f5604107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f5602800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f5604107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44274/0 (socket says 172.21.15.37:44274) 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f5602800700 1 -- 172.21.15.37:0/3376275343 learned_addr learned my addr 172.21.15.37:0/3376275343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f5602800700 1 -- 172.21.15.37:0/3376275343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56041084a0 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f5602800700 1 --2- 172.21.15.37:0/3376275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f5604107f60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55f4011b60 tx=0x7f55f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55f4003970 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f4003b10 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.392+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55f4024610 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.393+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5604106690 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.393+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5604106bd0 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.393+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f55f4024ca0 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.393+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f55f4011e00 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.394+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56040637c0 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.397+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f55f401a9f0 con 0x7f56041086e0 2024-03-21T08:59:56.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.476+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5604106f10 con 0x7f56041086e0 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.477+0000 7f55fb400700 1 -- 172.21.15.37:0/3376275343 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f55f401a430 con 0x7f56041086e0 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 msgr2=0x7f5604107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 --2- 172.21.15.37:0/3376275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f5604107f60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55f4011b60 tx=0x7f55f4011c40 comp rx=0 tx=0).stop 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 shutdown_connections 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 --2- 172.21.15.37:0/3376275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56041086e0 0x7f5604107f60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 >> 172.21.15.37:0/3376275343 conn(0x7f5604102050 msgr2=0x7f5604102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 shutdown_connections 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:55.479+0000 7f5609400700 1 -- 172.21.15.37:0/3376275343 wait complete. 2024-03-21T08:59:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (3/15)... 2024-03-21T08:59:56.539 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:56.258+0000 7f4bc8a2d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:59:56.924 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T08:59:56.544+0000 7f4bc8a2d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:59:56.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: Activating manager daemon x 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: mgrmap e2: x(active, starting, since 0.0112216s) 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: Manager daemon x is now available 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' 2024-03-21T08:59:56.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:56 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' 2024-03-21T08:59:57.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:57 smithi037 ceph-mon[31830]: from='mgr.14100 172.21.15.37:0/140915432' entity='mgr.x' 2024-03-21T08:59:57.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:57 smithi037 ceph-mon[31830]: mgrmap e3: x(active, since 1.02177s) 2024-03-21T08:59:59.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:59 smithi037 ceph-mon[31830]: mgrmap e4: x(active, since 2s) 2024-03-21T08:59:59.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 08:59:59 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4233206552' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:59:59.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:59.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:59:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:59:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:59:59.927 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:59:29.363310+0000", 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:59:59.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.854+0000 7f42dba00700 1 Processor -- start 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.854+0000 7f42dba00700 1 -- start start 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.855+0000 7f42dba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.855+0000 7f42dba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d407c7e0 con 0x7f42d407dda0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.855+0000 7f42d9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.855+0000 7f42d9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44336/0 (socket says 172.21.15.37:44336) 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.855+0000 7f42d9000700 1 -- 172.21.15.37:0/1727911521 learned_addr learned my addr 172.21.15.37:0/1727911521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.856+0000 7f42d9000700 1 -- 172.21.15.37:0/1727911521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d407c960 con 0x7f42d407dda0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.856+0000 7f42d9000700 1 --2- 172.21.15.37:0/1727911521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d407c2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42c80098a0 tx=0x7f42c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dc2b2445bb92ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.856+0000 7f42d3400700 1 -- 172.21.15.37:0/1727911521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42c8004160 con 0x7f42d407dda0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42d3400700 1 -- 172.21.15.37:0/1727911521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c8004300 con 0x7f42d407dda0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42d3400700 1 -- 172.21.15.37:0/1727911521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42c8011b50 con 0x7f42d407dda0 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42dba00700 1 -- 172.21.15.37:0/1727911521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 msgr2=0x7f42d407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42dba00700 1 --2- 172.21.15.37:0/1727911521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d407c2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42c80098a0 tx=0x7f42c8009bb0 comp rx=0 tx=0).stop 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42dba00700 1 -- 172.21.15.37:0/1727911521 shutdown_connections 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42dba00700 1 --2- 172.21.15.37:0/1727911521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d407c2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.857+0000 7f42dba00700 1 -- 172.21.15.37:0/1727911521 >> 172.21.15.37:0/1727911521 conn(0x7f42d41031d0 msgr2=0x7f42d41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.858+0000 7f42dba00700 1 -- 172.21.15.37:0/1727911521 shutdown_connections 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.858+0000 7f42dba00700 1 -- 172.21.15.37:0/1727911521 wait complete. 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.859+0000 7f42dba00700 1 Processor -- start 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.859+0000 7f42dba00700 1 -- start start 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.859+0000 7f42dba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.859+0000 7f42dba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d410e160 con 0x7f42d407dda0 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.860+0000 7f42d9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.860+0000 7f42d9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44340/0 (socket says 172.21.15.37:44340) 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.860+0000 7f42d9000700 1 -- 172.21.15.37:0/4233206552 learned_addr learned my addr 172.21.15.37:0/4233206552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T08:59:59.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.860+0000 7f42d9000700 1 -- 172.21.15.37:0/4233206552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d419e250 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.860+0000 7f42d9000700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d419dd10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42c8011b60 tx=0x7f42c8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.861+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42c8003b20 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.861+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c8003cc0 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.861+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42c8024840 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.861+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d419e4e0 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.861+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d41a11c0 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.862+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f42c8022020 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.862+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d40637c0 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.863+0000 7f42d1600700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f42bc0391d0 0x7f42bc03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.863+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f42c8011e00 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.863+0000 7f42d3e00700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f42bc0391d0 0x7f42bc03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.864+0000 7f42d3e00700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f42bc0391d0 0x7f42bc03b690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42c0006fd0 tx=0x7f42c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:58.869+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42c8017cf0 con 0x7f42d407dda0 2024-03-21T08:59:59.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.187+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f42d41a1390 con 0x7f42d407dda0 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.188+0000 7f42d1600700 1 -- 172.21.15.37:0/4233206552 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f42d41a1390 con 0x7f42d407dda0 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f42bc0391d0 msgr2=0x7f42bc03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f42bc0391d0 0x7f42bc03b690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42c0006fd0 tx=0x7f42c0006e40 comp rx=0 tx=0).stop 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 msgr2=0x7f42d419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d419dd10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42c8011b60 tx=0x7f42c8011c40 comp rx=0 tx=0).stop 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 shutdown_connections 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f42bc0391d0 0x7f42bc03b690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 --2- 172.21.15.37:0/4233206552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42d407dda0 0x7f42d419dd10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 >> 172.21.15.37:0/4233206552 conn(0x7f42d41031d0 msgr2=0x7f42d4103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 shutdown_connections 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T08:59:59.191+0000 7f42dba00700 1 -- 172.21.15.37:0/4233206552 wait complete. 2024-03-21T08:59:59.932 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-03-21T09:00:01.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1078136232' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T09:00:01.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T09:00:01.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T09:00:01.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:00:01.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T09:00:01.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.514+0000 7f65c0a00700 1 Processor -- start 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.514+0000 7f65c0a00700 1 -- start start 2024-03-21T09:00:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.514+0000 7f65c0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc10a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.514+0000 7f65c0a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc10a7f0 con 0x7f65bc109e00 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.515+0000 7f65b9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc10a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.515+0000 7f65b9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc10a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44356/0 (socket says 172.21.15.37:44356) 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.515+0000 7f65b9e00700 1 -- 172.21.15.37:0/1115701713 learned_addr learned my addr 172.21.15.37:0/1115701713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.515+0000 7f65b9e00700 1 -- 172.21.15.37:0/1115701713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc10afb0 con 0x7f65bc109e00 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.515+0000 7f65b9e00700 1 --2- 172.21.15.37:0/1115701713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc10a220 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0098a0 tx=0x7f65ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9b6fc2c51fd7cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.516+0000 7f65b9400700 1 -- 172.21.15.37:0/1115701713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac004160 con 0x7f65bc109e00 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.516+0000 7f65b9400700 1 -- 172.21.15.37:0/1115701713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65ac004300 con 0x7f65bc109e00 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.516+0000 7f65b9400700 1 -- 172.21.15.37:0/1115701713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac011b50 con 0x7f65bc109e00 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.516+0000 7f65c0a00700 1 -- 172.21.15.37:0/1115701713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 msgr2=0x7f65bc10a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.516+0000 7f65c0a00700 1 --2- 172.21.15.37:0/1115701713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc10a220 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0098a0 tx=0x7f65ac009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:01.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.517+0000 7f65c0a00700 1 -- 172.21.15.37:0/1115701713 shutdown_connections 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.517+0000 7f65c0a00700 1 --2- 172.21.15.37:0/1115701713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc10a220 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.517+0000 7f65c0a00700 1 -- 172.21.15.37:0/1115701713 >> 172.21.15.37:0/1115701713 conn(0x7f65bc07d2c0 msgr2=0x7f65bc07d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.517+0000 7f65c0a00700 1 -- 172.21.15.37:0/1115701713 shutdown_connections 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.517+0000 7f65c0a00700 1 -- 172.21.15.37:0/1115701713 wait complete. 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.518+0000 7f65c0a00700 1 Processor -- start 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.518+0000 7f65c0a00700 1 -- start start 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.518+0000 7f65c0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.518+0000 7f65c0a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc10e360 con 0x7f65bc109e00 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.519+0000 7f65b9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.519+0000 7f65b9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44372/0 (socket says 172.21.15.37:44372) 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.519+0000 7f65b9e00700 1 -- 172.21.15.37:0/1078136232 learned_addr learned my addr 172.21.15.37:0/1078136232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.519+0000 7f65b9e00700 1 -- 172.21.15.37:0/1078136232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc19e2e0 con 0x7f65bc109e00 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.519+0000 7f65b9e00700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc19dda0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65ac011850 tx=0x7f65ac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:01.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.520+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac003af0 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.520+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65ac020070 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.520+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac0195a0 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.520+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc19e570 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.520+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc1a1220 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.521+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f65ac019850 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.521+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc0637c0 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.522+0000 7f65b3400700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f65a0039180 0x7f65a003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.522+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f65ac011e00 con 0x7f65bc109e00 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.522+0000 7f65b1400700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f65a0039180 0x7f65a003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.523+0000 7f65b1400700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f65a0039180 0x7f65a003b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006fd0 tx=0x7f65a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:01.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.527+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f65ac003c90 con 0x7f65bc109e00 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.751+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f65bc1a1440 con 0x7f65bc109e00 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.755+0000 7f65b3400700 1 -- 172.21.15.37:0/1078136232 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f65bc1a1440 con 0x7f65bc109e00 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f65a0039180 msgr2=0x7f65a003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f65a0039180 0x7f65a003b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006fd0 tx=0x7f65a4006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 msgr2=0x7f65bc19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc19dda0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65ac011850 tx=0x7f65ac011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 shutdown_connections 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f65a0039180 0x7f65a003b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 --2- 172.21.15.37:0/1078136232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65bc109e00 0x7f65bc19dda0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 >> 172.21.15.37:0/1078136232 conn(0x7f65bc07d2c0 msgr2=0x7f65bc107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.758+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 shutdown_connections 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:00.759+0000 7f65c0a00700 1 -- 172.21.15.37:0/1078136232 wait complete. 2024-03-21T09:00:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-03-21T09:00:02.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3960459799' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T09:00:02.674 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:02 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: ignoring --setuser ceph since I am not root 2024-03-21T09:00:02.674 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:02 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: ignoring --setgroup ceph since I am not root 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.071+0000 7fa0f9c00700 1 Processor -- start 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.072+0000 7fa0f9c00700 1 -- start start 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.072+0000 7fa0f9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f410aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.072+0000 7fa0f9c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f407be50 con 0x7fa0f4108640 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.073+0000 7fa0f2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f410aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.073+0000 7fa0f2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f410aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44380/0 (socket says 172.21.15.37:44380) 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.073+0000 7fa0f2a00700 1 -- 172.21.15.37:0/2942337853 learned_addr learned my addr 172.21.15.37:0/2942337853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:03.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.073+0000 7fa0f2a00700 1 -- 172.21.15.37:0/2942337853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f410af70 con 0x7fa0f4108640 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.074+0000 7fa0f2a00700 1 --2- 172.21.15.37:0/2942337853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f410aa30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40098a0 tx=0x7fa0e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6464b6ea917ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.074+0000 7fa0f2000700 1 -- 172.21.15.37:0/2942337853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e4004160 con 0x7fa0f4108640 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.074+0000 7fa0f2000700 1 -- 172.21.15.37:0/2942337853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0e4004300 con 0x7fa0f4108640 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.074+0000 7fa0f2000700 1 -- 172.21.15.37:0/2942337853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e4011b50 con 0x7fa0f4108640 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 -- 172.21.15.37:0/2942337853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 msgr2=0x7fa0f410aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 --2- 172.21.15.37:0/2942337853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f410aa30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40098a0 tx=0x7fa0e4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 -- 172.21.15.37:0/2942337853 shutdown_connections 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 --2- 172.21.15.37:0/2942337853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f410aa30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 -- 172.21.15.37:0/2942337853 >> 172.21.15.37:0/2942337853 conn(0x7fa0f4102050 msgr2=0x7fa0f4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 -- 172.21.15.37:0/2942337853 shutdown_connections 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.075+0000 7fa0f9c00700 1 -- 172.21.15.37:0/2942337853 wait complete. 2024-03-21T09:00:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.076+0000 7fa0f9c00700 1 Processor -- start 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.076+0000 7fa0f9c00700 1 -- start start 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.076+0000 7fa0f9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.076+0000 7fa0f9c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f410bf20 con 0x7fa0f4108640 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.077+0000 7fa0f2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.077+0000 7fa0f2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44396/0 (socket says 172.21.15.37:44396) 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.077+0000 7fa0f2a00700 1 -- 172.21.15.37:0/3960459799 learned_addr learned my addr 172.21.15.37:0/3960459799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.077+0000 7fa0f2a00700 1 -- 172.21.15.37:0/3960459799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f419e2e0 con 0x7fa0f4108640 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.077+0000 7fa0f2a00700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f419dda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011b60 tx=0x7fa0e4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.078+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e4003970 con 0x7fa0f4108640 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.078+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0e4003b10 con 0x7fa0f4108640 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.078+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f419e5d0 con 0x7fa0f4108640 2024-03-21T09:00:03.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.078+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e4024610 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.078+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f41a1220 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.079+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7fa0e401a9f0 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.079+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f40637c0 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.080+0000 7fa0ebe00700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa0d8039180 0x7fa0d803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.080+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa0e4011e00 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.080+0000 7fa0ea000700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa0d8039180 0x7fa0d803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.081+0000 7fa0ea000700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa0d8039180 0x7fa0d803b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc006fd0 tx=0x7fa0dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.085+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0e402ce20 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.369+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa0f41a1a50 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.423+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa0e40212a0 con 0x7fa0f4108640 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.426+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa0d8039180 msgr2=0x7fa0d803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:03.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.426+0000 7fa0f9c00700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa0d8039180 0x7fa0d803b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc006fd0 tx=0x7fa0dc006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.426+0000 7fa0ebe00700 1 -- 172.21.15.37:0/3960459799 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fa0e401a430 con 0x7fa0f4108640 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.426+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 msgr2=0x7fa0f419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.426+0000 7fa0f9c00700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f419dda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011b60 tx=0x7fa0e4011c40 comp rx=0 tx=0).stop 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.427+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 shutdown_connections 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.427+0000 7fa0f9c00700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa0d8039180 0x7fa0d803b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.427+0000 7fa0f9c00700 1 --2- 172.21.15.37:0/3960459799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0f4108640 0x7fa0f419dda0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.427+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 >> 172.21.15.37:0/3960459799 conn(0x7fa0f4102050 msgr2=0x7fa0f4102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.427+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 shutdown_connections 2024-03-21T09:00:03.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:02.427+0000 7fa0f9c00700 1 -- 172.21.15.37:0/3960459799 wait complete. 2024-03-21T09:00:03.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:03.235+0000 7efedfabe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:00:03.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:03 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3960459799' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T09:00:03.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:03 smithi037 ceph-mon[31830]: mgrmap e5: x(active, since 5s) 2024-03-21T09:00:03.674 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:03.360+0000 7efedfabe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:00:04.310 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:04.188+0000 7efedfabe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:00:04.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:04 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2936361404' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T09:00:04.623 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:04.315+0000 7efedfabe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:00:04.624 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:04.499+0000 7efedfabe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:00:04.624 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:04.628+0000 7efedfabe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:00:04.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:00:04.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T09:00:04.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T09:00:04.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T09:00:04.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.689+0000 7fa9aca00700 1 Processor -- start 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.689+0000 7fa9aca00700 1 -- start start 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.689+0000 7fa9aca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a810a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.689+0000 7fa9aca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a810a800 con 0x7fa9a8109e10 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.690+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a810a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.690+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a810a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44428/0 (socket says 172.21.15.37:44428) 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.690+0000 7fa9a5e00700 1 -- 172.21.15.37:0/1290678781 learned_addr learned my addr 172.21.15.37:0/1290678781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.690+0000 7fa9a5e00700 1 -- 172.21.15.37:0/1290678781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a810afc0 con 0x7fa9a8109e10 2024-03-21T09:00:04.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.690+0000 7fa9a5e00700 1 --2- 172.21.15.37:0/1290678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a810a230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9980098a0 tx=0x7fa998009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25b939bcb04f7c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.691+0000 7fa9a4a00700 1 -- 172.21.15.37:0/1290678781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa998004160 con 0x7fa9a8109e10 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.691+0000 7fa9a4a00700 1 -- 172.21.15.37:0/1290678781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa998004300 con 0x7fa9a8109e10 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.691+0000 7fa9a4a00700 1 -- 172.21.15.37:0/1290678781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa998011b50 con 0x7fa9a8109e10 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 -- 172.21.15.37:0/1290678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 msgr2=0x7fa9a810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 --2- 172.21.15.37:0/1290678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a810a230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9980098a0 tx=0x7fa998009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 -- 172.21.15.37:0/1290678781 shutdown_connections 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 --2- 172.21.15.37:0/1290678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a810a230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 -- 172.21.15.37:0/1290678781 >> 172.21.15.37:0/1290678781 conn(0x7fa9a807d220 msgr2=0x7fa9a807d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 -- 172.21.15.37:0/1290678781 shutdown_connections 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.692+0000 7fa9aca00700 1 -- 172.21.15.37:0/1290678781 wait complete. 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.693+0000 7fa9aca00700 1 Processor -- start 2024-03-21T09:00:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.693+0000 7fa9aca00700 1 -- start start 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.693+0000 7fa9aca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.693+0000 7fa9aca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a810e370 con 0x7fa9a8109e10 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.694+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.694+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44436/0 (socket says 172.21.15.37:44436) 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.694+0000 7fa9a5e00700 1 -- 172.21.15.37:0/2936361404 learned_addr learned my addr 172.21.15.37:0/2936361404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.694+0000 7fa9a5e00700 1 -- 172.21.15.37:0/2936361404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a819e320 con 0x7fa9a8109e10 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.694+0000 7fa9a5e00700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a819dde0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa998011850 tx=0x7fa998011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.695+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa998003af0 con 0x7fa9a8109e10 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.695+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa998020070 con 0x7fa9a8109e10 2024-03-21T09:00:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.695+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9980195a0 con 0x7fa9a8109e10 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.695+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a819e5b0 con 0x7fa9a8109e10 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.695+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a81a12a0 con 0x7fa9a8109e10 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.696+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a80637c0 con 0x7fa9a8109e10 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.697+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fa99801a430 con 0x7fa9a8109e10 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.697+0000 7fa99ea00700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 0x7fa98c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.697+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa998011e00 con 0x7fa9a8109e10 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.697+0000 7fa9a5400700 1 -- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 msgr2=0x7fa98c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.697+0000 7fa9a5400700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 0x7fa98c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:00:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.702+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa99801a720 con 0x7fa9a8109e10 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.898+0000 7fa9a5400700 1 -- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 msgr2=0x7fa98c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.898+0000 7fa9a5400700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 0x7fa98c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.987+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa9a81a1510 con 0x7fa9a8109e10 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.988+0000 7fa99ea00700 1 -- 172.21.15.37:0/2936361404 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fa99804cfd0 con 0x7fa9a8109e10 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 msgr2=0x7fa98c03b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 0x7fa98c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 msgr2=0x7fa9a819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a819dde0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa998011850 tx=0x7fa998011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 shutdown_connections 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7fa98c0391d0 0x7fa98c03b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 --2- 172.21.15.37:0/2936361404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a8109e10 0x7fa9a819dde0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 >> 172.21.15.37:0/2936361404 conn(0x7fa9a807d220 msgr2=0x7fa9a81074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 shutdown_connections 2024-03-21T09:00:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:03.992+0000 7fa9aca00700 1 -- 172.21.15.37:0/2936361404 wait complete. 2024-03-21T09:00:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-03-21T09:00:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-03-21T09:00:04.874 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:04.743+0000 7efedfabe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:00:05.173 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:05 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:05.094+0000 7efedfabe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:00:05.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:05 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:05.241+0000 7efedfabe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:00:05.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:05 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:05.360+0000 7efedfabe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:00:05.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:05 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:05.487+0000 7efedfabe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:00:05.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:05 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:05.710+0000 7efedfabe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:00:07.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:07.026+0000 7efedfabe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:00:07.424 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:07.204+0000 7efedfabe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:00:07.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:07.635+0000 7efedfabe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:00:07.924 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:07.757+0000 7efedfabe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:00:08.271 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:08.022+0000 7efedfabe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:00:08.658 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:08.276+0000 7efedfabe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:00:08.659 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:08.662+0000 7efedfabe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:00:09.297 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:08.986+0000 7efedfabe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:00:09.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:09 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:09.302+0000 7efedfabe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:00:10.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:10 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:10.366+0000 7efedfabe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:00:11.424 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:11.085+0000 7efedfabe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:00:11.424 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:11.219+0000 7efedfabe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:00:11.924 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:11.511+0000 7efedfabe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:00:11.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: Active manager daemon x restarted 2024-03-21T09:00:11.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: Activating manager daemon x 2024-03-21T09:00:11.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T09:00:11.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: mgrmap e6: x(active, starting, since 0.00852944s) 2024-03-21T09:00:11.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:00:11.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T09:00:11.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:00:11.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:00:11.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:00:11.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:11 smithi037 ceph-mon[31830]: Manager daemon x is now available 2024-03-21T09:00:12.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: Found migration_current of "None". Setting to last migration. 2024-03-21T09:00:12.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:12.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:12.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:12.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:12.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:00:12.917 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:00:12.917 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: mgrmap e7: x(active, since 1.01454s) 2024-03-21T09:00:12.917 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:12 smithi037 ceph-mon[31830]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T09:00:13.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:00:13.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T09:00:13.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T09:00:13.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.338+0000 7f43aba00700 1 Processor -- start 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.338+0000 7f43aba00700 1 -- start start 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.339+0000 7f43aba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a410a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.339+0000 7f43aba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a410a7d0 con 0x7f43a4109de0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.339+0000 7f43a9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a410a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.339+0000 7f43a9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a410a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56684/0 (socket says 172.21.15.37:56684) 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.339+0000 7f43a9000700 1 -- 172.21.15.37:0/691153804 learned_addr learned my addr 172.21.15.37:0/691153804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.339+0000 7f43a9000700 1 -- 172.21.15.37:0/691153804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a410af90 con 0x7f43a4109de0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.340+0000 7f43a9000700 1 --2- 172.21.15.37:0/691153804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a410a200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f43980098a0 tx=0x7f4398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ed394284dfede4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.340+0000 7f43a3400700 1 -- 172.21.15.37:0/691153804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4398004160 con 0x7f43a4109de0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.340+0000 7f43a3400700 1 -- 172.21.15.37:0/691153804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4398004300 con 0x7f43a4109de0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.340+0000 7f43a3400700 1 -- 172.21.15.37:0/691153804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4398011b50 con 0x7f43a4109de0 2024-03-21T09:00:13.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 -- 172.21.15.37:0/691153804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 msgr2=0x7f43a410a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 --2- 172.21.15.37:0/691153804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a410a200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f43980098a0 tx=0x7f4398009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 -- 172.21.15.37:0/691153804 shutdown_connections 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 --2- 172.21.15.37:0/691153804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a410a200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 -- 172.21.15.37:0/691153804 >> 172.21.15.37:0/691153804 conn(0x7f43a407d2c0 msgr2=0x7f43a407d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 -- 172.21.15.37:0/691153804 shutdown_connections 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.341+0000 7f43aba00700 1 -- 172.21.15.37:0/691153804 wait complete. 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.342+0000 7f43aba00700 1 Processor -- start 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.342+0000 7f43aba00700 1 -- start start 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.342+0000 7f43aba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.342+0000 7f43aba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a410e340 con 0x7f43a4109de0 2024-03-21T09:00:13.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.343+0000 7f43a9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.343+0000 7f43a9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56700/0 (socket says 172.21.15.37:56700) 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.343+0000 7f43a9000700 1 -- 172.21.15.37:0/2152073948 learned_addr learned my addr 172.21.15.37:0/2152073948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.343+0000 7f43a9000700 1 -- 172.21.15.37:0/2152073948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a419e2e0 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.343+0000 7f43a9000700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a419dda0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4398011850 tx=0x7f4398011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.344+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4398003af0 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.344+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4398020070 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.344+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a419e570 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.344+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43980195a0 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.344+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a41a1220 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.345+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f439801a430 con 0x7f43a4109de0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.345+0000 7f43a1600700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.345+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 --> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f438c03bda0 con 0x7f438c0391d0 2024-03-21T09:00:13.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.345+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4398011e00 con 0x7f43a4109de0 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.346+0000 7f43a3e00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 msgr2=0x7f438c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.346+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.547+0000 7f43a3e00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 msgr2=0x7f438c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.547+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.948+0000 7f43a3e00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 msgr2=0x7f438c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:05.948+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:06.749+0000 7f43a3e00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 msgr2=0x7f438c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:06.749+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:08.351+0000 7f43a3e00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 msgr2=0x7f438c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/276911571 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:08.351+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:11.520+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) v1 ==== 46061+0+0 (secure 0 0 0) 0x7f4398022d70 con 0x7f43a4109de0 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:11.521+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 msgr2=0x7f438c03b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:00:13.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:11.521+0000 7f43a1600700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.524+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7f439804eef0 con 0x7f43a4109de0 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.525+0000 7f43a1600700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f438c03cac0 0x7f438c03eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.525+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f438c03bda0 con 0x7f438c03cac0 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.527+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f438c03cac0 0x7f438c03eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.531+0000 7f43a3e00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f438c03cac0 0x7f438c03eeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4390003a10 tx=0x7f43900092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.534+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f438c03bda0 con 0x7f438c03cac0 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.542+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f43a40637c0 con 0x7f438c03cac0 2024-03-21T09:00:13.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.542+0000 7f43a1600700 1 -- 172.21.15.37:0/2152073948 <== mgr.14120 v2:172.21.15.37:6800/2468741411 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f43a40637c0 con 0x7f438c03cac0 2024-03-21T09:00:13.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f438c03cac0 msgr2=0x7f438c03eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:13.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f438c03cac0 0x7f438c03eeb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4390003a10 tx=0x7f43900092b0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 msgr2=0x7f43a419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:13.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a419dda0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4398011850 tx=0x7f4398011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:13.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 shutdown_connections 2024-03-21T09:00:13.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f438c03cac0 0x7f438c03eeb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43a4109de0 0x7f43a419dda0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 --2- 172.21.15.37:0/2152073948 >> [v2:172.21.15.37:6800/276911571,v1:172.21.15.37:6801/276911571] conn(0x7f438c0391d0 0x7f438c03b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:13.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 >> 172.21.15.37:0/2152073948 conn(0x7f43a407d2c0 msgr2=0x7f43a41074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:13.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 shutdown_connections 2024-03-21T09:00:13.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:12.543+0000 7f43aba00700 1 -- 172.21.15.37:0/2152073948 wait complete. 2024-03-21T09:00:13.231 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-03-21T09:00:13.231 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-03-21T09:00:13.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:13 smithi037 ceph-mon[31830]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T09:00:13.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:13 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:13.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:13 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:14 smithi037 ceph-mon[31830]: mgrmap e8: x(active, since 2s) 2024-03-21T09:00:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:14 smithi037 ceph-mon[31830]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:14 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:14 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:14 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:14] ENGINE Bus STARTING 2024-03-21T09:00:14.867 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.787+0000 7f7cd3400700 1 Processor -- start 2024-03-21T09:00:14.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.787+0000 7f7cd3400700 1 -- start start 2024-03-21T09:00:14.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.788+0000 7f7cd3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:14.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.788+0000 7f7cd3400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc07c7e0 con 0x7f7ccc07dda0 2024-03-21T09:00:14.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.788+0000 7f7cd0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:14.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.788+0000 7f7cd0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56780/0 (socket says 172.21.15.37:56780) 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.788+0000 7f7cd0a00700 1 -- 172.21.15.37:0/3716221155 learned_addr learned my addr 172.21.15.37:0/3716221155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.789+0000 7f7cd0a00700 1 -- 172.21.15.37:0/3716221155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc07c960 con 0x7f7ccc07dda0 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.789+0000 7f7cd0a00700 1 --2- 172.21.15.37:0/3716221155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc07c2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00098a0 tx=0x7f7cc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab28fcfd738d56a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.789+0000 7f7ccb400700 1 -- 172.21.15.37:0/3716221155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0004160 con 0x7f7ccc07dda0 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.789+0000 7f7ccb400700 1 -- 172.21.15.37:0/3716221155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc0004300 con 0x7f7ccc07dda0 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.789+0000 7f7ccb400700 1 -- 172.21.15.37:0/3716221155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0011b50 con 0x7f7ccc07dda0 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.790+0000 7f7cd3400700 1 -- 172.21.15.37:0/3716221155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 msgr2=0x7f7ccc07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.790+0000 7f7cd3400700 1 --2- 172.21.15.37:0/3716221155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc07c2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00098a0 tx=0x7f7cc0009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.790+0000 7f7cd3400700 1 -- 172.21.15.37:0/3716221155 shutdown_connections 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.790+0000 7f7cd3400700 1 --2- 172.21.15.37:0/3716221155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc07c2a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.790+0000 7f7cd3400700 1 -- 172.21.15.37:0/3716221155 >> 172.21.15.37:0/3716221155 conn(0x7f7ccc1031d0 msgr2=0x7f7ccc1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.790+0000 7f7cd3400700 1 -- 172.21.15.37:0/3716221155 shutdown_connections 2024-03-21T09:00:14.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.791+0000 7f7cd3400700 1 -- 172.21.15.37:0/3716221155 wait complete. 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.791+0000 7f7cd3400700 1 Processor -- start 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd3400700 1 -- start start 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd3400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc10e160 con 0x7f7ccc07dda0 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc19dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56784/0 (socket says 172.21.15.37:56784) 2024-03-21T09:00:14.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd0a00700 1 -- 172.21.15.37:0/224432766 learned_addr learned my addr 172.21.15.37:0/224432766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.792+0000 7f7cd0a00700 1 -- 172.21.15.37:0/224432766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc19e210 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.793+0000 7f7cd0a00700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc19dcd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0011b60 tx=0x7f7cc0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.793+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0003b20 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.793+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc0003cc0 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.793+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc19e4a0 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.793+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0024840 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.793+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc1a1180 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.795+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f7cc0022020 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.795+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc0637c0 con 0x7f7ccc07dda0 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.795+0000 7f7cc9600700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7cb4039180 0x7f7cb403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:14.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.795+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7cc0011e00 con 0x7f7ccc07dda0 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.795+0000 7f7ccbe00700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7cb4039180 0x7f7cb403b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.796+0000 7f7ccbe00700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7cb4039180 0x7f7cb403b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006fd0 tx=0x7f7cb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:13.801+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cc004ccd0 con 0x7f7ccc07dda0 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.062+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ccc1043b0 con 0x7f7cb4039180 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.078+0000 7f7cc9600700 1 -- 172.21.15.37:0/224432766 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7ccc1043b0 con 0x7f7cb4039180 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.080+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7cb4039180 msgr2=0x7f7cb403b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.080+0000 7f7cd3400700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7cb4039180 0x7f7cb403b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006fd0 tx=0x7f7cb8006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.080+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 msgr2=0x7f7ccc19dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.080+0000 7f7cd3400700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc19dcd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0011b60 tx=0x7f7cc0011c40 comp rx=0 tx=0).stop 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.081+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 shutdown_connections 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.081+0000 7f7cd3400700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7cb4039180 0x7f7cb403b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.081+0000 7f7cd3400700 1 --2- 172.21.15.37:0/224432766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ccc07dda0 0x7f7ccc19dcd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.081+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 >> 172.21.15.37:0/224432766 conn(0x7f7ccc1031d0 msgr2=0x7f7ccc103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:14.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.081+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 shutdown_connections 2024-03-21T09:00:14.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:14.081+0000 7f7cd3400700 1 -- 172.21.15.37:0/224432766 wait complete. 2024-03-21T09:00:15.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:15 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:14] ENGINE Serving on https://172.21.15.37:7150 2024-03-21T09:00:15.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:15 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:14] ENGINE Serving on http://172.21.15.37:8765 2024-03-21T09:00:15.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:15 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:14] ENGINE Bus STARTED 2024-03-21T09:00:15.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:15 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:16.390 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.464+0000 7fa8c1800700 1 Processor -- start 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.464+0000 7fa8c1800700 1 -- start start 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.464+0000 7fa8c1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.464+0000 7fa8c1800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc1085e0 con 0x7fa8bc107bf0 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.465+0000 7fa8baa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.465+0000 7fa8baa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42208/0 (socket says 172.21.15.37:42208) 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.465+0000 7fa8baa00700 1 -- 172.21.15.37:0/8659769 learned_addr learned my addr 172.21.15.37:0/8659769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.465+0000 7fa8baa00700 1 -- 172.21.15.37:0/8659769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc108d50 con 0x7fa8bc107bf0 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.466+0000 7fa8baa00700 1 --2- 172.21.15.37:0/8659769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc108010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009e80 tx=0x7fa8ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b5c924ce7d2a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.466+0000 7fa8b9600700 1 -- 172.21.15.37:0/8659769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8ac017070 con 0x7fa8bc107bf0 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.466+0000 7fa8b9600700 1 -- 172.21.15.37:0/8659769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8ac00b8c0 con 0x7fa8bc107bf0 2024-03-21T09:00:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.466+0000 7fa8b9600700 1 -- 172.21.15.37:0/8659769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8ac0045c0 con 0x7fa8bc107bf0 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.467+0000 7fa8c1800700 1 -- 172.21.15.37:0/8659769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 msgr2=0x7fa8bc108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.467+0000 7fa8c1800700 1 --2- 172.21.15.37:0/8659769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc108010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009e80 tx=0x7fa8ac00b330 comp rx=0 tx=0).stop 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.467+0000 7fa8c1800700 1 -- 172.21.15.37:0/8659769 shutdown_connections 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.467+0000 7fa8c1800700 1 --2- 172.21.15.37:0/8659769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc108010 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.467+0000 7fa8c1800700 1 -- 172.21.15.37:0/8659769 >> 172.21.15.37:0/8659769 conn(0x7fa8bc1031d0 msgr2=0x7fa8bc1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.467+0000 7fa8c1800700 1 -- 172.21.15.37:0/8659769 shutdown_connections 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.468+0000 7fa8c1800700 1 -- 172.21.15.37:0/8659769 wait complete. 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.469+0000 7fa8c1800700 1 Processor -- start 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.469+0000 7fa8c1800700 1 -- start start 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.470+0000 7fa8c1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.470+0000 7fa8c1800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc10c100 con 0x7fa8bc107bf0 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.470+0000 7fa8baa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.470+0000 7fa8baa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42212/0 (socket says 172.21.15.37:42212) 2024-03-21T09:00:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.470+0000 7fa8baa00700 1 -- 172.21.15.37:0/3699062863 learned_addr learned my addr 172.21.15.37:0/3699062863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.470+0000 7fa8baa00700 1 -- 172.21.15.37:0/3699062863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc19e260 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.471+0000 7fa8baa00700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc19dd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009530 tx=0x7fa8ac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.471+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8ac020020 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.471+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8ac003b50 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.471+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8ac017040 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.471+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc19e4f0 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.471+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc1a11d0 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.472+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fa8ac023020 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.472+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc0637c0 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.473+0000 7fa8b3400700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fa8a00391d0 0x7fa8a003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.473+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa8ac00f050 con 0x7fa8bc107bf0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.473+0000 7fa8ba000700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fa8a00391d0 0x7fa8a003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.474+0000 7fa8ba000700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fa8a00391d0 0x7fa8a003b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006fd0 tx=0x7fa8a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:16.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.479+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa8ac02a020 con 0x7fa8bc107bf0 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.707+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8bc104350 con 0x7fa8a00391d0 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.709+0000 7fa8b3400700 1 -- 172.21.15.37:0/3699062863 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa8bc104350 con 0x7fa8a00391d0 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fa8a00391d0 msgr2=0x7fa8a003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fa8a00391d0 0x7fa8a003b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006fd0 tx=0x7fa8a8006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 msgr2=0x7fa8bc19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc19dd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009530 tx=0x7fa8ac00bd50 comp rx=0 tx=0).stop 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 shutdown_connections 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fa8a00391d0 0x7fa8a003b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.711+0000 7fa8c1800700 1 --2- 172.21.15.37:0/3699062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8bc107bf0 0x7fa8bc19dd20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.712+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 >> 172.21.15.37:0/3699062863 conn(0x7fa8bc1031d0 msgr2=0x7fa8bc103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.712+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 shutdown_connections 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:15.712+0000 7fa8c1800700 1 -- 172.21.15.37:0/3699062863 wait complete. 2024-03-21T09:00:16.399 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-03-21T09:00:16.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:16 smithi037 ceph-mon[31830]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:17.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-mon[31830]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:17.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-mon[31830]: Generating ssh key... 2024-03-21T09:00:17.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:17.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: Generating public/private rsa key pair. 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: Your identification has been saved in /tmp/tmpfcouncc2/key. 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: Your public key has been saved in /tmp/tmpfcouncc2/key.pub. 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: The key fingerprint is: 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: SHA256:o734B40AMwKJYSTO6/HUYDgVjCacpObDNhSIH2Tt1vg ceph-2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: The key's randomart image is: 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: +---[RSA 3072]----+ 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: |@X*+. | 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: |%**.= | 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: |+O * * | 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: |= = * o | 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: | O o o .So | 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: |o * Eoo.. | 2024-03-21T09:00:17.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: | . . . .. | 2024-03-21T09:00:17.926 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: | . .. | 2024-03-21T09:00:17.926 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: | ..o. | 2024-03-21T09:00:17.926 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: +----[SHA256]-----+ 2024-03-21T09:00:18.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.061+0000 7eff3f200700 1 Processor -- start 2024-03-21T09:00:18.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.061+0000 7eff3f200700 1 -- start start 2024-03-21T09:00:18.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.062+0000 7eff3f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:18.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.062+0000 7eff3f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4007c890 con 0x7eff4007de10 2024-03-21T09:00:18.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.062+0000 7eff3de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:18.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.063+0000 7eff3de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42228/0 (socket says 172.21.15.37:42228) 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.063+0000 7eff3de00700 1 -- 172.21.15.37:0/2585913026 learned_addr learned my addr 172.21.15.37:0/2585913026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.063+0000 7eff3de00700 1 -- 172.21.15.37:0/2585913026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4007ca10 con 0x7eff4007de10 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.064+0000 7eff3de00700 1 --2- 172.21.15.37:0/2585913026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4007c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff300098a0 tx=0x7eff30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770cd874fc7c332e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.064+0000 7eff3ca00700 1 -- 172.21.15.37:0/2585913026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff30004160 con 0x7eff4007de10 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.064+0000 7eff3ca00700 1 -- 172.21.15.37:0/2585913026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff30004300 con 0x7eff4007de10 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.064+0000 7eff3ca00700 1 -- 172.21.15.37:0/2585913026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff30011b50 con 0x7eff4007de10 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 -- 172.21.15.37:0/2585913026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 msgr2=0x7eff4007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 --2- 172.21.15.37:0/2585913026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4007c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff300098a0 tx=0x7eff30009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 -- 172.21.15.37:0/2585913026 shutdown_connections 2024-03-21T09:00:18.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 --2- 172.21.15.37:0/2585913026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4007c2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 -- 172.21.15.37:0/2585913026 >> 172.21.15.37:0/2585913026 conn(0x7eff40102fd0 msgr2=0x7eff401053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 -- 172.21.15.37:0/2585913026 shutdown_connections 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.065+0000 7eff3f200700 1 -- 172.21.15.37:0/2585913026 wait complete. 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.066+0000 7eff3f200700 1 Processor -- start 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.066+0000 7eff3f200700 1 -- start start 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.067+0000 7eff3f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.067+0000 7eff3f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4010dfc0 con 0x7eff4007de10 2024-03-21T09:00:18.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.067+0000 7eff3de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.067+0000 7eff3de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42244/0 (socket says 172.21.15.37:42244) 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.067+0000 7eff3de00700 1 -- 172.21.15.37:0/240087536 learned_addr learned my addr 172.21.15.37:0/240087536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.067+0000 7eff3de00700 1 -- 172.21.15.37:0/240087536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4019e060 con 0x7eff4007de10 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.068+0000 7eff3de00700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4019db20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff30011b60 tx=0x7eff30011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.068+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff30003b20 con 0x7eff4007de10 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.068+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff30003cc0 con 0x7eff4007de10 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.068+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff30024840 con 0x7eff4007de10 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.068+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4019e2f0 con 0x7eff4007de10 2024-03-21T09:00:18.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.068+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff401a0fe0 con 0x7eff4007de10 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.069+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7eff30022020 con 0x7eff4007de10 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.069+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff400637c0 con 0x7eff4007de10 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.070+0000 7eff36a00700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7eff240391d0 0x7eff2403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.070+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7eff30011e00 con 0x7eff4007de10 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.070+0000 7eff3d400700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7eff240391d0 0x7eff2403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.071+0000 7eff3d400700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7eff240391d0 0x7eff2403b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006fd0 tx=0x7eff2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.075+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7eff3000b3e0 con 0x7eff4007de10 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.307+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7eff401041b0 con 0x7eff240391d0 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.582+0000 7eff36a00700 1 -- 172.21.15.37:0/240087536 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7eff401041b0 con 0x7eff240391d0 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7eff240391d0 msgr2=0x7eff2403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:18.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7eff240391d0 0x7eff2403b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006fd0 tx=0x7eff2c006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 msgr2=0x7eff4019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4019db20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff30011b60 tx=0x7eff30011c40 comp rx=0 tx=0).stop 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 shutdown_connections 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7eff240391d0 0x7eff2403b690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 --2- 172.21.15.37:0/240087536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff4007de10 0x7eff4019db20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.585+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 >> 172.21.15.37:0/240087536 conn(0x7eff40102fd0 msgr2=0x7eff40103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.586+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 shutdown_connections 2024-03-21T09:00:18.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:17.586+0000 7eff3f200700 1 -- 172.21.15.37:0/240087536 wait complete. 2024-03-21T09:00:19.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:19 smithi037 ceph-mon[31830]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:19.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:00:19.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.850+0000 7f4873a00700 1 Processor -- start 2024-03-21T09:00:19.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.850+0000 7f4873a00700 1 -- start start 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.851+0000 7f4873a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.851+0000 7f4873a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c1074c0 con 0x7f486c106ad0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.851+0000 7f4871000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.851+0000 7f4871000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42260/0 (socket says 172.21.15.37:42260) 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.851+0000 7f4871000700 1 -- 172.21.15.37:0/762278216 learned_addr learned my addr 172.21.15.37:0/762278216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.852+0000 7f4871000700 1 -- 172.21.15.37:0/762278216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c107cd0 con 0x7f486c106ad0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.852+0000 7f4871000700 1 --2- 172.21.15.37:0/762278216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c106ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48600098a0 tx=0x7f4860009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf89b9bf7806cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.852+0000 7f486b400700 1 -- 172.21.15.37:0/762278216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4860004160 con 0x7f486c106ad0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.852+0000 7f486b400700 1 -- 172.21.15.37:0/762278216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4860004300 con 0x7f486c106ad0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.852+0000 7f486b400700 1 -- 172.21.15.37:0/762278216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4860011b50 con 0x7f486c106ad0 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 -- 172.21.15.37:0/762278216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 msgr2=0x7f486c106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 --2- 172.21.15.37:0/762278216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c106ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48600098a0 tx=0x7f4860009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 -- 172.21.15.37:0/762278216 shutdown_connections 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 --2- 172.21.15.37:0/762278216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c106ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 -- 172.21.15.37:0/762278216 >> 172.21.15.37:0/762278216 conn(0x7f486c102070 msgr2=0x7f486c1044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 -- 172.21.15.37:0/762278216 shutdown_connections 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.853+0000 7f4873a00700 1 -- 172.21.15.37:0/762278216 wait complete. 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.854+0000 7f4873a00700 1 Processor -- start 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.855+0000 7f4873a00700 1 -- start start 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.855+0000 7f4873a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.855+0000 7f4873a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c10b080 con 0x7f486c106ad0 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.855+0000 7f4871000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.855+0000 7f4871000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c19dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42262/0 (socket says 172.21.15.37:42262) 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.855+0000 7f4871000700 1 -- 172.21.15.37:0/3059005895 learned_addr learned my addr 172.21.15.37:0/3059005895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.856+0000 7f4871000700 1 -- 172.21.15.37:0/3059005895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c19e400 con 0x7f486c106ad0 2024-03-21T09:00:19.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.856+0000 7f4871000700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c19dec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4860011850 tx=0x7f4860011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.856+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4860003af0 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.856+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4860020070 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.856+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486c19e690 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.856+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48600195a0 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.857+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486c1a1330 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.858+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486c0637c0 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.858+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f4860019850 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.858+0000 7f4869600700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f48540391d0 0x7f485403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.858+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4860011e00 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.859+0000 7f486be00700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f48540391d0 0x7f485403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.859+0000 7f486be00700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f48540391d0 0x7f485403b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4858006fd0 tx=0x7f4858006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:18.864+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4860019b40 con 0x7f486c106ad0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.094+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f486c0008d0 con 0x7f48540391d0 2024-03-21T09:00:19.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.096+0000 7f4869600700 1 -- 172.21.15.37:0/3059005895 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f486c0008d0 con 0x7f48540391d0 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f48540391d0 msgr2=0x7f485403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f48540391d0 0x7f485403b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4858006fd0 tx=0x7f4858006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 msgr2=0x7f486c19dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c19dec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4860011850 tx=0x7f4860011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 shutdown_connections 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f48540391d0 0x7f485403b690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 --2- 172.21.15.37:0/3059005895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f486c106ad0 0x7f486c19dec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 >> 172.21.15.37:0/3059005895 conn(0x7f486c102070 msgr2=0x7f486c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 shutdown_connections 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:19.099+0000 7f4873a00700 1 -- 172.21.15.37:0/3059005895 wait complete. 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T09:00:19.765 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-03-21T09:00:21.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:21 smithi037 ceph-mon[31830]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:22.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:22 smithi037 ceph-mon[31830]: Deploying cephadm binary to smithi037 2024-03-21T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-03-21T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.343+0000 7ff4ff200700 1 Processor -- start 2024-03-21T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.343+0000 7ff4ff200700 1 -- start start 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.343+0000 7ff4ff200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff500081de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.344+0000 7ff4ff200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000823b0 con 0x7ff5000819c0 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.344+0000 7ff4fde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff500081de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.344+0000 7ff4fde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff500081de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42278/0 (socket says 172.21.15.37:42278) 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.344+0000 7ff4fde00700 1 -- 172.21.15.37:0/347004782 learned_addr learned my addr 172.21.15.37:0/347004782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.345+0000 7ff4fde00700 1 -- 172.21.15.37:0/347004782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50010f1b0 con 0x7ff5000819c0 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.345+0000 7ff4fde00700 1 --2- 172.21.15.37:0/347004782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff500081de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00098a0 tx=0x7ff4f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68a7a6420ab5556 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.345+0000 7ff4fca00700 1 -- 172.21.15.37:0/347004782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0004160 con 0x7ff5000819c0 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.345+0000 7ff4fca00700 1 -- 172.21.15.37:0/347004782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f0004300 con 0x7ff5000819c0 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.345+0000 7ff4fca00700 1 -- 172.21.15.37:0/347004782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0011b50 con 0x7ff5000819c0 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.346+0000 7ff4ff200700 1 -- 172.21.15.37:0/347004782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 msgr2=0x7ff500081de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.346+0000 7ff4ff200700 1 --2- 172.21.15.37:0/347004782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff500081de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00098a0 tx=0x7ff4f0009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.346+0000 7ff4ff200700 1 -- 172.21.15.37:0/347004782 shutdown_connections 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.346+0000 7ff4ff200700 1 --2- 172.21.15.37:0/347004782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff500081de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.346+0000 7ff4ff200700 1 -- 172.21.15.37:0/347004782 >> 172.21.15.37:0/347004782 conn(0x7ff50007d040 msgr2=0x7ff50007f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.347+0000 7ff4ff200700 1 -- 172.21.15.37:0/347004782 shutdown_connections 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.347+0000 7ff4ff200700 1 -- 172.21.15.37:0/347004782 wait complete. 2024-03-21T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4ff200700 1 Processor -- start 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4ff200700 1 -- start start 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4ff200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff5001a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4ff200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500112560 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4fde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff5001a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4fde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff5001a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42294/0 (socket says 172.21.15.37:42294) 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.348+0000 7ff4fde00700 1 -- 172.21.15.37:0/705020295 learned_addr learned my addr 172.21.15.37:0/705020295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.349+0000 7ff4fde00700 1 -- 172.21.15.37:0/705020295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5001a2420 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.349+0000 7ff4fde00700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff5001a1ee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0011b60 tx=0x7ff4f0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.349+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0003b20 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.349+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f0003cc0 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.349+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5001a26b0 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.349+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0024840 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.350+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5001a53a0 con 0x7ff5000819c0 2024-03-21T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.351+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7ff4f0022020 con 0x7ff5000819c0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.351+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5000637c0 con 0x7ff5000819c0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.351+0000 7ff4f6a00700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7ff4e40391d0 0x7ff4e403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.351+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff4f0011e00 con 0x7ff5000819c0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.352+0000 7ff4fd400700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7ff4e40391d0 0x7ff4e403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.352+0000 7ff4fd400700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7ff4e40391d0 0x7ff4e403b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec006fd0 tx=0x7ff4ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.357+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff4f004ccb0 con 0x7ff5000819c0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:20.587+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) v1 -- 0x7ff50007e1d0 con 0x7ff4e40391d0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.229+0000 7ff4f6a00700 1 -- 172.21.15.37:0/705020295 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ff50007e1d0 con 0x7ff4e40391d0 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7ff4e40391d0 msgr2=0x7ff4e403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7ff4e40391d0 0x7ff4e403b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec006fd0 tx=0x7ff4ec006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 msgr2=0x7ff5001a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff5001a1ee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0011b60 tx=0x7ff4f0011c40 comp rx=0 tx=0).stop 2024-03-21T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 shutdown_connections 2024-03-21T09:00:27.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7ff4e40391d0 0x7ff4e403b690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:27.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 --2- 172.21.15.37:0/705020295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5000819c0 0x7ff5001a1ee0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:27.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.234+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 >> 172.21.15.37:0/705020295 conn(0x7ff50007d040 msgr2=0x7ff50007dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:27.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.235+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 shutdown_connections 2024-03-21T09:00:27.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:27.235+0000 7ff4ff200700 1 -- 172.21.15.37:0/705020295 wait complete. 2024-03-21T09:00:27.979 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-03-21T09:00:28.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:28 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:28.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:28 smithi037 ceph-mon[31830]: Added host smithi037 2024-03-21T09:00:28.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:28 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:29.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T09:00:29.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.615+0000 7f1e07200700 1 Processor -- start 2024-03-21T09:00:29.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.616+0000 7f1e07200700 1 -- start start 2024-03-21T09:00:29.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.616+0000 7f1e07200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0010a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:29.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.616+0000 7f1e07200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010a5d0 con 0x7f1e00109be0 2024-03-21T09:00:29.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.617+0000 7f1e05e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0010a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:29.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.617+0000 7f1e05e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0010a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39272/0 (socket says 172.21.15.37:39272) 2024-03-21T09:00:29.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.617+0000 7f1e05e00700 1 -- 172.21.15.37:0/3864470543 learned_addr learned my addr 172.21.15.37:0/3864470543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.617+0000 7f1e05e00700 1 -- 172.21.15.37:0/3864470543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0010ad90 con 0x7f1e00109be0 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.617+0000 7f1e05e00700 1 --2- 172.21.15.37:0/3864470543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0010a000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1df40098a0 tx=0x7f1df4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d683ff2a0dbdacab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.618+0000 7f1e04a00700 1 -- 172.21.15.37:0/3864470543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df4004160 con 0x7f1e00109be0 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.618+0000 7f1e04a00700 1 -- 172.21.15.37:0/3864470543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1df4004300 con 0x7f1e00109be0 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.618+0000 7f1e04a00700 1 -- 172.21.15.37:0/3864470543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df4011b50 con 0x7f1e00109be0 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 -- 172.21.15.37:0/3864470543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 msgr2=0x7f1e0010a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 --2- 172.21.15.37:0/3864470543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0010a000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1df40098a0 tx=0x7f1df4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:29.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 -- 172.21.15.37:0/3864470543 shutdown_connections 2024-03-21T09:00:29.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 --2- 172.21.15.37:0/3864470543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0010a000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:29.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 -- 172.21.15.37:0/3864470543 >> 172.21.15.37:0/3864470543 conn(0x7f1e0007d040 msgr2=0x7f1e0007d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:29.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 -- 172.21.15.37:0/3864470543 shutdown_connections 2024-03-21T09:00:29.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.619+0000 7f1e07200700 1 -- 172.21.15.37:0/3864470543 wait complete. 2024-03-21T09:00:29.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.620+0000 7f1e07200700 1 Processor -- start 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.621+0000 7f1e07200700 1 -- start start 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.621+0000 7f1e07200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.621+0000 7f1e07200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010e140 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.621+0000 7f1e05e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.621+0000 7f1e05e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39278/0 (socket says 172.21.15.37:39278) 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.621+0000 7f1e05e00700 1 -- 172.21.15.37:0/1003933042 learned_addr learned my addr 172.21.15.37:0/1003933042 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.622+0000 7f1e05e00700 1 -- 172.21.15.37:0/1003933042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0019e080 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.622+0000 7f1e05e00700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0019db40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011670 tx=0x7f1df4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.622+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df40039c0 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.622+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1df4011e50 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.622+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df401a5c0 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.622+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0019e310 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.623+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e001a1000 con 0x7f1e00109be0 2024-03-21T09:00:29.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.624+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f1df4024430 con 0x7f1e00109be0 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.624+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e000637c0 con 0x7f1e00109be0 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.624+0000 7f1dfaa00700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f1de8039180 0x7f1de803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.624+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1df404d540 con 0x7f1e00109be0 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.625+0000 7f1e05400700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f1de8039180 0x7f1de803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.625+0000 7f1e05400700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f1de8039180 0x7f1de803b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1df0006fd0 tx=0x7f1df0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.630+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1df400f2e0 con 0x7f1e00109be0 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.870+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1e00107c20 con 0x7f1de8039180 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.877+0000 7f1dfaa00700 1 -- 172.21.15.37:0/1003933042 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1e00107c20 con 0x7f1de8039180 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f1de8039180 msgr2=0x7f1de803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f1de8039180 0x7f1de803b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1df0006fd0 tx=0x7f1df0006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 msgr2=0x7f1e0019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0019db40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011670 tx=0x7f1df4011b60 comp rx=0 tx=0).stop 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 shutdown_connections 2024-03-21T09:00:29.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f1de8039180 0x7f1de803b640 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:29.557 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 --2- 172.21.15.37:0/1003933042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00109be0 0x7f1e0019db40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:29.557 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 >> 172.21.15.37:0/1003933042 conn(0x7f1e0007d040 msgr2=0x7f1e00107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:29.557 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 shutdown_connections 2024-03-21T09:00:29.557 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:28.881+0000 7f1e07200700 1 -- 172.21.15.37:0/1003933042 wait complete. 2024-03-21T09:00:29.557 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-03-21T09:00:30.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:29 smithi037 ceph-mon[31830]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:30.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:29 smithi037 ceph-mon[31830]: Saving service mon spec with placement count:5 2024-03-21T09:00:30.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:29 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:31.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T09:00:31.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.198+0000 7f7ae2a00700 1 Processor -- start 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.198+0000 7f7ae2a00700 1 -- start start 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.198+0000 7f7ae2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc107e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.198+0000 7f7ae2a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc108400 con 0x7f7adc107a10 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.198+0000 7f7ae1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.199+0000 7f7ae1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39288/0 (socket says 172.21.15.37:39288) 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.199+0000 7f7ae1600700 1 -- 172.21.15.37:0/2526795064 learned_addr learned my addr 172.21.15.37:0/2526795064 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.199+0000 7f7ae1600700 1 -- 172.21.15.37:0/2526795064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc108bc0 con 0x7f7adc107a10 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.200+0000 7f7ae1600700 1 --2- 172.21.15.37:0/2526795064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc107e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00098a0 tx=0x7f7ad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bb7306be0d970f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:31.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.200+0000 7f7adbe00700 1 -- 172.21.15.37:0/2526795064 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ad0004160 con 0x7f7adc107a10 2024-03-21T09:00:31.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.200+0000 7f7adbe00700 1 -- 172.21.15.37:0/2526795064 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ad0004300 con 0x7f7adc107a10 2024-03-21T09:00:31.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.200+0000 7f7adbe00700 1 -- 172.21.15.37:0/2526795064 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ad0011b50 con 0x7f7adc107a10 2024-03-21T09:00:31.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 -- 172.21.15.37:0/2526795064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 msgr2=0x7f7adc107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:31.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 --2- 172.21.15.37:0/2526795064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc107e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00098a0 tx=0x7f7ad0009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:31.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 -- 172.21.15.37:0/2526795064 shutdown_connections 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 --2- 172.21.15.37:0/2526795064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc107e30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 -- 172.21.15.37:0/2526795064 >> 172.21.15.37:0/2526795064 conn(0x7f7adc102fd0 msgr2=0x7f7adc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 -- 172.21.15.37:0/2526795064 shutdown_connections 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.201+0000 7f7ae2a00700 1 -- 172.21.15.37:0/2526795064 wait complete. 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.202+0000 7f7ae2a00700 1 Processor -- start 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.202+0000 7f7ae2a00700 1 -- start start 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae2a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc10bf70 con 0x7f7adc107a10 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:31.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc19dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39294/0 (socket says 172.21.15.37:39294) 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae1600700 1 -- 172.21.15.37:0/4173267173 learned_addr learned my addr 172.21.15.37:0/4173267173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae1600700 1 -- 172.21.15.37:0/4173267173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc19e0f0 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.203+0000 7f7ae1600700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc19dbb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0000c00 tx=0x7f7ad0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.204+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ad0003b20 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.204+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ad0003cc0 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.204+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ad0024910 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.204+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc19e380 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.204+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc1a1070 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.205+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc0637c0 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.205+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f7ad0022020 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.206+0000 7f7ada000700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7ac4039230 0x7f7ac403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.206+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7ad0011e00 con 0x7f7adc107a10 2024-03-21T09:00:31.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.206+0000 7f7ae0c00700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7ac4039230 0x7f7ac403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:31.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.207+0000 7f7ae0c00700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7ac4039230 0x7f7ac403b6f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8006fd0 tx=0x7f7ac8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:31.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.212+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ad001ad80 con 0x7f7adc107a10 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.454+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7adc104160 con 0x7f7ac4039230 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.462+0000 7f7ada000700 1 -- 172.21.15.37:0/4173267173 <== mgr.14120 v2:172.21.15.37:6800/2468741411 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7adc104160 con 0x7f7ac4039230 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7ac4039230 msgr2=0x7f7ac403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7ac4039230 0x7f7ac403b6f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8006fd0 tx=0x7f7ac8006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 msgr2=0x7f7adc19dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc19dbb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0000c00 tx=0x7f7ad0011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:31.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 shutdown_connections 2024-03-21T09:00:31.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f7ac4039230 0x7f7ac403b6f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:31.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 --2- 172.21.15.37:0/4173267173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7adc107a10 0x7f7adc19dbb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:31.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 >> 172.21.15.37:0/4173267173 conn(0x7f7adc102fd0 msgr2=0x7f7adc103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:31.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 shutdown_connections 2024-03-21T09:00:31.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:30.465+0000 7f7ae2a00700 1 -- 172.21.15.37:0/4173267173 wait complete. 2024-03-21T09:00:31.742 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:31 smithi037 ceph-mon[31830]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:31.743 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:31 smithi037 ceph-mon[31830]: Saving service mgr spec with placement count:2 2024-03-21T09:00:31.743 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:31 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:32 smithi037 ceph-mon[31830]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:00:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:32 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/669000016' entity='client.admin' 2024-03-21T09:00:33.559 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.942+0000 7f82ffe00700 1 Processor -- start 2024-03-21T09:00:33.559 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.942+0000 7f82ffe00700 1 -- start start 2024-03-21T09:00:33.559 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.942+0000 7f82ffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f810a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:33.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.942+0000 7f82ffe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f810a7c0 con 0x7f82f8109dd0 2024-03-21T09:00:33.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.943+0000 7f82fd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f810a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:33.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.943+0000 7f82fd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f810a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39306/0 (socket says 172.21.15.37:39306) 2024-03-21T09:00:33.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.943+0000 7f82fd400700 1 -- 172.21.15.37:0/2457778090 learned_addr learned my addr 172.21.15.37:0/2457778090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.943+0000 7f82fd400700 1 -- 172.21.15.37:0/2457778090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f810af80 con 0x7f82f8109dd0 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.944+0000 7f82fd400700 1 --2- 172.21.15.37:0/2457778090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f810a1f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0098a0 tx=0x7f82ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d71c13518f2cc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.944+0000 7f82f7e00700 1 -- 172.21.15.37:0/2457778090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82ec004160 con 0x7f82f8109dd0 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.944+0000 7f82f7e00700 1 -- 172.21.15.37:0/2457778090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82ec004300 con 0x7f82f8109dd0 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.944+0000 7f82f7e00700 1 -- 172.21.15.37:0/2457778090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82ec011b50 con 0x7f82f8109dd0 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 -- 172.21.15.37:0/2457778090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 msgr2=0x7f82f810a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 --2- 172.21.15.37:0/2457778090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f810a1f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0098a0 tx=0x7f82ec009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 -- 172.21.15.37:0/2457778090 shutdown_connections 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 --2- 172.21.15.37:0/2457778090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f810a1f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 -- 172.21.15.37:0/2457778090 >> 172.21.15.37:0/2457778090 conn(0x7f82f807d1c0 msgr2=0x7f82f807d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 -- 172.21.15.37:0/2457778090 shutdown_connections 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.945+0000 7f82ffe00700 1 -- 172.21.15.37:0/2457778090 wait complete. 2024-03-21T09:00:33.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.946+0000 7f82ffe00700 1 Processor -- start 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.946+0000 7f82ffe00700 1 -- start start 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.947+0000 7f82ffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.947+0000 7f82ffe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f810e330 con 0x7f82f8109dd0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.947+0000 7f82fd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.947+0000 7f82fd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39310/0 (socket says 172.21.15.37:39310) 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.947+0000 7f82fd400700 1 -- 172.21.15.37:0/669000016 learned_addr learned my addr 172.21.15.37:0/669000016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.947+0000 7f82fd400700 1 -- 172.21.15.37:0/669000016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f819e2e0 con 0x7f82f8109dd0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.948+0000 7f82fd400700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f819dda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82ec011850 tx=0x7f82ec011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.948+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82ec003af0 con 0x7f82f8109dd0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.948+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82ec020070 con 0x7f82f8109dd0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.948+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f819e570 con 0x7f82f8109dd0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.948+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82ec0195a0 con 0x7f82f8109dd0 2024-03-21T09:00:33.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.948+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f81a1260 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.949+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f82ec019850 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.949+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f80637c0 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.950+0000 7f82f6000700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f82e00391d0 0x7f82e003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.950+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f82ec011e00 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.950+0000 7f82fca00700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f82e00391d0 0x7f82e003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.951+0000 7f82fca00700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f82e00391d0 0x7f82e003b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82e4006fd0 tx=0x7f82e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:31.955+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82ec023530 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.191+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f82f81a18b0 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.195+0000 7f82f6000700 1 -- 172.21.15.37:0/669000016 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) v1 ==== 142+0+0 (secure 0 0 0) 0x7f82ec04d0e0 con 0x7f82f8109dd0 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.198+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f82e00391d0 msgr2=0x7f82e003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.198+0000 7f82ffe00700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f82e00391d0 0x7f82e003b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82e4006fd0 tx=0x7f82e4006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.198+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 msgr2=0x7f82f819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.198+0000 7f82ffe00700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f819dda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82ec011850 tx=0x7f82ec011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:33.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.199+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 shutdown_connections 2024-03-21T09:00:33.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.199+0000 7f82ffe00700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f82e00391d0 0x7f82e003b690 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:33.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.199+0000 7f82ffe00700 1 --2- 172.21.15.37:0/669000016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82f8109dd0 0x7f82f819dda0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:33.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.199+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 >> 172.21.15.37:0/669000016 conn(0x7f82f807d1c0 msgr2=0x7f82f81074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:33.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.199+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 shutdown_connections 2024-03-21T09:00:33.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:32.199+0000 7f82ffe00700 1 -- 172.21.15.37:0/669000016 wait complete. 2024-03-21T09:00:34.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:34 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:34.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:34 smithi037 ceph-mon[31830]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:00:34.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:34 smithi037 ceph-mon[31830]: from='mgr.14120 172.21.15.37:0/3972656355' entity='mgr.x' 2024-03-21T09:00:35.148 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.200+0000 7f360f200700 1 Processor -- start 2024-03-21T09:00:35.148 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.200+0000 7f360f200700 1 -- start start 2024-03-21T09:00:35.148 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.200+0000 7f360f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361007c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:35.148 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.200+0000 7f360f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361007c7c0 con 0x7f361007ddd0 2024-03-21T09:00:35.148 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.201+0000 7f360de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361007c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:35.148 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.201+0000 7f360de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361007c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35332/0 (socket says 172.21.15.37:35332) 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.201+0000 7f360de00700 1 -- 172.21.15.37:0/2149277509 learned_addr learned my addr 172.21.15.37:0/2149277509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.201+0000 7f360de00700 1 -- 172.21.15.37:0/2149277509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361007c940 con 0x7f361007ddd0 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.202+0000 7f360de00700 1 --2- 172.21.15.37:0/2149277509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361007c280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f36000098a0 tx=0x7f3600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce294756b77bbce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.202+0000 7f360ca00700 1 -- 172.21.15.37:0/2149277509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3600004160 con 0x7f361007ddd0 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.202+0000 7f360ca00700 1 -- 172.21.15.37:0/2149277509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3600004300 con 0x7f361007ddd0 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.202+0000 7f360ca00700 1 -- 172.21.15.37:0/2149277509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3600011b20 con 0x7f361007ddd0 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 -- 172.21.15.37:0/2149277509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 msgr2=0x7f361007c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 --2- 172.21.15.37:0/2149277509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361007c280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f36000098a0 tx=0x7f3600009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 -- 172.21.15.37:0/2149277509 shutdown_connections 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 --2- 172.21.15.37:0/2149277509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361007c280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 -- 172.21.15.37:0/2149277509 >> 172.21.15.37:0/2149277509 conn(0x7f3610102fd0 msgr2=0x7f36101053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 -- 172.21.15.37:0/2149277509 shutdown_connections 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.203+0000 7f360f200700 1 -- 172.21.15.37:0/2149277509 wait complete. 2024-03-21T09:00:35.149 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.204+0000 7f360f200700 1 Processor -- start 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.204+0000 7f360f200700 1 -- start start 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.205+0000 7f360f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.205+0000 7f360f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361010dff0 con 0x7f361007ddd0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.205+0000 7f360de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.205+0000 7f360de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35338/0 (socket says 172.21.15.37:35338) 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.205+0000 7f360de00700 1 -- 172.21.15.37:0/2353661238 learned_addr learned my addr 172.21.15.37:0/2353661238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.206+0000 7f360de00700 1 -- 172.21.15.37:0/2353661238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361019e020 con 0x7f361007ddd0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.206+0000 7f360de00700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361019dae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f36000117e0 tx=0x7f3600011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.206+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3600003ba0 con 0x7f361007ddd0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.206+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f360001a910 con 0x7f361007ddd0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.206+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36000248f0 con 0x7f361007ddd0 2024-03-21T09:00:35.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.206+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361019e2b0 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.207+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361019e710 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.208+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f3600022020 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.208+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36100637c0 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.208+0000 7f3606a00700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f35f40391d0 0x7f35f403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.208+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f360004d590 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.209+0000 7f360d400700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f35f40391d0 0x7f35f403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.209+0000 7f360d400700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f35f40391d0 0x7f35f403b690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35fc006fd0 tx=0x7f35fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.216+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f360001ab70 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.446+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f36101a11a0 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.451+0000 7f3606a00700 1 -- 172.21.15.37:0/2353661238 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f360000f2e0 con 0x7f361007ddd0 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f35f40391d0 msgr2=0x7f35f403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f35f40391d0 0x7f35f403b690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35fc006fd0 tx=0x7f35fc006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 msgr2=0x7f361019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:35.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361019dae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f36000117e0 tx=0x7f3600011ba0 comp rx=0 tx=0).stop 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 shutdown_connections 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f35f40391d0 0x7f35f403b690 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 --2- 172.21.15.37:0/2353661238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f361007ddd0 0x7f361019dae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.454+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 >> 172.21.15.37:0/2353661238 conn(0x7f3610102fd0 msgr2=0x7f3610103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.455+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 shutdown_connections 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:34.455+0000 7f360f200700 1 -- 172.21.15.37:0/2353661238 wait complete. 2024-03-21T09:00:35.152 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-03-21T09:00:35.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:35 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2353661238' entity='client.admin' 2024-03-21T09:00:36.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:36 smithi037 ceph-mon[31830]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:00:36.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:36 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/967320247' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T09:00:36.924 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: ignoring --setuser ceph since I am not root 2024-03-21T09:00:36.924 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: ignoring --setgroup ceph since I am not root 2024-03-21T09:00:37.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.813+0000 7fcbaee00700 1 Processor -- start 2024-03-21T09:00:37.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.813+0000 7fcbaee00700 1 -- start start 2024-03-21T09:00:37.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.814+0000 7fcbaee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.814+0000 7fcbaee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba807c800 con 0x7fcba807ddc0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.814+0000 7fcba7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.814+0000 7fcba7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35346/0 (socket says 172.21.15.37:35346) 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.814+0000 7fcba7e00700 1 -- 172.21.15.37:0/118744550 learned_addr learned my addr 172.21.15.37:0/118744550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.815+0000 7fcba7e00700 1 -- 172.21.15.37:0/118744550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba807c980 con 0x7fcba807ddc0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.815+0000 7fcba7e00700 1 --2- 172.21.15.37:0/118744550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba807c2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb980098a0 tx=0x7fcb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eabd288ca6757b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.815+0000 7fcba6a00700 1 -- 172.21.15.37:0/118744550 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb98004160 con 0x7fcba807ddc0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.815+0000 7fcba6a00700 1 -- 172.21.15.37:0/118744550 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb98004300 con 0x7fcba807ddc0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.815+0000 7fcba6a00700 1 -- 172.21.15.37:0/118744550 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb98011b20 con 0x7fcba807ddc0 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 -- 172.21.15.37:0/118744550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 msgr2=0x7fcba807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:37.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 --2- 172.21.15.37:0/118744550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba807c2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb980098a0 tx=0x7fcb98009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 -- 172.21.15.37:0/118744550 shutdown_connections 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 --2- 172.21.15.37:0/118744550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba807c2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 -- 172.21.15.37:0/118744550 >> 172.21.15.37:0/118744550 conn(0x7fcba81031d0 msgr2=0x7fcba81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 -- 172.21.15.37:0/118744550 shutdown_connections 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.816+0000 7fcbaee00700 1 -- 172.21.15.37:0/118744550 wait complete. 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.817+0000 7fcbaee00700 1 Processor -- start 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.818+0000 7fcbaee00700 1 -- start start 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.818+0000 7fcbaee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.818+0000 7fcbaee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba810e190 con 0x7fcba807ddc0 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.818+0000 7fcba7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.818+0000 7fcba7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba819dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35348/0 (socket says 172.21.15.37:35348) 2024-03-21T09:00:37.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.818+0000 7fcba7e00700 1 -- 172.21.15.37:0/967320247 learned_addr learned my addr 172.21.15.37:0/967320247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcba7e00700 1 -- 172.21.15.37:0/967320247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba819e1c0 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcba7e00700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba819dc80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcb980117e0 tx=0x7fcb98011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb98003ba0 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb9801a910 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba819e450 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb980248f0 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.819+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba819e8b0 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.821+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fcb98022020 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.821+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba80637c0 con 0x7fcba807ddc0 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.821+0000 7fcba4c00700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fcb8c0391d0 0x7fcb8c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.821+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcb9804d750 con 0x7fcba807ddc0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.821+0000 7fcba7400700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fcb8c0391d0 0x7fcb8c03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.822+0000 7fcba7400700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fcb8c0391d0 0x7fcb8c03b690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006fd0 tx=0x7fcb94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:35.827+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb9802cdf0 con 0x7fcba807ddc0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.125+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fcba81a16f0 con 0x7fcba807ddc0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.453+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fcb9801ab30 con 0x7fcba807ddc0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.456+0000 7fcba4c00700 1 -- 172.21.15.37:0/967320247 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fcb9802dc90 con 0x7fcba807ddc0 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fcb8c0391d0 msgr2=0x7fcb8c03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fcb8c0391d0 0x7fcb8c03b690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006fd0 tx=0x7fcb94006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 msgr2=0x7fcba819dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba819dc80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcb980117e0 tx=0x7fcb98011ba0 comp rx=0 tx=0).stop 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 shutdown_connections 2024-03-21T09:00:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7fcb8c0391d0 0x7fcb8c03b690 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:37.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 --2- 172.21.15.37:0/967320247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba807ddc0 0x7fcba819dc80 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcb980117e0 tx=0x7fcb98011ba0 comp rx=0 tx=0).stop 2024-03-21T09:00:37.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.459+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 >> 172.21.15.37:0/967320247 conn(0x7fcba81031d0 msgr2=0x7fcba8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:37.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.460+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 shutdown_connections 2024-03-21T09:00:37.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:36.460+0000 7fcbaee00700 1 -- 172.21.15.37:0/967320247 wait complete. 2024-03-21T09:00:37.537 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/967320247' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T09:00:37.537 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:37 smithi037 ceph-mon[31830]: mgrmap e9: x(active, since 24s) 2024-03-21T09:00:37.537 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:37.326+0000 7f45f8fca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:00:37.537 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:37.453+0000 7f45f8fca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:00:38.405 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:38.281+0000 7f45f8fca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:00:38.406 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:38.409+0000 7f45f8fca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:00:38.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3163865463' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T09:00:38.674 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:38.594+0000 7f45f8fca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:00:38.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:38.724+0000 7f45f8fca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:00:38.985 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:38.844+0000 7f45f8fca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:00:38.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:00:39.000 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T09:00:39.000 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.923+0000 7feb19000700 1 Processor -- start 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.924+0000 7feb19000700 1 -- start start 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.924+0000 7feb19000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb14107de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.924+0000 7feb19000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb141083b0 con 0x7feb141079c0 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.925+0000 7feb13400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb14107de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.925+0000 7feb13400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb14107de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35364/0 (socket says 172.21.15.37:35364) 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.925+0000 7feb13400700 1 -- 172.21.15.37:0/3173866575 learned_addr learned my addr 172.21.15.37:0/3173866575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.925+0000 7feb13400700 1 -- 172.21.15.37:0/3173866575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14108bc0 con 0x7feb141079c0 2024-03-21T09:00:39.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.925+0000 7feb13400700 1 --2- 172.21.15.37:0/3173866575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb14107de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb040098a0 tx=0x7feb04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4349616d80127dc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb12000700 1 -- 172.21.15.37:0/3173866575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb04004160 con 0x7feb141079c0 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb12000700 1 -- 172.21.15.37:0/3173866575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb04004300 con 0x7feb141079c0 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb12000700 1 -- 172.21.15.37:0/3173866575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb04011b20 con 0x7feb141079c0 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb19000700 1 -- 172.21.15.37:0/3173866575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 msgr2=0x7feb14107de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb19000700 1 --2- 172.21.15.37:0/3173866575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb14107de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb040098a0 tx=0x7feb04009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb19000700 1 -- 172.21.15.37:0/3173866575 shutdown_connections 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb19000700 1 --2- 172.21.15.37:0/3173866575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb14107de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.926+0000 7feb19000700 1 -- 172.21.15.37:0/3173866575 >> 172.21.15.37:0/3173866575 conn(0x7feb14102fc0 msgr2=0x7feb141053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.927+0000 7feb19000700 1 -- 172.21.15.37:0/3173866575 shutdown_connections 2024-03-21T09:00:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.927+0000 7feb19000700 1 -- 172.21.15.37:0/3173866575 wait complete. 2024-03-21T09:00:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb19000700 1 Processor -- start 2024-03-21T09:00:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb19000700 1 -- start start 2024-03-21T09:00:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb19000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb1419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb19000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1410bf70 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb13400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb1419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb13400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb1419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35374/0 (socket says 172.21.15.37:35374) 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.928+0000 7feb13400700 1 -- 172.21.15.37:0/3163865463 learned_addr learned my addr 172.21.15.37:0/3163865463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.929+0000 7feb13400700 1 -- 172.21.15.37:0/3163865463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1419e110 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.929+0000 7feb13400700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb1419dbd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb04000c00 tx=0x7feb04011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.929+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb04003b60 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.929+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb0401a910 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.929+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1419e3a0 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.929+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb04024930 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.930+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1419e800 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.931+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7feb04022020 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.931+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb140637c0 con 0x7feb141079c0 2024-03-21T09:00:39.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.931+0000 7feb0be00700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 0x7feaf803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.931+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feb0404d440 con 0x7feb141079c0 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.931+0000 7feb12a00700 1 -- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 msgr2=0x7feaf803b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.932+0000 7feb12a00700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 0x7feaf803b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:37.940+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feb0404cd90 con 0x7feb141079c0 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.133+0000 7feb12a00700 1 -- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 msgr2=0x7feaf803b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.133+0000 7feb12a00700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 0x7feaf803b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.224+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7feb141a1620 con 0x7feb141079c0 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.225+0000 7feb0be00700 1 -- 172.21.15.37:0/3163865463 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7feb04021d80 con 0x7feb141079c0 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 msgr2=0x7feaf803b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 0x7feaf803b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 msgr2=0x7feb1419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb1419dbd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb04000c00 tx=0x7feb04011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:39.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 shutdown_connections 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7feaf8039230 0x7feaf803b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 --2- 172.21.15.37:0/3163865463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb141079c0 0x7feb1419dbd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.227+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 >> 172.21.15.37:0/3163865463 conn(0x7feb14102fc0 msgr2=0x7feb14103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.228+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 shutdown_connections 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:38.228+0000 7feb19000700 1 -- 172.21.15.37:0/3163865463 wait complete. 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-03-21T09:00:39.008 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-03-21T09:00:39.299 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:39.198+0000 7f45f8fca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:00:39.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:39.350+0000 7f45f8fca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:00:39.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:39.466+0000 7f45f8fca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:00:39.812 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:39.597+0000 7f45f8fca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:00:40.173 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:39.822+0000 7f45f8fca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:00:41.304 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:41.131+0000 7f45f8fca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:00:41.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:41.307+0000 7f45f8fca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:00:42.118 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:41.736+0000 7f45f8fca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:00:42.118 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:41.857+0000 7f45f8fca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:00:42.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:42.122+0000 7f45f8fca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:00:42.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:42.375+0000 7f45f8fca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:00:43.081 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:42.762+0000 7f45f8fca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:00:43.398 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:43.085+0000 7f45f8fca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:00:43.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:43.402+0000 7f45f8fca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:00:44.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:44 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:44.471+0000 7f45f8fca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:00:45.597 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:45.188+0000 7f45f8fca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:00:45.598 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:45.320+0000 7f45f8fca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:00:45.619+0000 7f45f8fca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: Active manager daemon x restarted 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: Activating manager daemon x 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: mgrmap e10: x(active, starting, since 0.00512992s) 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:00:45.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:00:45.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:00:45.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:45 smithi037 ceph-mon[31830]: Manager daemon x is now available 2024-03-21T09:00:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:00:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:00:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:46 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:46] ENGINE Bus STARTING 2024-03-21T09:00:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:46 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:46] ENGINE Serving on http://172.21.15.37:8765 2024-03-21T09:00:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:46 smithi037 ceph-mon[31830]: mgrmap e11: x(active, since 1.00882s) 2024-03-21T09:00:47.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:00:47.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T09:00:47.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T09:00:47.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:00:47.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.607+0000 7f04b0600700 1 Processor -- start 2024-03-21T09:00:47.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.607+0000 7f04b0600700 1 -- start start 2024-03-21T09:00:47.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.608+0000 7f04b0600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a810aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:47.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.608+0000 7f04b0600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a807bdf0 con 0x7f04a81086a0 2024-03-21T09:00:47.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.608+0000 7f04adc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a810aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:47.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.608+0000 7f04adc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a810aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35388/0 (socket says 172.21.15.37:35388) 2024-03-21T09:00:47.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.608+0000 7f04adc00700 1 -- 172.21.15.37:0/2561328788 learned_addr learned my addr 172.21.15.37:0/2561328788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:47.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.609+0000 7f04adc00700 1 -- 172.21.15.37:0/2561328788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a810afd0 con 0x7f04a81086a0 2024-03-21T09:00:47.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.609+0000 7f04adc00700 1 --2- 172.21.15.37:0/2561328788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a810aa90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f049c009e80 tx=0x7f049c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88be2916d8a3af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:47.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.609+0000 7f04a7e00700 1 -- 172.21.15.37:0/2561328788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f049c017070 con 0x7f04a81086a0 2024-03-21T09:00:47.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.609+0000 7f04a7e00700 1 -- 172.21.15.37:0/2561328788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f049c00b8c0 con 0x7f04a81086a0 2024-03-21T09:00:47.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.609+0000 7f04a7e00700 1 -- 172.21.15.37:0/2561328788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f049c0045c0 con 0x7f04a81086a0 2024-03-21T09:00:47.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 -- 172.21.15.37:0/2561328788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 msgr2=0x7f04a810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:47.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 --2- 172.21.15.37:0/2561328788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a810aa90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f049c009e80 tx=0x7f049c00b330 comp rx=0 tx=0).stop 2024-03-21T09:00:47.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 -- 172.21.15.37:0/2561328788 shutdown_connections 2024-03-21T09:00:47.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 --2- 172.21.15.37:0/2561328788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a810aa90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:47.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 -- 172.21.15.37:0/2561328788 >> 172.21.15.37:0/2561328788 conn(0x7f04a8102050 msgr2=0x7f04a8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:47.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 -- 172.21.15.37:0/2561328788 shutdown_connections 2024-03-21T09:00:47.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.610+0000 7f04b0600700 1 -- 172.21.15.37:0/2561328788 wait complete. 2024-03-21T09:00:47.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.611+0000 7f04b0600700 1 Processor -- start 2024-03-21T09:00:47.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.611+0000 7f04b0600700 1 -- start start 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.612+0000 7f04b0600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a8107f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.612+0000 7f04b0600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a810bfd0 con 0x7f04a81086a0 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.612+0000 7f04adc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a8107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.612+0000 7f04adc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a8107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35396/0 (socket says 172.21.15.37:35396) 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.612+0000 7f04adc00700 1 -- 172.21.15.37:0/1273336074 learned_addr learned my addr 172.21.15.37:0/1273336074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.612+0000 7f04adc00700 1 -- 172.21.15.37:0/1273336074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8108460 con 0x7f04a81086a0 2024-03-21T09:00:47.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.613+0000 7f04adc00700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a8107f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f049c009530 tx=0x7f049c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.613+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f049c020020 con 0x7f04a81086a0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.613+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f049c00bba0 con 0x7f04a81086a0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.613+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f049c017040 con 0x7f04a81086a0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.613+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a8106650 con 0x7f04a81086a0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.613+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a8106b90 con 0x7f04a81086a0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.615+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f049c023020 con 0x7f04a81086a0 2024-03-21T09:00:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.615+0000 7f04a6000700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.615+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 --> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f049003bda0 con 0x7f04900391d0 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.615+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f049c00f050 con 0x7f04a81086a0 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.615+0000 7f04ad200700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 msgr2=0x7f049003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.615+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.816+0000 7f04ad200700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 msgr2=0x7f049003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:39.816+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:00:47.355 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:40.217+0000 7f04ad200700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 msgr2=0x7f049003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:47.356 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:40.217+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T09:00:47.356 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:41.018+0000 7f04ad200700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 msgr2=0x7f049003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:47.356 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:41.018+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T09:00:47.356 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:42.620+0000 7f04ad200700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 msgr2=0x7f049003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2468741411 2024-03-21T09:00:47.357 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:42.620+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T09:00:47.357 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:45.626+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) v1 ==== 46074+0+0 (secure 0 0 0) 0x7f049c012430 con 0x7f04a81086a0 2024-03-21T09:00:47.357 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:45.627+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 msgr2=0x7f049003b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:00:47.357 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:45.627+0000 7f04a6000700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:47.357 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.628+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) v1 ==== 46201+0+0 (secure 0 0 0) 0x7f049c04ec00 con 0x7f04a81086a0 2024-03-21T09:00:47.357 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.629+0000 7f04a6000700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f049003cc50 0x7f049003f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.629+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f049c00ee50 con 0x7f049003cc50 2024-03-21T09:00:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.630+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f049003cc50 0x7f049003f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.631+0000 7f04ad200700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f049003cc50 0x7f049003f040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0494003a10 tx=0x7f04940092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.632+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f049c00ee50 con 0x7f049003cc50 2024-03-21T09:00:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.639+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f04a80637c0 con 0x7f049003cc50 2024-03-21T09:00:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.640+0000 7f04a6000700 1 -- 172.21.15.37:0/1273336074 <== mgr.14152 v2:172.21.15.37:6800/1300592626 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f04a80637c0 con 0x7f049003cc50 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.640+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f049003cc50 msgr2=0x7f049003f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.640+0000 7f04b0600700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f049003cc50 0x7f049003f040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0494003a10 tx=0x7f04940092b0 comp rx=0 tx=0).stop 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.640+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 msgr2=0x7f04a8107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.640+0000 7f04b0600700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a8107f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f049c009530 tx=0x7f049c003d00 comp rx=0 tx=0).stop 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 shutdown_connections 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f049003cc50 0x7f049003f040 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a81086a0 0x7f04a8107f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 --2- 172.21.15.37:0/1273336074 >> [v2:172.21.15.37:6800/2468741411,v1:172.21.15.37:6801/2468741411] conn(0x7f04900391d0 0x7f049003b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 >> 172.21.15.37:0/1273336074 conn(0x7f04a8102050 msgr2=0x7f04a8102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 shutdown_connections 2024-03-21T09:00:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:46.641+0000 7f04b0600700 1 -- 172.21.15.37:0/1273336074 wait complete. 2024-03-21T09:00:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-03-21T09:00:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-03-21T09:00:47.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:47 smithi037 ceph-mon[31830]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T09:00:47.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:47 smithi037 ceph-mon[31830]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T09:00:47.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:47 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:46] ENGINE Serving on https://172.21.15.37:7150 2024-03-21T09:00:47.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:47 smithi037 ceph-mon[31830]: [21/Mar/2024:09:00:46] ENGINE Bus STARTED 2024-03-21T09:00:48.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:48 smithi037 ceph-mon[31830]: mgrmap e12: x(active, since 2s) 2024-03-21T09:00:48.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:00:48.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:48 smithi037 ceph-mon[31830]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:49.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T09:00:49.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.332+0000 7f755f200700 1 Processor -- start 2024-03-21T09:00:49.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.332+0000 7f755f200700 1 -- start start 2024-03-21T09:00:49.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.333+0000 7f755f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f7558111080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.333+0000 7f755f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580823f0 con 0x7f755810ec90 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.333+0000 7f7557e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f7558111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.333+0000 7f7557e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f7558111080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34304/0 (socket says 172.21.15.37:34304) 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.333+0000 7f7557e00700 1 -- 172.21.15.37:0/505067442 learned_addr learned my addr 172.21.15.37:0/505067442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.334+0000 7f7557e00700 1 -- 172.21.15.37:0/505067442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75581115c0 con 0x7f755810ec90 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.334+0000 7f7557e00700 1 --2- 172.21.15.37:0/505067442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f7558111080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75480098a0 tx=0x7f7548009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dea6e349c4bf94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.334+0000 7f7556a00700 1 -- 172.21.15.37:0/505067442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7548004160 con 0x7f755810ec90 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f7556a00700 1 -- 172.21.15.37:0/505067442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548004300 con 0x7f755810ec90 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f7556a00700 1 -- 172.21.15.37:0/505067442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7548011b20 con 0x7f755810ec90 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f755f200700 1 -- 172.21.15.37:0/505067442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 msgr2=0x7f7558111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:49.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f755f200700 1 --2- 172.21.15.37:0/505067442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f7558111080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75480098a0 tx=0x7f7548009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f755f200700 1 -- 172.21.15.37:0/505067442 shutdown_connections 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f755f200700 1 --2- 172.21.15.37:0/505067442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f7558111080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.335+0000 7f755f200700 1 -- 172.21.15.37:0/505067442 >> 172.21.15.37:0/505067442 conn(0x7f755807d1c0 msgr2=0x7f755807f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.336+0000 7f755f200700 1 -- 172.21.15.37:0/505067442 shutdown_connections 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.336+0000 7f755f200700 1 -- 172.21.15.37:0/505067442 wait complete. 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.337+0000 7f755f200700 1 Processor -- start 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.337+0000 7f755f200700 1 -- start start 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.337+0000 7f755f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f75581a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.338+0000 7f755f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75581125c0 con 0x7f755810ec90 2024-03-21T09:00:49.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.338+0000 7f7557e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f75581a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:49.453 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.338+0000 7f7557e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f75581a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34318/0 (socket says 172.21.15.37:34318) 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.338+0000 7f7557e00700 1 -- 172.21.15.37:0/1570713814 learned_addr learned my addr 172.21.15.37:0/1570713814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.338+0000 7f7557e00700 1 -- 172.21.15.37:0/1570713814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75581a2620 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.339+0000 7f7557e00700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f75581a20e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f75480116a0 tx=0x7f7548011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.339+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7548003a40 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.339+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548003be0 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.339+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f754801a750 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.339+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75581a28b0 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.339+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75581a2d10 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.340+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f7548024aa0 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.341+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75580637c0 con 0x7f755810ec90 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.341+0000 7f7554c00700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f753c0391e0 0x7f753c03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:49.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.341+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f754804d970 con 0x7f755810ec90 2024-03-21T09:00:49.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.341+0000 7f7557400700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f753c0391e0 0x7f753c03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:49.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.342+0000 7f7557400700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f753c0391e0 0x7f753c03b6a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7544006fd0 tx=0x7f7544006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:49.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.347+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7548011e20 con 0x7f755810ec90 2024-03-21T09:00:49.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.600+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f755807e370 con 0x7f753c0391e0 2024-03-21T09:00:49.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.738+0000 7f7554c00700 1 -- 172.21.15.37:0/1570713814 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f755807e370 con 0x7f753c0391e0 2024-03-21T09:00:49.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f753c0391e0 msgr2=0x7f753c03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f753c0391e0 0x7f753c03b6a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7544006fd0 tx=0x7f7544006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 msgr2=0x7f75581a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f75581a20e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f75480116a0 tx=0x7f7548011ba0 comp rx=0 tx=0).stop 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 shutdown_connections 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f753c0391e0 0x7f753c03b6a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 --2- 172.21.15.37:0/1570713814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f755810ec90 0x7f75581a20e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 >> 172.21.15.37:0/1570713814 conn(0x7f755807d1c0 msgr2=0x7f755807dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 shutdown_connections 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:48.741+0000 7f755f200700 1 -- 172.21.15.37:0/1570713814 wait complete. 2024-03-21T09:00:49.458 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-03-21T09:00:50.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:00:50.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:00:50.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:50 smithi037 ceph-mon[31830]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$q4ZDYRfsJRMyw/cL0woJC.MYuVJ3ijM5tnJD0iJtLzZoMewTZyljq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711011650, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.237+0000 7ff7d9400700 1 Processor -- start 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.237+0000 7ff7d9400700 1 -- start start 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.237+0000 7ff7d9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d4108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.237+0000 7ff7d9400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d41085e0 con 0x7ff7d4107bf0 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.238+0000 7ff7d2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d4108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:51.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.238+0000 7ff7d2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d4108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34326/0 (socket says 172.21.15.37:34326) 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.238+0000 7ff7d2800700 1 -- 172.21.15.37:0/2305276613 learned_addr learned my addr 172.21.15.37:0/2305276613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.239+0000 7ff7d2800700 1 -- 172.21.15.37:0/2305276613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4108d50 con 0x7ff7d4107bf0 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.239+0000 7ff7d2800700 1 --2- 172.21.15.37:0/2305276613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d4108010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40098a0 tx=0x7ff7c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d86863e5fcc0a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.239+0000 7ff7d1400700 1 -- 172.21.15.37:0/2305276613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c4004160 con 0x7ff7d4107bf0 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.239+0000 7ff7d1400700 1 -- 172.21.15.37:0/2305276613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7c4004300 con 0x7ff7d4107bf0 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.239+0000 7ff7d1400700 1 -- 172.21.15.37:0/2305276613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c4011b20 con 0x7ff7d4107bf0 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 -- 172.21.15.37:0/2305276613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 msgr2=0x7ff7d4108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 --2- 172.21.15.37:0/2305276613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d4108010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40098a0 tx=0x7ff7c4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 -- 172.21.15.37:0/2305276613 shutdown_connections 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 --2- 172.21.15.37:0/2305276613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d4108010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 -- 172.21.15.37:0/2305276613 >> 172.21.15.37:0/2305276613 conn(0x7ff7d41031d0 msgr2=0x7ff7d41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 -- 172.21.15.37:0/2305276613 shutdown_connections 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.240+0000 7ff7d9400700 1 -- 172.21.15.37:0/2305276613 wait complete. 2024-03-21T09:00:51.515 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.241+0000 7ff7d9400700 1 Processor -- start 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.242+0000 7ff7d9400700 1 -- start start 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.242+0000 7ff7d9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.242+0000 7ff7d9400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d410c100 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.242+0000 7ff7d2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.242+0000 7ff7d2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d419dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34334/0 (socket says 172.21.15.37:34334) 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.242+0000 7ff7d2800700 1 -- 172.21.15.37:0/3013626684 learned_addr learned my addr 172.21.15.37:0/3013626684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.243+0000 7ff7d2800700 1 -- 172.21.15.37:0/3013626684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d419e220 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.243+0000 7ff7d2800700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d419dce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40117e0 tx=0x7ff7c4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.243+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c4003bf0 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.243+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7c401a910 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.243+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c4024970 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.243+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d419e4b0 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.244+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d419e910 con 0x7ff7d4107bf0 2024-03-21T09:00:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.245+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7ff7c4022020 con 0x7ff7d4107bf0 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.245+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d40637c0 con 0x7ff7d4107bf0 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.246+0000 7ff7cb400700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff7b8039280 0x7ff7b803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.246+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff7c404d6c0 con 0x7ff7d4107bf0 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.246+0000 7ff7d1e00700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff7b8039280 0x7ff7b803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.247+0000 7ff7d1e00700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff7b8039280 0x7ff7b803b740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.252+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff7c400f2e0 con 0x7ff7d4107bf0 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.507+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff7d41a1250 con 0x7ff7b8039280 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.769+0000 7ff7cb400700 1 -- 172.21.15.37:0/3013626684 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ff7d41a1250 con 0x7ff7b8039280 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff7b8039280 msgr2=0x7ff7b803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff7b8039280 0x7ff7b803b740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 msgr2=0x7ff7d419dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d419dce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40117e0 tx=0x7ff7c4011ba0 comp rx=0 tx=0).stop 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 shutdown_connections 2024-03-21T09:00:51.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff7b8039280 0x7ff7b803b740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:51.518 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 --2- 172.21.15.37:0/3013626684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4107bf0 0x7ff7d419dce0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:51.518 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.772+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 >> 172.21.15.37:0/3013626684 conn(0x7ff7d41031d0 msgr2=0x7ff7d4103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:51.518 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.773+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 shutdown_connections 2024-03-21T09:00:51.518 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:50.773+0000 7ff7d9400700 1 -- 172.21.15.37:0/3013626684 wait complete. 2024-03-21T09:00:51.518 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-03-21T09:00:52.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:51 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:00:52.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:51 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:00:53.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:53 smithi037 ceph-mon[31830]: mgrmap e13: x(active, since 6s) 2024-03-21T09:00:53.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:53 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1569742471' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T09:00:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T09:00:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.157+0000 7fc43a600700 1 Processor -- start 2024-03-21T09:00:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.157+0000 7fc43a600700 1 -- start start 2024-03-21T09:00:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.158+0000 7fc43a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc434106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.158+0000 7fc43a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc434107490 con 0x7fc434106aa0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.158+0000 7fc433400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc434106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.158+0000 7fc433400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc434106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34348/0 (socket says 172.21.15.37:34348) 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.158+0000 7fc433400700 1 -- 172.21.15.37:0/492756199 learned_addr learned my addr 172.21.15.37:0/492756199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.159+0000 7fc433400700 1 -- 172.21.15.37:0/492756199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434107ca0 con 0x7fc434106aa0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.159+0000 7fc433400700 1 --2- 172.21.15.37:0/492756199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc434106ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4240098a0 tx=0x7fc424009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a99d942cab47d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.159+0000 7fc432000700 1 -- 172.21.15.37:0/492756199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc424004160 con 0x7fc434106aa0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.159+0000 7fc432000700 1 -- 172.21.15.37:0/492756199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc424004300 con 0x7fc434106aa0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.159+0000 7fc432000700 1 -- 172.21.15.37:0/492756199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc424011b20 con 0x7fc434106aa0 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 -- 172.21.15.37:0/492756199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 msgr2=0x7fc434106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 --2- 172.21.15.37:0/492756199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc434106ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4240098a0 tx=0x7fc424009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 -- 172.21.15.37:0/492756199 shutdown_connections 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 --2- 172.21.15.37:0/492756199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc434106ec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:53.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 -- 172.21.15.37:0/492756199 >> 172.21.15.37:0/492756199 conn(0x7fc434102060 msgr2=0x7fc434104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 -- 172.21.15.37:0/492756199 shutdown_connections 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.161+0000 7fc43a600700 1 -- 172.21.15.37:0/492756199 wait complete. 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.162+0000 7fc43a600700 1 Processor -- start 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.162+0000 7fc43a600700 1 -- start start 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.163+0000 7fc43a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc43419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.163+0000 7fc43a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43410b050 con 0x7fc434106aa0 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.163+0000 7fc433400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc43419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.163+0000 7fc433400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc43419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34358/0 (socket says 172.21.15.37:34358) 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.163+0000 7fc433400700 1 -- 172.21.15.37:0/1569742471 learned_addr learned my addr 172.21.15.37:0/1569742471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.164+0000 7fc433400700 1 -- 172.21.15.37:0/1569742471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43419e2f0 con 0x7fc434106aa0 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.164+0000 7fc433400700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc43419ddb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc424000c00 tx=0x7fc424011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.164+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc424003b60 con 0x7fc434106aa0 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.164+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc42401a910 con 0x7fc434106aa0 2024-03-21T09:00:53.981 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.164+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc424024930 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.164+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43419e580 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.165+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4341a1230 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.166+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fc424022020 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.166+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4340637c0 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.167+0000 7fc42be00700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc418039320 0x7fc41803b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.167+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc42404d600 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.167+0000 7fc432a00700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc418039320 0x7fc41803b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.168+0000 7fc432a00700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc418039320 0x7fc41803b7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc420006fd0 tx=0x7fc420006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.172+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc42400f2e0 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.404+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc43402dc60 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.405+0000 7fc42be00700 1 -- 172.21.15.37:0/1569742471 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc42401ab70 con 0x7fc434106aa0 2024-03-21T09:00:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.408+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc418039320 msgr2=0x7fc41803b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.408+0000 7fc43a600700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc418039320 0x7fc41803b7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc420006fd0 tx=0x7fc420006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.408+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 msgr2=0x7fc43419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.408+0000 7fc43a600700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc43419ddb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc424000c00 tx=0x7fc424011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.408+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 shutdown_connections 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.409+0000 7fc43a600700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc418039320 0x7fc41803b7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.409+0000 7fc43a600700 1 --2- 172.21.15.37:0/1569742471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc434106aa0 0x7fc43419ddb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.409+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 >> 172.21.15.37:0/1569742471 conn(0x7fc434102060 msgr2=0x7fc434102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.409+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 shutdown_connections 2024-03-21T09:00:53.983 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:52.409+0000 7fc43a600700 1 -- 172.21.15.37:0/1569742471 wait complete. 2024-03-21T09:00:54.010 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T09:00:54.011 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-21T09:00:54.037 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T09:00:54.037 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-21T09:00:54.037 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-21T09:00:54.037 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T09:00:54.045 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-03-21T09:00:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037.front.sepia.ceph.com:8443/ 2024-03-21T09:00:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-03-21T09:00:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: Password: ueneqprk4b 2024-03-21T09:00:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:54.046 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config directory 2024-03-21T09:00:54.049 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-03-21T09:00:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.655+0000 7fcfe9a00700 1 Processor -- start 2024-03-21T09:00:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.655+0000 7fcfe9a00700 1 -- start start 2024-03-21T09:00:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.656+0000 7fcfe9a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe410a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.656+0000 7fcfe9a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe410a7c0 con 0x7fcfe4109dd0 2024-03-21T09:00:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.656+0000 7fcfe2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe410a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:55.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.656+0000 7fcfe2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe410a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41000/0 (socket says 172.21.15.37:41000) 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.656+0000 7fcfe2a00700 1 -- 172.21.15.37:0/115751635 learned_addr learned my addr 172.21.15.37:0/115751635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.657+0000 7fcfe2a00700 1 -- 172.21.15.37:0/115751635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe410af80 con 0x7fcfe4109dd0 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.657+0000 7fcfe2a00700 1 --2- 172.21.15.37:0/115751635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe410a1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40098a0 tx=0x7fcfd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838f5cf6e405553e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.657+0000 7fcfe2000700 1 -- 172.21.15.37:0/115751635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4004160 con 0x7fcfe4109dd0 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.657+0000 7fcfe2000700 1 -- 172.21.15.37:0/115751635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4004300 con 0x7fcfe4109dd0 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.657+0000 7fcfe2000700 1 -- 172.21.15.37:0/115751635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4011b20 con 0x7fcfe4109dd0 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.658+0000 7fcfe9a00700 1 -- 172.21.15.37:0/115751635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 msgr2=0x7fcfe410a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.658+0000 7fcfe9a00700 1 --2- 172.21.15.37:0/115751635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe410a1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40098a0 tx=0x7fcfd4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.658+0000 7fcfe9a00700 1 -- 172.21.15.37:0/115751635 shutdown_connections 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.658+0000 7fcfe9a00700 1 --2- 172.21.15.37:0/115751635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe410a1f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:55.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.658+0000 7fcfe9a00700 1 -- 172.21.15.37:0/115751635 >> 172.21.15.37:0/115751635 conn(0x7fcfe407d1c0 msgr2=0x7fcfe407d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.659+0000 7fcfe9a00700 1 -- 172.21.15.37:0/115751635 shutdown_connections 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.659+0000 7fcfe9a00700 1 -- 172.21.15.37:0/115751635 wait complete. 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.660+0000 7fcfe9a00700 1 Processor -- start 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.660+0000 7fcfe9a00700 1 -- start start 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.660+0000 7fcfe9a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.660+0000 7fcfe9a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe410e330 con 0x7fcfe4109dd0 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.661+0000 7fcfe2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.661+0000 7fcfe2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41016/0 (socket says 172.21.15.37:41016) 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.661+0000 7fcfe2a00700 1 -- 172.21.15.37:0/302914534 learned_addr learned my addr 172.21.15.37:0/302914534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.661+0000 7fcfe2a00700 1 -- 172.21.15.37:0/302914534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe419e250 con 0x7fcfe4109dd0 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.661+0000 7fcfe2a00700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe419dd10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4000c00 tx=0x7fcfd4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:55.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.662+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4003b60 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.662+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfd401a910 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.662+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4024930 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.662+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe419e4e0 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.662+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe419e940 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.663+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fcfd4022020 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.663+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe40637c0 con 0x7fcfe4109dd0 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.663+0000 7fcfdbe00700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcfc80392d0 0x7fcfc803b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:55.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.663+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcfd404d540 con 0x7fcfe4109dd0 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.664+0000 7fcfda000700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcfc80392d0 0x7fcfc803b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.664+0000 7fcfda000700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcfc80392d0 0x7fcfc803b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ad30 tx=0x7fcfcc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.669+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcfd400f2e0 con 0x7fcfe4109dd0 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.897+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fcfe41a13a0 con 0x7fcfe4109dd0 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.898+0000 7fcfdbe00700 1 -- 172.21.15.37:0/302914534 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fcfd401ab70 con 0x7fcfe4109dd0 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcfc80392d0 msgr2=0x7fcfc803b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcfc80392d0 0x7fcfc803b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ad30 tx=0x7fcfcc0093f0 comp rx=0 tx=0).stop 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 msgr2=0x7fcfe419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe419dd10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4000c00 tx=0x7fcfd4011c10 comp rx=0 tx=0).stop 2024-03-21T09:00:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 shutdown_connections 2024-03-21T09:00:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcfc80392d0 0x7fcfc803b790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 --2- 172.21.15.37:0/302914534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfe4109dd0 0x7fcfe419dd10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 >> 172.21.15.37:0/302914534 conn(0x7fcfe407d1c0 msgr2=0x7fcfe4107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 shutdown_connections 2024-03-21T09:00:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:54.901+0000 7fcfe9a00700 1 -- 172.21.15.37:0/302914534 wait complete. 2024-03-21T09:00:57.190 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.167+0000 7f3f52200700 1 Processor -- start 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.168+0000 7f3f52200700 1 -- start start 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.168+0000 7f3f52200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 0x7f3f4c108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.168+0000 7f3f52200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c1085e0 con 0x7f3f4c107bf0 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.169+0000 7f3f4b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 0x7f3f4c108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.169+0000 7f3f4b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 0x7f3f4c108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41026/0 (socket says 172.21.15.37:41026) 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.169+0000 7f3f4b400700 1 -- 172.21.15.37:0/1884995179 learned_addr learned my addr 172.21.15.37:0/1884995179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.169+0000 7f3f4b400700 1 -- 172.21.15.37:0/1884995179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c108d50 con 0x7f3f4c107bf0 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.169+0000 7f3f4b400700 1 --2- 172.21.15.37:0/1884995179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 0x7f3f4c108010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0098a0 tx=0x7f3f3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b759faca8b5ae74c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:57.191 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.170+0000 7f3f4a000700 1 -- 172.21.15.37:0/1884995179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c004160 con 0x7f3f4c107bf0 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.170+0000 7f3f4a000700 1 -- 172.21.15.37:0/1884995179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f3c004300 con 0x7f3f4c107bf0 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.170+0000 7f3f4a000700 1 -- 172.21.15.37:0/1884995179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c011b20 con 0x7f3f4c107bf0 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.170+0000 7f3f52200700 1 -- 172.21.15.37:0/1884995179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 msgr2=0x7f3f4c108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.170+0000 7f3f52200700 1 --2- 172.21.15.37:0/1884995179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 0x7f3f4c108010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0098a0 tx=0x7f3f3c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.171+0000 7f3f52200700 1 -- 172.21.15.37:0/1884995179 shutdown_connections 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.171+0000 7f3f52200700 1 --2- 172.21.15.37:0/1884995179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c107bf0 0x7f3f4c108010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.171+0000 7f3f52200700 1 -- 172.21.15.37:0/1884995179 >> 172.21.15.37:0/1884995179 conn(0x7f3f4c1031d0 msgr2=0x7f3f4c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.171+0000 7f3f52200700 1 -- 172.21.15.37:0/1884995179 shutdown_connections 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.171+0000 7f3f52200700 1 -- 172.21.15.37:0/1884995179 wait complete. 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.172+0000 7f3f52200700 1 Processor -- start 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.172+0000 7f3f52200700 1 -- start start 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.173+0000 7f3f52200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 0x7f3f4c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.173+0000 7f3f52200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c10c100 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.173+0000 7f3f4b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 0x7f3f4c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.173+0000 7f3f4b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 0x7f3f4c19c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41040/0 (socket says 172.21.15.37:41040) 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.173+0000 7f3f4b400700 1 -- 172.21.15.37:0/2786496108 learned_addr learned my addr 172.21.15.37:0/2786496108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.173+0000 7f3f4b400700 1 -- 172.21.15.37:0/2786496108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c19c610 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.174+0000 7f3f4b400700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 0x7f3f4c19c0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.174+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c0036a0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.174+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f3c01a460 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.174+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c01b440 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.174+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c19c8a0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.174+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c19f370 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.175+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3f3c01a980 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.175+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c0637c0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.176+0000 7f3f43e00700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3f300392d0 0x7f3f3003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.176+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3f3c04daa0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.176+0000 7f3f4aa00700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3f300392d0 0x7f3f3003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.177+0000 7f3f4aa00700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3f300392d0 0x7f3f3003b790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.181+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f3c022be0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.528+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3f4c19faf0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.531+0000 7f3f43e00700 1 -- 172.21.15.37:0/2786496108 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3f3c01a6c0 con 0x7f3f4c19bcb0 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3f300392d0 msgr2=0x7f3f3003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3f300392d0 0x7f3f3003b790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).stop 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 msgr2=0x7f3f4c19c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 0x7f3f4c19c0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c011ba0 comp rx=0 tx=0).stop 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 shutdown_connections 2024-03-21T09:00:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3f300392d0 0x7f3f3003b790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 --2- 172.21.15.37:0/2786496108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c19bcb0 0x7f3f4c19c0d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 >> 172.21.15.37:0/2786496108 conn(0x7f3f4c1031d0 msgr2=0x7f3f4c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 shutdown_connections 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-21T09:00:56.534+0000 7f3f52200700 1 -- 172.21.15.37:0/2786496108 wait complete. 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T09:00:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:00:57.196 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-03-21T09:00:57.292 INFO:tasks.cephadm:Fetching config... 2024-03-21T09:00:57.292 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:00:57.293 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T09:00:57.346 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T09:00:57.346 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:00:57.346 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T09:00:57.392 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T09:00:57.393 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:00:57.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T09:00:57.519 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T09:00:57.519 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:00:57.519 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T09:00:57.614 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T09:00:57.615 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T09:00:57.749 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:00:57.788 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:00:57 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2786496108' entity='client.admin' 2024-03-21T09:00:57.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T09:00:57.865 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:00:57.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T09:00:57.929 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDWFjJwTk0JXDNZVM+eD+47FdGJp861NdGsWCKMRGGHLmHn8derFmAj42SOPFAX8H0DpCWjW+FUKysh6/B4ZcarVht7xawCpHd6T+a7nzTDOAk1v2AzE3R0R3LmmPNcSUxn2CHeda0BmX11CgYkCMGroghj6yUkXmgp+ZC+OG230OgjDRlJ6r8L7OTUZWdkY+eSTmt3Ph8LKmV8rG6TVKtmGGF0ti7c+l04vgyHzBFIn9bftigYlx6AnAtg71GXnjkYymoqwgTOYxe8BDUXOx3+tHvr6BmdNJobBR7scXaihm6W8/hsW98ZZtvMpsX0rulPt98BmKLzPi0n4AbPEjakhqoNjmPbORKd0JtEnJzpoFW2p0W4MGzI5sxZKm3pUvcCuApnx4cRRt53eOCAKdfQjbEwhMY05XZq0Lh+4TiH668GpcGcRY7dai5c/BrHB1KhPtaQ6DLTeZJK1WMQfXLO9TXRYu7U03jPhN0lMkEeRbPr7vFCib/IxtDercwmwc= ceph-2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:00:57.944 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T09:00:58.380 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:01.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 -- 172.21.15.37:0/455030338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 msgr2=0x7fe214104220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:01.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 --2- 172.21.15.37:0/455030338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe214104220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe204009910 tx=0x7fe204009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 -- 172.21.15.37:0/455030338 shutdown_connections 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 --2- 172.21.15.37:0/455030338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe214104220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 -- 172.21.15.37:0/455030338 >> 172.21.15.37:0/455030338 conn(0x7fe2140ff3c0 msgr2=0x7fe2141017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 -- 172.21.15.37:0/455030338 shutdown_connections 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.029+0000 7fe21ac00700 1 -- 172.21.15.37:0/455030338 wait complete. 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe21ac00700 1 Processor -- start 2024-03-21T09:01:01.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe21ac00700 1 -- start start 2024-03-21T09:01:01.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe21ac00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe2141994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:01.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe21ac00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe214109e10 con 0x7fe214103e00 2024-03-21T09:01:01.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe213e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe2141994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:01.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe213e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe2141994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41058/0 (socket says 172.21.15.37:41058) 2024-03-21T09:01:01.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.031+0000 7fe213e00700 1 -- 172.21.15.37:0/3477576906 learned_addr learned my addr 172.21.15.37:0/3477576906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:01.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.032+0000 7fe213e00700 1 -- 172.21.15.37:0/3477576906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2141999e0 con 0x7fe214103e00 2024-03-21T09:01:01.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.032+0000 7fe213e00700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe2141994a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe204005950 tx=0x7fe204005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:01.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.032+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe204026070 con 0x7fe214103e00 2024-03-21T09:01:01.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.032+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe20400bc30 con 0x7fe214103e00 2024-03-21T09:01:01.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.032+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe214199c70 con 0x7fe214103e00 2024-03-21T09:01:01.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.032+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe204003b30 con 0x7fe214103e00 2024-03-21T09:01:01.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.033+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21419c960 con 0x7fe214103e00 2024-03-21T09:01:01.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.034+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fe204029070 con 0x7fe214103e00 2024-03-21T09:01:01.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.034+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21410a590 con 0x7fe214103e00 2024-03-21T09:01:01.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.034+0000 7fe210c00700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe1f8039280 0x7fe1f803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:01.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.034+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe20401e070 con 0x7fe214103e00 2024-03-21T09:01:01.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.035+0000 7fe213400700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe1f8039280 0x7fe1f803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:01.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.035+0000 7fe213400700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe1f8039280 0x7fe1f803b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe200006fd0 tx=0x7fe200006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:01.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.040+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe204013070 con 0x7fe214103e00 2024-03-21T09:01:01.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.268+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fe21402db40 con 0x7fe214103e00 2024-03-21T09:01:01.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.273+0000 7fe210c00700 1 -- 172.21.15.37:0/3477576906 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe204030020 con 0x7fe214103e00 2024-03-21T09:01:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe1f8039280 msgr2=0x7fe1f803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe1f8039280 0x7fe1f803b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe200006fd0 tx=0x7fe200006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 msgr2=0x7fe2141994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe2141994a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe204005950 tx=0x7fe204005030 comp rx=0 tx=0).stop 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 shutdown_connections 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe1f8039280 0x7fe1f803b740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 --2- 172.21.15.37:0/3477576906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe214103e00 0x7fe2141994a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.276+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 >> 172.21.15.37:0/3477576906 conn(0x7fe2140ff3c0 msgr2=0x7fe2140ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.277+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 shutdown_connections 2024-03-21T09:01:01.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:01.277+0000 7fe21ac00700 1 -- 172.21.15.37:0/3477576906 wait complete. 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3477576906' entity='client.admin' 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:01.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:02.101 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T09:01:02.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T09:01:02.553 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:04.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 -- 172.21.15.37:0/3640098345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 msgr2=0x7f22cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:04.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 --2- 172.21.15.37:0/3640098345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc104210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009910 tx=0x7f22bc009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:04.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 -- 172.21.15.37:0/3640098345 shutdown_connections 2024-03-21T09:01:04.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 --2- 172.21.15.37:0/3640098345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc104210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:04.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 -- 172.21.15.37:0/3640098345 >> 172.21.15.37:0/3640098345 conn(0x7f22cc0ff3b0 msgr2=0x7f22cc1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:04.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 -- 172.21.15.37:0/3640098345 shutdown_connections 2024-03-21T09:01:04.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.674+0000 7f22d3000700 1 -- 172.21.15.37:0/3640098345 wait complete. 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.675+0000 7f22d3000700 1 Processor -- start 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.675+0000 7f22d3000700 1 -- start start 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.675+0000 7f22d3000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.675+0000 7f22d3000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc109e80 con 0x7f22cc103df0 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.676+0000 7f22cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.676+0000 7f22cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38518/0 (socket says 172.21.15.37:38518) 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.676+0000 7f22cbe00700 1 -- 172.21.15.37:0/3824780937 learned_addr learned my addr 172.21.15.37:0/3824780937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.676+0000 7f22cbe00700 1 -- 172.21.15.37:0/3824780937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc199a60 con 0x7f22cc103df0 2024-03-21T09:01:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.676+0000 7f22cbe00700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc199520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22bc006010 tx=0x7f22bc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:04.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.677+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22bc027070 con 0x7f22cc103df0 2024-03-21T09:01:04.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.677+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22bc00be40 con 0x7f22cc103df0 2024-03-21T09:01:04.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.677+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22bc00f460 con 0x7f22cc103df0 2024-03-21T09:01:04.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.677+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc199cf0 con 0x7f22cc103df0 2024-03-21T09:01:04.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.677+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc19c990 con 0x7f22cc103df0 2024-03-21T09:01:04.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.678+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f22bc02a070 con 0x7f22cc103df0 2024-03-21T09:01:04.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.678+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc10a600 con 0x7f22cc103df0 2024-03-21T09:01:04.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.678+0000 7f22c8c00700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f22b0039280 0x7f22b003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:04.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.679+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f22bc01e030 con 0x7f22cc103df0 2024-03-21T09:01:04.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.679+0000 7f22cb400700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f22b0039280 0x7f22b003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:04.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.680+0000 7f22cb400700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f22b0039280 0x7f22b003b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:04.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.684+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22bc013070 con 0x7f22cc103df0 2024-03-21T09:01:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.915+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f22cc02db40 con 0x7f22b0039280 2024-03-21T09:01:04.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.923+0000 7f22c8c00700 1 -- 172.21.15.37:0/3824780937 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22cc02db40 con 0x7f22b0039280 2024-03-21T09:01:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f22b0039280 msgr2=0x7f22b003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f22b0039280 0x7f22b003b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 msgr2=0x7f22cc199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc199520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22bc006010 tx=0x7f22bc005050 comp rx=0 tx=0).stop 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 shutdown_connections 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f22b0039280 0x7f22b003b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 --2- 172.21.15.37:0/3824780937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22cc103df0 0x7f22cc199520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 >> 172.21.15.37:0/3824780937 conn(0x7f22cc0ff3b0 msgr2=0x7f22cc0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 shutdown_connections 2024-03-21T09:01:04.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:04.926+0000 7f22d3000700 1 -- 172.21.15.37:0/3824780937 wait complete. 2024-03-21T09:01:05.749 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-03-21T09:01:05.750 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T09:01:05.750 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-03-21T09:01:05.768 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T09:01:05.769 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:05.827 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-03-21T09:01:05.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch host add smithi053 2024-03-21T09:01:06.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:05 smithi037 ceph-mon[31830]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:06.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:06.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:06.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:06.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:06.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:05 smithi037 ceph-mon[31830]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-21T09:01:06.270 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:07.115 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:06 smithi037 ceph-mon[31830]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:07.115 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:06 smithi037 ceph-mon[31830]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:07.115 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:06 smithi037 ceph-mon[31830]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:08.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:07 smithi037 ceph-mon[31830]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:08.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:08.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:08.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:08.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 -- 172.21.15.37:0/2066270411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 msgr2=0x7f1518107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:08.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 --2- 172.21.15.37:0/2066270411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518107db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1508009910 tx=0x7f1508009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:08.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 -- 172.21.15.37:0/2066270411 shutdown_connections 2024-03-21T09:01:08.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 --2- 172.21.15.37:0/2066270411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518107db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:08.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 -- 172.21.15.37:0/2066270411 >> 172.21.15.37:0/2066270411 conn(0x7f15180fd500 msgr2=0x7f15180ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:08.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 -- 172.21.15.37:0/2066270411 shutdown_connections 2024-03-21T09:01:08.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.201+0000 7f1517e00700 1 -- 172.21.15.37:0/2066270411 wait complete. 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.202+0000 7f1517e00700 1 Processor -- start 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.202+0000 7f1517e00700 1 -- start start 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.203+0000 7f1517e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.203+0000 7f1517e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151810c010 con 0x7f15181079d0 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.203+0000 7f1516a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.203+0000 7f1516a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38538/0 (socket says 172.21.15.37:38538) 2024-03-21T09:01:08.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.203+0000 7f1516a00700 1 -- 172.21.15.37:0/669239435 learned_addr learned my addr 172.21.15.37:0/669239435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f1516a00700 1 -- 172.21.15.37:0/669239435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518199bd0 con 0x7f15181079d0 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f1516a00700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518199690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1508006010 tx=0x7f1508005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1508026070 con 0x7f15181079d0 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f150800bbd0 con 0x7f15181079d0 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1508003910 con 0x7f15181079d0 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1518199ec0 con 0x7f15181079d0 2024-03-21T09:01:08.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.204+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151819df60 con 0x7f15181079d0 2024-03-21T09:01:08.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.205+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1518103110 con 0x7f15181079d0 2024-03-21T09:01:08.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.205+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f1508029070 con 0x7f15181079d0 2024-03-21T09:01:08.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.206+0000 7f150fe00700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f14f8039280 0x7f14f803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:08.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.206+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f150801e070 con 0x7f15181079d0 2024-03-21T09:01:08.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.206+0000 7f150e000700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f14f8039280 0x7f14f803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:08.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.207+0000 7f150e000700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f14f8039280 0x7f14f803b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1500006fd0 tx=0x7f1500006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:08.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.211+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1508013070 con 0x7f15181079d0 2024-03-21T09:01:08.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:08.443+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) v1 -- 0x7f151819a9f0 con 0x7f14f8039280 2024-03-21T09:01:09.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:08 smithi037 ceph-mon[31830]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:09.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:08 smithi037 ceph-mon[31830]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:10.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:09 smithi037 ceph-mon[31830]: Deploying cephadm binary to smithi053 2024-03-21T09:01:11.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:10 smithi037 ceph-mon[31830]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:12 smithi037 ceph-mon[31830]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:13.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.494+0000 7f150fe00700 1 -- 172.21.15.37:0/669239435 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f151819a9f0 con 0x7f14f8039280 2024-03-21T09:01:13.490 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-03-21T09:01:13.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f14f8039280 msgr2=0x7f14f803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:13.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f14f8039280 0x7f14f803b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1500006fd0 tx=0x7f1500006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 msgr2=0x7f1518199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518199690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1508006010 tx=0x7f1508005030 comp rx=0 tx=0).stop 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 shutdown_connections 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f14f8039280 0x7f14f803b740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 --2- 172.21.15.37:0/669239435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15181079d0 0x7f1518199690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.498+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 >> 172.21.15.37:0/669239435 conn(0x7f15180fd500 msgr2=0x7f1518075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.499+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 shutdown_connections 2024-03-21T09:01:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:13.499+0000 7f1517e00700 1 -- 172.21.15.37:0/669239435 wait complete. 2024-03-21T09:01:14.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T09:01:14.746 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:14.900 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:14.900 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:14 smithi037 ceph-mon[31830]: Added host smithi053 2024-03-21T09:01:14.900 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:14.900 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:15.689 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:15 smithi037 ceph-mon[31830]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:15.689 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:16.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 -- 172.21.15.37:0/3683795688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 msgr2=0x7f44cc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:16.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 --2- 172.21.15.37:0/3683795688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1021b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009910 tx=0x7f44bc009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:16.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 -- 172.21.15.37:0/3683795688 shutdown_connections 2024-03-21T09:01:16.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 --2- 172.21.15.37:0/3683795688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1021b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:16.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 -- 172.21.15.37:0/3683795688 >> 172.21.15.37:0/3683795688 conn(0x7f44cc0fd6a0 msgr2=0x7f44cc0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:16.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 -- 172.21.15.37:0/3683795688 shutdown_connections 2024-03-21T09:01:16.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.779+0000 7f44d0c00700 1 -- 172.21.15.37:0/3683795688 wait complete. 2024-03-21T09:01:16.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.780+0000 7f44d0c00700 1 Processor -- start 2024-03-21T09:01:16.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.781+0000 7f44d0c00700 1 -- start start 2024-03-21T09:01:16.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.781+0000 7f44d0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:16.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.781+0000 7f44d0c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc10c1a0 con 0x7f44cc101dd0 2024-03-21T09:01:16.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.781+0000 7f44c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:16.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.782+0000 7f44c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34306/0 (socket says 172.21.15.37:34306) 2024-03-21T09:01:16.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.782+0000 7f44c9e00700 1 -- 172.21.15.37:0/3526659802 learned_addr learned my addr 172.21.15.37:0/3526659802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:16.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.782+0000 7f44c9e00700 1 -- 172.21.15.37:0/3526659802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc197be0 con 0x7f44cc101dd0 2024-03-21T09:01:16.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.782+0000 7f44c9e00700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1976a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.782+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44bc026070 con 0x7f44cc101dd0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.782+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44bc00bbd0 con 0x7f44cc101dd0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.783+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc197ed0 con 0x7f44cc101dd0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.783+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44bc003910 con 0x7f44cc101dd0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.783+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc19bf70 con 0x7f44cc101dd0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.784+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc1032b0 con 0x7f44cc101dd0 2024-03-21T09:01:16.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.784+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f44bc029070 con 0x7f44cc101dd0 2024-03-21T09:01:16.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.784+0000 7f44c2a00700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f44b0039280 0x7f44b003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:16.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.784+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f44bc01e070 con 0x7f44cc101dd0 2024-03-21T09:01:16.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.784+0000 7f44c9400700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f44b0039280 0x7f44b003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:16.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.785+0000 7f44c9400700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f44b0039280 0x7f44b003b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f44b8006fd0 tx=0x7f44b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:16.790+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f44bc013070 con 0x7f44cc101dd0 2024-03-21T09:01:17.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.018+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44cc198a00 con 0x7f44b0039280 2024-03-21T09:01:17.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.021+0000 7f44c2a00700 1 -- 172.21.15.37:0/3526659802 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f44cc198a00 con 0x7f44b0039280 2024-03-21T09:01:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:01:17.016 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.023+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f44b0039280 msgr2=0x7f44b003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.023+0000 7f44d0c00700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f44b0039280 0x7f44b003b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f44b8006fd0 tx=0x7f44b8006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.023+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 msgr2=0x7f44cc1976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.023+0000 7f44d0c00700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1976a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc005030 comp rx=0 tx=0).stop 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.024+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 shutdown_connections 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.024+0000 7f44d0c00700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f44b0039280 0x7f44b003b740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.024+0000 7f44d0c00700 1 --2- 172.21.15.37:0/3526659802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44cc101dd0 0x7f44cc1976a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.024+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 >> 172.21.15.37:0/3526659802 conn(0x7f44cc0fd6a0 msgr2=0x7f44cc1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.024+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 shutdown_connections 2024-03-21T09:01:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:17.024+0000 7f44d0c00700 1 -- 172.21.15.37:0/3526659802 wait complete. 2024-03-21T09:01:17.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:17 smithi037 ceph-mon[31830]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:17.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:17.736 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-03-21T09:01:17.737 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T09:01:17.737 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-03-21T09:01:17.756 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T09:01:17.756 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:17.814 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-03-21T09:01:17.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch host add smithi071 2024-03-21T09:01:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:18.544 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:18 smithi037 ceph-mon[31830]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:01:19.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:19 smithi037 ceph-mon[31830]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.527+0000 7efd4e400700 1 -- 172.21.15.37:0/3742371361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 msgr2=0x7efd481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.527+0000 7efd4e400700 1 --2- 172.21.15.37:0/3742371361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd481043d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd38009910 tx=0x7efd38009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.528+0000 7efd4e400700 1 -- 172.21.15.37:0/3742371361 shutdown_connections 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.528+0000 7efd4e400700 1 --2- 172.21.15.37:0/3742371361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd481043d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.528+0000 7efd4e400700 1 -- 172.21.15.37:0/3742371361 >> 172.21.15.37:0/3742371361 conn(0x7efd480ff840 msgr2=0x7efd48101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.528+0000 7efd4e400700 1 -- 172.21.15.37:0/3742371361 shutdown_connections 2024-03-21T09:01:20.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.528+0000 7efd4e400700 1 -- 172.21.15.37:0/3742371361 wait complete. 2024-03-21T09:01:20.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.529+0000 7efd4e400700 1 Processor -- start 2024-03-21T09:01:20.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.530+0000 7efd4e400700 1 -- start start 2024-03-21T09:01:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.530+0000 7efd4e400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd4819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.530+0000 7efd4e400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4810e3c0 con 0x7efd48103ff0 2024-03-21T09:01:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.530+0000 7efd47400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd4819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.530+0000 7efd47400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd4819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34330/0 (socket says 172.21.15.37:34330) 2024-03-21T09:01:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.530+0000 7efd47400700 1 -- 172.21.15.37:0/4106385708 learned_addr learned my addr 172.21.15.37:0/4106385708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.531+0000 7efd47400700 1 -- 172.21.15.37:0/4106385708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4819fba0 con 0x7efd48103ff0 2024-03-21T09:01:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.531+0000 7efd47400700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd4819f660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd38005850 tx=0x7efd38005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.531+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd38026070 con 0x7efd48103ff0 2024-03-21T09:01:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.532+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd3800bbd0 con 0x7efd48103ff0 2024-03-21T09:01:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.532+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4819fe90 con 0x7efd48103ff0 2024-03-21T09:01:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.532+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd38003970 con 0x7efd48103ff0 2024-03-21T09:01:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.532+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd48199900 con 0x7efd48103ff0 2024-03-21T09:01:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.533+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7efd38029070 con 0x7efd48103ff0 2024-03-21T09:01:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.533+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd481054d0 con 0x7efd48103ff0 2024-03-21T09:01:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.533+0000 7efd3fe00700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd2c039280 0x7efd2c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.534+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd3801e070 con 0x7efd48103ff0 2024-03-21T09:01:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.534+0000 7efd46a00700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd2c039280 0x7efd2c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.534+0000 7efd46a00700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd2c039280 0x7efd2c03b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd34006fd0 tx=0x7efd34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.539+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd38013070 con 0x7efd48103ff0 2024-03-21T09:01:20.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:20.776+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) v1 -- 0x7efd4819ac60 con 0x7efd2c039280 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: Updating smithi053:/etc/ceph/ceph.conf 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:20.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:20 smithi037 ceph-mon[31830]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:22.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:22 smithi037 ceph-mon[31830]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:22.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:22 smithi037 ceph-mon[31830]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:22.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:22.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:22.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:23.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:23 smithi037 ceph-mon[31830]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:23.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:23 smithi037 ceph-mon[31830]: Deploying cephadm binary to smithi071 2024-03-21T09:01:25.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:25 smithi037 ceph-mon[31830]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:25.777 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-03-21T09:01:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.781+0000 7efd3fe00700 1 -- 172.21.15.37:0/4106385708 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7efd4819ac60 con 0x7efd2c039280 2024-03-21T09:01:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd2c039280 msgr2=0x7efd2c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd2c039280 0x7efd2c03b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd34006fd0 tx=0x7efd34006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 msgr2=0x7efd4819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd4819f660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd38005850 tx=0x7efd38005030 comp rx=0 tx=0).stop 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 shutdown_connections 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd2c039280 0x7efd2c03b740 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 --2- 172.21.15.37:0/4106385708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd48103ff0 0x7efd4819f660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.785+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 >> 172.21.15.37:0/4106385708 conn(0x7efd480ff840 msgr2=0x7efd48106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.786+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 shutdown_connections 2024-03-21T09:01:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:25.786+0000 7efd4e400700 1 -- 172.21.15.37:0/4106385708 wait complete. 2024-03-21T09:01:26.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T09:01:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:27.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:26 smithi037 ceph-mon[31830]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:27.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:27.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:26 smithi037 ceph-mon[31830]: Added host smithi071 2024-03-21T09:01:27.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:27.078 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:28.087 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:27 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:29.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 -- 172.21.15.37:0/3944607623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 msgr2=0x7f241c074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 --2- 172.21.15.37:0/3944607623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c074890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2410009910 tx=0x7f2410009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 -- 172.21.15.37:0/3944607623 shutdown_connections 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 --2- 172.21.15.37:0/3944607623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c074890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 -- 172.21.15.37:0/3944607623 >> 172.21.15.37:0/3944607623 conn(0x7f241c0fd6d0 msgr2=0x7f241c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 -- 172.21.15.37:0/3944607623 shutdown_connections 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.005+0000 7f2424400700 1 -- 172.21.15.37:0/3944607623 wait complete. 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.006+0000 7f2424400700 1 Processor -- start 2024-03-21T09:01:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.006+0000 7f2424400700 1 -- start start 2024-03-21T09:01:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.007+0000 7f2424400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.007+0000 7f2424400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c10e3f0 con 0x7f241c0744b0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.007+0000 7f2421a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.007+0000 7f2421a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58034/0 (socket says 172.21.15.37:58034) 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.007+0000 7f2421a00700 1 -- 172.21.15.37:0/1120883766 learned_addr learned my addr 172.21.15.37:0/1120883766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.007+0000 7f2421a00700 1 -- 172.21.15.37:0/1120883766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c19fbc0 con 0x7f241c0744b0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.008+0000 7f2421a00700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c19f680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2410006010 tx=0x7f2410005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.008+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2410026070 con 0x7f241c0744b0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.008+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f241000bc30 con 0x7f241c0744b0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.008+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24100039a0 con 0x7f241c0744b0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.008+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c19fe50 con 0x7f241c0744b0 2024-03-21T09:01:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.008+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c199750 con 0x7f241c0744b0 2024-03-21T09:01:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.009+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c075bf0 con 0x7f241c0744b0 2024-03-21T09:01:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.010+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f2410029070 con 0x7f241c0744b0 2024-03-21T09:01:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.010+0000 7f241a000700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2404039280 0x7f240403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.010+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f241001e070 con 0x7f241c0744b0 2024-03-21T09:01:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.010+0000 7f2421000700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2404039280 0x7f240403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.011+0000 7f2421000700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2404039280 0x7f240403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2408006fd0 tx=0x7f2408006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.016+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2410013070 con 0x7f241c0744b0 2024-03-21T09:01:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.253+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f241c19ac80 con 0x7f2404039280 2024-03-21T09:01:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.255+0000 7f241a000700 1 -- 172.21.15.37:0/1120883766 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f241c19ac80 con 0x7f2404039280 2024-03-21T09:01:29.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:01:29.250 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-03-21T09:01:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2404039280 msgr2=0x7f240403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2404039280 0x7f240403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2408006fd0 tx=0x7f2408006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 msgr2=0x7f241c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c19f680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2410006010 tx=0x7f2410005030 comp rx=0 tx=0).stop 2024-03-21T09:01:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 shutdown_connections 2024-03-21T09:01:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2404039280 0x7f240403b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 --2- 172.21.15.37:0/1120883766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c0744b0 0x7f241c19f680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 >> 172.21.15.37:0/1120883766 conn(0x7f241c0fd6d0 msgr2=0x7f241c105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 shutdown_connections 2024-03-21T09:01:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:29.258+0000 7f2424400700 1 -- 172.21.15.37:0/1120883766 wait complete. 2024-03-21T09:01:29.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:29 smithi037 ceph-mon[31830]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:29.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:29.989 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T09:01:29.990 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T09:01:30.434 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:30.881 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:30 smithi037 ceph-mon[31830]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:01:31.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:31 smithi037 ceph-mon[31830]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 -- 172.21.15.37:0/268785117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 msgr2=0x7fc4501021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 --2- 172.21.15.37:0/268785117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc4501021d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc444009910 tx=0x7fc444009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 -- 172.21.15.37:0/268785117 shutdown_connections 2024-03-21T09:01:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 --2- 172.21.15.37:0/268785117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc4501021d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 -- 172.21.15.37:0/268785117 >> 172.21.15.37:0/268785117 conn(0x7fc4500fd6a0 msgr2=0x7fc4500ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 -- 172.21.15.37:0/268785117 shutdown_connections 2024-03-21T09:01:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.511+0000 7fc457c00700 1 -- 172.21.15.37:0/268785117 wait complete. 2024-03-21T09:01:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.512+0000 7fc457c00700 1 Processor -- start 2024-03-21T09:01:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.513+0000 7fc457c00700 1 -- start start 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.513+0000 7fc457c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc450197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.513+0000 7fc457c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45010c1c0 con 0x7fc450101df0 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.513+0000 7fc455200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc450197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.513+0000 7fc455200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc450197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58050/0 (socket says 172.21.15.37:58050) 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.513+0000 7fc455200700 1 -- 172.21.15.37:0/1125262657 learned_addr learned my addr 172.21.15.37:0/1125262657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.514+0000 7fc455200700 1 -- 172.21.15.37:0/1125262657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc450197b90 con 0x7fc450101df0 2024-03-21T09:01:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.514+0000 7fc455200700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc450197650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc444006010 tx=0x7fc444005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.514+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc444026070 con 0x7fc450101df0 2024-03-21T09:01:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.514+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc44400bbd0 con 0x7fc450101df0 2024-03-21T09:01:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.514+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc444003910 con 0x7fc450101df0 2024-03-21T09:01:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.514+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc450197e80 con 0x7fc450101df0 2024-03-21T09:01:32.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.515+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45019bf20 con 0x7fc450101df0 2024-03-21T09:01:32.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.516+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4501032d0 con 0x7fc450101df0 2024-03-21T09:01:32.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.516+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fc444029070 con 0x7fc450101df0 2024-03-21T09:01:32.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.516+0000 7fc44d600700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc438039280 0x7fc43803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:32.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.516+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc44401e070 con 0x7fc450101df0 2024-03-21T09:01:32.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.517+0000 7fc44fe00700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc438039280 0x7fc43803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:32.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.517+0000 7fc44fe00700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc438039280 0x7fc43803b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006fd0 tx=0x7fc43c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.522+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc444013070 con 0x7fc450101df0 2024-03-21T09:01:32.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:32.758+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc4501989b0 con 0x7fc450101df0 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:32.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:32.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-21T09:01:32.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:32.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:32 smithi037 ceph-mon[31830]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.676+0000 7fc44d600700 1 -- 172.21.15.37:0/1125262657 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc444030090 con 0x7fc450101df0 2024-03-21T09:01:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc438039280 msgr2=0x7fc43803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc438039280 0x7fc43803b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006fd0 tx=0x7fc43c006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 msgr2=0x7fc450197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc450197650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc444006010 tx=0x7fc444005030 comp rx=0 tx=0).stop 2024-03-21T09:01:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 shutdown_connections 2024-03-21T09:01:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fc438039280 0x7fc43803b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 --2- 172.21.15.37:0/1125262657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc450101df0 0x7fc450197650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.680+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 >> 172.21.15.37:0/1125262657 conn(0x7fc4500fd6a0 msgr2=0x7fc4501037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.681+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 shutdown_connections 2024-03-21T09:01:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:33.681+0000 7fc457c00700 1 -- 172.21.15.37:0/1125262657 wait complete. 2024-03-21T09:01:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-03-21T09:01:33.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:33 smithi037 ceph-mon[31830]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:33 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1125262657' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T09:01:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:34.457 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-03-21T09:01:34.457 INFO:tasks.cephadm:Adding mon.b on smithi053 2024-03-21T09:01:34.457 INFO:tasks.cephadm:Adding mon.c on smithi071 2024-03-21T09:01:34.457 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c' 2024-03-21T09:01:34.694 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:34.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:34 smithi037 ceph-mon[31830]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:34.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:34 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1125262657' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T09:01:34.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:34 smithi037 ceph-mon[31830]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 -- 172.21.15.71:0/214144125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 msgr2=0x7f3868106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 --2- 172.21.15.71:0/214144125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f3868106e40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3858009910 tx=0x7f3858009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 -- 172.21.15.71:0/214144125 shutdown_connections 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 --2- 172.21.15.71:0/214144125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f3868106e40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 -- 172.21.15.71:0/214144125 >> 172.21.15.71:0/214144125 conn(0x7f38680776d0 msgr2=0x7f38680fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 -- 172.21.15.71:0/214144125 shutdown_connections 2024-03-21T09:01:35.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.974+0000 7f386d2b8700 1 -- 172.21.15.71:0/214144125 wait complete. 2024-03-21T09:01:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.975+0000 7f386d2b8700 1 Processor -- start 2024-03-21T09:01:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.975+0000 7f386d2b8700 1 -- start start 2024-03-21T09:01:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.975+0000 7f386d2b8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f38680fca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.975+0000 7f386d2b8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386810b0a0 con 0x7f3868106a60 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.976+0000 7f3866ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f38680fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.976+0000 7f3866ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f38680fca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.71:50762/0 (socket says 172.21.15.71:50762) 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.976+0000 7f3866ffd700 1 -- 172.21.15.71:0/4269893199 learned_addr learned my addr 172.21.15.71:0/4269893199 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.976+0000 7f3866ffd700 1 -- 172.21.15.71:0/4269893199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680fe950 con 0x7f3868106a60 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.976+0000 7f3866ffd700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f38680fca60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3858005850 tx=0x7f3858005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.977+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3858026070 con 0x7f3868106a60 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.977+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38680fd000 con 0x7f3868106a60 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.977+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38680fd4c0 con 0x7f3868106a60 2024-03-21T09:01:35.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.977+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f385800bbd0 con 0x7f3868106a60 2024-03-21T09:01:35.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.977+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38580039b0 con 0x7f3868106a60 2024-03-21T09:01:35.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.978+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38681021a0 con 0x7f3868106a60 2024-03-21T09:01:35.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.978+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3858029070 con 0x7f3868106a60 2024-03-21T09:01:35.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.978+0000 7f384ffff700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3850039220 0x7f385003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:35.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.978+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f385801e070 con 0x7f3868106a60 2024-03-21T09:01:35.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.979+0000 7f38667fc700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3850039220 0x7f385003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:35.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.980+0000 7f38667fc700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3850039220 0x7f385003b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f385c006fd0 tx=0x7f385c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:35.993+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3858013070 con 0x7f3868106a60 2024-03-21T09:01:36.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.200+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f38680fe000 con 0x7f3850039220 2024-03-21T09:01:36.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.216+0000 7f384ffff700 1 -- 172.21.15.71:0/4269893199 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f38680fe000 con 0x7f3850039220 2024-03-21T09:01:36.210 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mon update... 2024-03-21T09:01:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3850039220 msgr2=0x7f385003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3850039220 0x7f385003b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f385c006fd0 tx=0x7f385c006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 msgr2=0x7f38680fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f38680fca60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3858005850 tx=0x7f3858005030 comp rx=0 tx=0).stop 2024-03-21T09:01:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 shutdown_connections 2024-03-21T09:01:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3850039220 0x7f385003b6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 --2- 172.21.15.71:0/4269893199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3868106a60 0x7f38680fca60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 >> 172.21.15.71:0/4269893199 conn(0x7f38680776d0 msgr2=0x7f3868069580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 shutdown_connections 2024-03-21T09:01:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:36.217+0000 7f386d2b8700 1 -- 172.21.15.71:0/4269893199 wait complete. 2024-03-21T09:01:36.848 DEBUG:teuthology.orchestra.run.smithi053:mon.b> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T09:01:36.851 DEBUG:teuthology.orchestra.run.smithi071:mon.c> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T09:01:36.853 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T09:01:36.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:01:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:37.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:37.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c;count:3 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:37 smithi037 ceph-mon[31830]: Deploying daemon mon.c on smithi071 2024-03-21T09:01:39.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:39 smithi037 ceph-mon[31830]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:39.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:39 smithi071 systemd[1]: Starting Ceph mon.c for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.232+0000 7f2813542700 1 -- 172.21.15.71:0/1041315840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 msgr2=0x7f280c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.232+0000 7f2813542700 1 --2- 172.21.15.71:0/1041315840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c1031a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2808009910 tx=0x7f2808009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 -- 172.21.15.71:0/1041315840 shutdown_connections 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 --2- 172.21.15.71:0/1041315840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c1031a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 -- 172.21.15.71:0/1041315840 >> 172.21.15.71:0/1041315840 conn(0x7f280c0fc550 msgr2=0x7f280c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 -- 172.21.15.71:0/1041315840 shutdown_connections 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 -- 172.21.15.71:0/1041315840 wait complete. 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 Processor -- start 2024-03-21T09:01:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.233+0000 7f2813542700 1 -- start start 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.234+0000 7f2813542700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.234+0000 7f2813542700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c10c210 con 0x7f280c102dc0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.234+0000 7f28112de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.234+0000 7f28112de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.71:50790/0 (socket says 172.21.15.71:50790) 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.234+0000 7f28112de700 1 -- 172.21.15.71:0/2746161353 learned_addr learned my addr 172.21.15.71:0/2746161353 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.235+0000 7f28112de700 1 -- 172.21.15.71:0/2746161353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c102910 con 0x7f280c102dc0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.235+0000 7f28112de700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c100a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2808006010 tx=0x7f2808005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.236+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2808026070 con 0x7f280c102dc0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.236+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c100f60 con 0x7f280c102dc0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.236+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c101420 con 0x7f280c102dc0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.236+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f280800bc30 con 0x7f280c102dc0 2024-03-21T09:01:40.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.236+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2808003a50 con 0x7f280c102dc0 2024-03-21T09:01:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.237+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c06a7b0 con 0x7f280c102dc0 2024-03-21T09:01:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.237+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f2808029070 con 0x7f280c102dc0 2024-03-21T09:01:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.237+0000 7f28027fc700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27f8039220 0x7f27f803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.237+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f280801e070 con 0x7f280c102dc0 2024-03-21T09:01:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.238+0000 7f2810add700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27f8039220 0x7f27f803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.239+0000 7f2810add700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27f8039220 0x7f27f803b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006fd0 tx=0x7f27fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:40.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.253+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2808013070 con 0x7f280c102dc0 2024-03-21T09:01:40.527 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 podman[36559]: 2024-03-21 09:01:40.323136741 +0000 UTC m=+0.095608096 container create afdc6583bd51cf60542b22daf8a40cc10b6d054d5952d518167765fcfebd4b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True) 2024-03-21T09:01:40.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 podman[36559]: 2024-03-21 09:01:40.244768929 +0000 UTC m=+0.017240302 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:01:40.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 podman[36559]: 2024-03-21 09:01:40.533627608 +0000 UTC m=+0.306098963 container init afdc6583bd51cf60542b22daf8a40cc10b6d054d5952d518167765fcfebd4b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T09:01:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.548+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f280c101f50 con 0x7f280c102dc0 2024-03-21T09:01:40.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.550+0000 7f28027fc700 1 -- 172.21.15.71:0/2746161353 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f280800de00 con 0x7f280c102dc0 2024-03-21T09:01:40.545 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T09:01:40.545 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","modified":"2024-03-21T08:59:24.797606Z","created":"2024-03-21T08:59:24.797606Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:01:40.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27f8039220 msgr2=0x7f27f803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:40.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27f8039220 0x7f27f803b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006fd0 tx=0x7f27fc006e40 comp rx=0 tx=0).stop 2024-03-21T09:01:40.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 msgr2=0x7f280c100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c100a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2808006010 tx=0x7f2808005050 comp rx=0 tx=0).stop 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 shutdown_connections 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27f8039220 0x7f27f803b6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 --2- 172.21.15.71:0/2746161353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c102dc0 0x7f280c100a20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 >> 172.21.15.71:0/2746161353 conn(0x7f280c0fc550 msgr2=0x7f280c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 shutdown_connections 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:40.552+0000 7f2813542700 1 -- 172.21.15.71:0/2746161353 wait complete. 2024-03-21T09:01:40.548 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 podman[36559]: 2024-03-21 09:01:40.53659461 +0000 UTC m=+0.309065959 container start afdc6583bd51cf60542b22daf8a40cc10b6d054d5952d518167765fcfebd4b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pidfile_write: ignore empty --pid-file 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: load: jerasure load: lrc 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: RocksDB version: 7.9.2 2024-03-21T09:01:40.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Git sha 0 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: DB SUMMARY 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: DB Session ID: RYAWQ4V5LXXIM2N3IQMS 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: CURRENT file: CURRENT 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.error_if_exists: 0 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.create_if_missing: 0 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.paranoid_checks: 1 2024-03-21T09:01:40.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.env: 0x556bddbc1b20 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.info_log: 0x556be027b920 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.statistics: (nil) 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.use_fsync: 0 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_log_file_size: 0 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.allow_fallocate: 1 2024-03-21T09:01:40.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.use_direct_reads: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.db_log_dir: 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.wal_dir: 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T09:01:40.857 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.write_buffer_manager: 0x556be024e1e0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.unordered_write: 0 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T09:01:40.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.row_cache: None 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.wal_filter: None 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.two_write_queues: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.wal_compression: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.atomic_flush: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T09:01:40.859 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T09:01:40.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.log_readahead_size: 0 2024-03-21T09:01:40.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T09:01:40.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T09:01:40.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T09:01:40.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_background_jobs: 2 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_background_compactions: -1 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_subcompactions: 1 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T09:01:40.862 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_open_files: -1 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_background_flushes: -1 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Compression algorithms supported: 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kZSTD supported: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kXpressCompression supported: 0 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kZlibCompression supported: 1 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kSnappyCompression supported: 1 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kLZ4Compression supported: 1 2024-03-21T09:01:40.863 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: kBZip2Compression supported: 0 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.merge_operator: 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_filter: None 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556be027ba20) 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-03-21T09:01:40.864 INFO:journalctl@ceph.mon.c.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_type: 0 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: data_block_index_type: 0 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_shortening: 1 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: checksum: 4 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: no_block_cache: 0 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache: 0x556be0281090 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_options: 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: capacity : 536870912 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: num_shard_bits : 4 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: strict_capacity_limit : 0 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-03-21T09:01:40.865 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_compressed: (nil) 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: persistent_cache: (nil) 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_size: 4096 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_size_deviation: 10 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_restart_interval: 16 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_block_restart_interval: 1 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: metadata_block_size: 4096 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: partition_filters: 0 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: use_delta_encoding: 1 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: filter_policy: bloomfilter 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: whole_key_filtering: 1 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: verify_compression: 0 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: format_version: 5 2024-03-21T09:01:40.866 INFO:journalctl@ceph.mon.c.smithi071.stdout: enable_index_compression: 1 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_align: 0 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout: max_auto_readahead_size: 262144 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout: prepopulate_block_cache: 0 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout: initial_auto_readahead_size: 8192 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression: NoCompression 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.num_levels: 7 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T09:01:40.867 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:01:40.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T09:01:40.869 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T09:01:40.870 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.table_properties_collectors: 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.inplace_update_support: 0 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.bloom_locality: 0 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.max_successive_merges: 0 2024-03-21T09:01:40.871 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.ttl: 2592000 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.enable_blob_files: false 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.min_blob_size: 0 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T09:01:40.872 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bc228336-0a8d-43b1-bec2-f09a63c1d813 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011700567374, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011700567808, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711011700, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bc228336-0a8d-43b1-bec2-f09a63c1d813", "db_session_id": "RYAWQ4V5LXXIM2N3IQMS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011700567890, "job": 1, "event": "recovery_finished"} 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556be0374000 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: DB pointer 0x556be035e000 2024-03-21T09:01:40.873 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** DB Stats ** 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T09:01:40.874 INFO:journalctl@ceph.mon.c.smithi071.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T09:01:40.875 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: Block cache BinnedLRUCache@0x556be0281090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: using public_addr v2:172.21.15.71:0/0 -> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: starting mon.c rank -1 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(???) e0 preinit fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:01:40.876 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).mds e1 new map 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout: e1 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout: legacy client fscid: -1 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout: No filesystems configured 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T09:01:40.877 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Added host smithi053 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:01:40.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi053:/etc/ceph/ceph.conf 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:40.879 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Deploying cephadm binary to smithi071 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Added host smithi071 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.880 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:40.881 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1125262657' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1125262657' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c;count:3 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:40.882 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: Deploying daemon mon.c on smithi071 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 ceph-mon[36597]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 bash[36559]: afdc6583bd51cf60542b22daf8a40cc10b6d054d5952d518167765fcfebd4b67 2024-03-21T09:01:40.883 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:40 smithi071 systemd[1]: Started Ceph mon.c for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:01:42.277 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T09:01:42.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:01:42.522 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: mon.a calling monitor election 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: mon.c calling monitor election 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:45.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: monmap e2: 2 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: fsmap 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: mgrmap e13: x(active, since 60s) 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: overall HEALTH_OK 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:46.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: mon.a calling monitor election 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: mon.c calling monitor election 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T09:01:46.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: monmap e2: 2 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: fsmap 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: mgrmap e13: x(active, since 60s) 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: overall HEALTH_OK 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:46.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 -- 172.21.15.71:0/4103025871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac0035c0 msgr2=0x7ffaac005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 --2- 172.21.15.71:0/4103025871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac0035c0 0x7ffaac005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffac4074460 tx=0x7ffac0009e80 comp rx=0 tx=0).stop 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 -- 172.21.15.71:0/4103025871 shutdown_connections 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 --2- 172.21.15.71:0/4103025871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac0035c0 0x7ffaac005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 -- 172.21.15.71:0/4103025871 >> 172.21.15.71:0/4103025871 conn(0x7ffac40fd760 msgr2=0x7ffac40ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 -- 172.21.15.71:0/4103025871 shutdown_connections 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.878+0000 7ffacb132700 1 -- 172.21.15.71:0/4103025871 wait complete. 2024-03-21T09:01:46.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.879+0000 7ffacb132700 1 Processor -- start 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.879+0000 7ffacb132700 1 -- start start 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.879+0000 7ffacb132700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.879+0000 7ffacb132700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffac419e3a0 0x7ffac41a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.879+0000 7ffacb132700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4110300 con 0x7ffac419e3a0 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.879+0000 7ffacb132700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4110180 con 0x7ffaac0035c0 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35334/0 (socket says 172.21.15.71:35334) 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 -- 172.21.15.71:0/1715954553 learned_addr learned my addr 172.21.15.71:0/1715954553 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 -- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 msgr2=0x7ffac419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 -- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 msgr2=0x7ffac419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T09:01:46.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T09:01:46.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffac8ece700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:01:46.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.880+0000 7ffabbfff700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffac419e3a0 0x7ffac41a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:46.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffabbfff700 1 -- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 msgr2=0x7ffac419de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:01:46.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffabbfff700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:46.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffabbfff700 1 -- 172.21.15.71:0/1715954553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac419e9e0 con 0x7ffac419e3a0 2024-03-21T09:01:46.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffabbfff700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffac419e3a0 0x7ffac41a2730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b4d0 tx=0x7ffab400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:46.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ffab4011630 con 0x7ffac419e3a0 2024-03-21T09:01:46.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac41a2c70 con 0x7ffac419e3a0 2024-03-21T09:01:46.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab4011cb0 con 0x7ffac419e3a0 2024-03-21T09:01:46.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.881+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ffab4010e60 con 0x7ffac419e3a0 2024-03-21T09:01:46.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.882+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac41a31d0 con 0x7ffac419e3a0 2024-03-21T09:01:46.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.883+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7ffab4018020 con 0x7ffac419e3a0 2024-03-21T09:01:46.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.883+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac4075c70 con 0x7ffac419e3a0 2024-03-21T09:01:46.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.883+0000 7ffab9ffb700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffab0038f70 0x7ffab003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:46.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.884+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffab404bc20 con 0x7ffac419e3a0 2024-03-21T09:01:46.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.884+0000 7ffac8ece700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffab0038f70 0x7ffab003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:46.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.885+0000 7ffac8ece700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffab0038f70 0x7ffab003b430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009a30 tx=0x7ffac00090b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:46.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:46.899+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffab401f020 con 0x7ffac419e3a0 2024-03-21T09:01:47.007 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:46 smithi037 ceph-mon[31830]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:47.008 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:47.008 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:46 smithi037 ceph-mon[31830]: Deploying daemon mon.b on smithi053 2024-03-21T09:01:47.008 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:46 smithi071 ceph-mon[36597]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:46 smithi071 ceph-mon[36597]: Deploying daemon mon.b on smithi053 2024-03-21T09:01:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.180+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffac41a34e0 con 0x7ffac419e3a0 2024-03-21T09:01:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.182+0000 7ffab9ffb700 1 -- 172.21.15.71:0/1715954553 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7ffab4014170 con 0x7ffac419e3a0 2024-03-21T09:01:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T09:01:47.177 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","modified":"2024-03-21T09:01:40.599555Z","created":"2024-03-21T08:59:24.797606Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T09:01:47.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffab0038f70 msgr2=0x7ffab003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:47.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffab0038f70 0x7ffab003b430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009a30 tx=0x7ffac00090b0 comp rx=0 tx=0).stop 2024-03-21T09:01:47.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffac419e3a0 msgr2=0x7ffac41a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffac419e3a0 0x7ffac41a2730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b4d0 tx=0x7ffab400b890 comp rx=0 tx=0).stop 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 shutdown_connections 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffab0038f70 0x7ffab003b430 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac0035c0 0x7ffac419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 --2- 172.21.15.71:0/1715954553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffac419e3a0 0x7ffac41a2730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 >> 172.21.15.71:0/1715954553 conn(0x7ffac40fd760 msgr2=0x7ffac40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 shutdown_connections 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:47.184+0000 7ffacb132700 1 -- 172.21.15.71:0/1715954553 wait complete. 2024-03-21T09:01:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 2 2024-03-21T09:01:47.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:01:47 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:01:47.599+0000 7f45ae200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-21T09:01:47.735 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:47 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/1715954553' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:01:48.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:47 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/1715954553' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:01:48.737 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T09:01:48.738 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:01:48.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:48 smithi037 ceph-mon[31830]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:01:49.085 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:48 smithi071 ceph-mon[36597]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 podman[36150]: 2024-03-21 09:01:49.264410535 +0000 UTC m=+0.017902024 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 podman[36150]: 2024-03-21 09:01:49.571442027 +0000 UTC m=+0.324933517 container init fe569e9ff56510e7daebd8bf8fef507e06d6cbeb45ac1fe4c036bc522f93e05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 podman[36150]: 2024-03-21 09:01:49.574400582 +0000 UTC m=+0.327892059 container start fe569e9ff56510e7daebd8bf8fef507e06d6cbeb45ac1fe4c036bc522f93e05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pidfile_write: ignore empty --pid-file 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: load: jerasure load: lrc 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: RocksDB version: 7.9.2 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Git sha 0 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: DB SUMMARY 2024-03-21T09:01:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: DB Session ID: 7RCBCHHDJGV8S4O2JO3D 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: CURRENT file: CURRENT 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.error_if_exists: 0 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.create_if_missing: 0 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.paranoid_checks: 1 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.env: 0x55e47f661b20 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T09:01:49.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.info_log: 0x55e481943920 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.statistics: (nil) 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.use_fsync: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_log_file_size: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.allow_fallocate: 1 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.use_direct_reads: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T09:01:49.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.db_log_dir: 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.wal_dir: 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.write_buffer_manager: 0x55e4819161e0 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T09:01:49.946 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.unordered_write: 0 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.row_cache: None 2024-03-21T09:01:49.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.wal_filter: None 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.two_write_queues: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.wal_compression: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.atomic_flush: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.log_readahead_size: 0 2024-03-21T09:01:49.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_background_jobs: 2 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_background_compactions: -1 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_subcompactions: 1 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T09:01:49.950 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_open_files: -1 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_background_flushes: -1 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Compression algorithms supported: 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kZSTD supported: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kXpressCompression supported: 0 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T09:01:49.951 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kZlibCompression supported: 1 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kSnappyCompression supported: 1 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kLZ4Compression supported: 1 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: kBZip2Compression supported: 0 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.merge_operator: 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_filter: None 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T09:01:49.952 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e481943a20) 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: cache_index_and_filter_blocks: 1 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: pin_top_level_index_and_filter: 1 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_type: 0 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: data_block_index_type: 0 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_shortening: 1 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: checksum: 4 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: no_block_cache: 0 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache: 0x55e481949090 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_name: BinnedLRUCache 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_options: 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: capacity : 536870912 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: num_shard_bits : 4 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: strict_capacity_limit : 0 2024-03-21T09:01:49.953 INFO:journalctl@ceph.mon.b.smithi053.stdout: high_pri_pool_ratio: 0.000 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_cache_compressed: (nil) 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: persistent_cache: (nil) 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_size: 4096 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_size_deviation: 10 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_restart_interval: 16 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: index_block_restart_interval: 1 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: metadata_block_size: 4096 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: partition_filters: 0 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: use_delta_encoding: 1 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: filter_policy: bloomfilter 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: whole_key_filtering: 1 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: verify_compression: 0 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: read_amp_bytes_per_bit: 0 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: format_version: 5 2024-03-21T09:01:49.954 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_index_compression: 1 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout: block_align: 0 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout: max_auto_readahead_size: 262144 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout: prepopulate_block_cache: 0 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout: initial_auto_readahead_size: 8192 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression: NoCompression 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.num_levels: 7 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T09:01:49.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:01:49.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T09:01:49.957 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T09:01:49.958 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.table_properties_collectors: 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.inplace_update_support: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.bloom_locality: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.max_successive_merges: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T09:01:49.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.ttl: 2592000 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.enable_blob_files: false 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.min_blob_size: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T09:01:49.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ee513258-baa1-4dbb-8897-52979b4cc294 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011709604271, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011709604702, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711011709, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ee513258-baa1-4dbb-8897-52979b4cc294", "db_session_id": "7RCBCHHDJGV8S4O2JO3D", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711011709604785, "job": 1, "event": "recovery_finished"} 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e481a3c000 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: DB pointer 0x55e481a26000 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T09:01:49.961 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** DB Stats ** 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** Compaction Stats [default] ** 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:49.962 INFO:journalctl@ceph.mon.b.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** Compaction Stats [default] ** 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:01:49.963 INFO:journalctl@ceph.mon.b.smithi053.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: Block cache BinnedLRUCache@0x55e481949090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: using public_addr v2:172.21.15.53:0/0 -> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: starting mon.b rank -1 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(???) e0 preinit fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).mds e1 new map 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: e1 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: legacy client fscid: -1 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-03-21T09:01:49.964 INFO:journalctl@ceph.mon.b.smithi053.stdout: No filesystems configured 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Added host smithi053 2024-03-21T09:01:49.965 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:49.966 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi053:/etc/ceph/ceph.conf 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Deploying cephadm binary to smithi071 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Added host smithi071 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:49.967 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-21T09:01:49.968 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1125262657' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1125262657' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c;count:3 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:49.969 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Deploying daemon mon.c on smithi071 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.a calling monitor election 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.c calling monitor election 2024-03-21T09:01:49.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: monmap e2: 2 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: fsmap 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mgrmap e13: x(active, since 60s) 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: overall HEALTH_OK 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: Deploying daemon mon.b on smithi053 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/1715954553' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 ceph-mon[36169]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 bash[36150]: fe569e9ff56510e7daebd8bf8fef507e06d6cbeb45ac1fe4c036bc522f93e05f 2024-03-21T09:01:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:49 smithi053 systemd[1]: Started Ceph mon.b for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:01:54.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 -- 172.21.15.71:0/2359581732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 msgr2=0x7f2cf0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:54.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/2359581732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2cf0005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d04101b90 tx=0x7f2d0000fe60 comp rx=0 tx=0).stop 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 -- 172.21.15.71:0/2359581732 shutdown_connections 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/2359581732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2cf0005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 -- 172.21.15.71:0/2359581732 >> 172.21.15.71:0/2359581732 conn(0x7f2d040fd730 msgr2=0x7f2d040ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 -- 172.21.15.71:0/2359581732 shutdown_connections 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.651+0000 7f2d0beb6700 1 -- 172.21.15.71:0/2359581732 wait complete. 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 Processor -- start 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 -- start start 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2d0419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 0x7f2d04198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04109cf0 con 0x7f2cf00035c0 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04109b70 con 0x7f2d04197c60 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0beb6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04109e70 con 0x7f2d0419db90 2024-03-21T09:01:54.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39304/0 (socket says 172.21.15.71:39304) 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 -- 172.21.15.71:0/590343332 learned_addr learned my addr 172.21.15.71:0/590343332 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 msgr2=0x7f2d04197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 msgr2=0x7f2d04197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d09451700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.652+0000 7f2d0a453700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 0x7f2d04198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d09c52700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2d0419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d0a453700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 msgr2=0x7f2d04198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d0a453700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 msgr2=0x7f2d04198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d0a453700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 0x7f2d04198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d0a453700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 0x7f2d04198110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:01:54.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d09c52700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 msgr2=0x7f2d04197720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:01:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d09c52700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d09c52700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 msgr2=0x7f2d04198110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:01:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d09c52700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 0x7f2d04198110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.653+0000 7f2d09c52700 1 -- 172.21.15.71:0/590343332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d041a4400 con 0x7f2cf00035c0 2024-03-21T09:01:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.654+0000 7f2d09c52700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2d0419d650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d04101b90 tx=0x7f2d00014fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:54.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.658+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0002b020 con 0x7f2cf00035c0 2024-03-21T09:01:54.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.658+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d041a4630 con 0x7f2cf00035c0 2024-03-21T09:01:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.658+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d041a4bc0 con 0x7f2cf00035c0 2024-03-21T09:01:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.658+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d0000f870 con 0x7f2cf00035c0 2024-03-21T09:01:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.658+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00022040 con 0x7f2cf00035c0 2024-03-21T09:01:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.659+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf0002110 con 0x7f2cf00035c0 2024-03-21T09:01:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.660+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f2d0002e020 con 0x7f2cf00035c0 2024-03-21T09:01:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.660+0000 7f2cfaffd700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2ce80396b0 0x7f2ce803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.660+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2d0004ec90 con 0x7f2cf00035c0 2024-03-21T09:01:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.660+0000 7f2d09451700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2ce80396b0 0x7f2ce803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:54.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.661+0000 7f2d09451700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2ce80396b0 0x7f2ce803bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009180 tx=0x7f2cf4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:54.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.676+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d00035020 con 0x7f2cf00035c0 2024-03-21T09:01:54.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.956+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2cf0002c00 con 0x7f2cf00035c0 2024-03-21T09:01:54.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: mon.c calling monitor election 2024-03-21T09:01:54.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:01:54.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: mon.a calling monitor election 2024-03-21T09:01:54.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:54.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: fsmap 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:54.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: mgrmap e13: x(active, since 69s) 2024-03-21T09:01:54.954 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T09:01:54.954 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:01:54.954 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T09:01:54.954 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: mon.b (rank 2) addr [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] is down (out of quorum) 2024-03-21T09:01:54.954 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:54.954 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:54.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.959+0000 7f2cfaffd700 1 -- 172.21.15.71:0/590343332 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1275 (secure 0 0 0) 0x7f2d000103d0 con 0x7f2cf00035c0 2024-03-21T09:01:54.955 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-21T09:01:54.955 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","modified":"2024-03-21T09:01:49.627607Z","created":"2024-03-21T08:59:24.797606Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T09:01:54.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2ce80396b0 msgr2=0x7f2ce803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:54.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2ce80396b0 0x7f2ce803bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009180 tx=0x7f2cf4008040 comp rx=0 tx=0).stop 2024-03-21T09:01:54.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 msgr2=0x7f2d0419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2d0419d650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d04101b90 tx=0x7f2d00014fc0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 shutdown_connections 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2ce80396b0 0x7f2ce803bb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d04197c60 0x7f2d04198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf00035c0 0x7f2d0419d650 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 --2- 172.21.15.71:0/590343332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d0419db90 0x7f2d04197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 >> 172.21.15.71:0/590343332 conn(0x7f2d040fd730 msgr2=0x7f2d041058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 shutdown_connections 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:01:54.961+0000 7f2d0beb6700 1 -- 172.21.15.71:0/590343332 wait complete. 2024-03-21T09:01:54.956 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 3 2024-03-21T09:01:55.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: mon.c calling monitor election 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: mon.a calling monitor election 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:55.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: fsmap 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: mgrmap e13: x(active, since 69s) 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: mon.b (rank 2) addr [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] is down (out of quorum) 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.508 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T09:01:55.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T09:01:55.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:55 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/590343332' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:01:55.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:55.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:55 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/590343332' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:01:55.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:55.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:55.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:55.959 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:01:56.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: mon.c calling monitor election 2024-03-21T09:01:56.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:01:56.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: mon.a calling monitor election 2024-03-21T09:01:56.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: fsmap 2024-03-21T09:01:56.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: mgrmap e13: x(active, since 69s) 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: mon.b (rank 2) addr [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] is down (out of quorum) 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/590343332' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:56.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: mon.b calling monitor election 2024-03-21T09:01:57.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: mon.b calling monitor election 2024-03-21T09:01:57.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: mon.c calling monitor election 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: mon.a calling monitor election 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: fsmap 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: mgrmap e13: x(active, since 71s) 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: Cluster is now healthy 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: overall HEALTH_OK 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.863 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: mon.b calling monitor election 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: mon.b calling monitor election 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: mon.c calling monitor election 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: mon.a calling monitor election 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: fsmap 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: mgrmap e13: x(active, since 71s) 2024-03-21T09:01:57.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T09:01:57.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: Cluster is now healthy 2024-03-21T09:01:57.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: overall HEALTH_OK 2024-03-21T09:01:57.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: mon.b calling monitor election 2024-03-21T09:01:57.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: mon.b calling monitor election 2024-03-21T09:01:57.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: mon.c calling monitor election 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: mon.a calling monitor election 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: fsmap 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: mgrmap e13: x(active, since 71s) 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: Cluster is now healthy 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: overall HEALTH_OK 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:57.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 -- 172.21.15.37:0/1906538623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec4103fe0 msgr2=0x7feec41043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 --2- 172.21.15.37:0/1906538623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec4103fe0 0x7feec41043c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009910 tx=0x7feeb4009c20 comp rx=0 tx=0).stop 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 -- 172.21.15.37:0/1906538623 shutdown_connections 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 --2- 172.21.15.37:0/1906538623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec4103fe0 0x7feec41043c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 -- 172.21.15.37:0/1906538623 >> 172.21.15.37:0/1906538623 conn(0x7feec40ff870 msgr2=0x7feec4101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 -- 172.21.15.37:0/1906538623 shutdown_connections 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.044+0000 7feecb000700 1 -- 172.21.15.37:0/1906538623 wait complete. 2024-03-21T09:01:58.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.045+0000 7feecb000700 1 Processor -- start 2024-03-21T09:01:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.045+0000 7feecb000700 1 -- start start 2024-03-21T09:01:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feecb000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 0x7feec419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feecb000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec419fd50 0x7feec41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feecb000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec4199f10 0x7feec419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feecb000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec410e370 con 0x7feec419fd50 2024-03-21T09:01:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feecb000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec410e1f0 con 0x7feec4199f10 2024-03-21T09:01:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feecb000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec410e4f0 con 0x7feec4103fe0 2024-03-21T09:01:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feebbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec419fd50 0x7feec41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feec3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 0x7feec419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feec3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 0x7feec419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45774/0 (socket says 172.21.15.37:45774) 2024-03-21T09:01:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feec3e00700 1 -- 172.21.15.37:0/4292192248 learned_addr learned my addr 172.21.15.37:0/4292192248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:01:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.046+0000 7feec9000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec4199f10 0x7feec419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec3e00700 1 -- 172.21.15.37:0/4292192248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec4199f10 msgr2=0x7feec419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec3e00700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec4199f10 0x7feec419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec3e00700 1 -- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec419fd50 msgr2=0x7feec41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec3e00700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec419fd50 0x7feec41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec3e00700 1 -- 172.21.15.37:0/4292192248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec41a66e0 con 0x7feec4103fe0 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec9000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec4199f10 0x7feec419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:01:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.047+0000 7feec3e00700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 0x7feec419f810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feeb400bdc0 tx=0x7feeb400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.048+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4027070 con 0x7feec4103fe0 2024-03-21T09:01:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.048+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feeb40050d0 con 0x7feec4103fe0 2024-03-21T09:01:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.048+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4022dc0 con 0x7feec4103fe0 2024-03-21T09:01:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.048+0000 7feecb000700 1 -- 172.21.15.37:0/4292192248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec41a6910 con 0x7feec4103fe0 2024-03-21T09:01:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.048+0000 7feecb000700 1 -- 172.21.15.37:0/4292192248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec41a6e00 con 0x7feec4103fe0 2024-03-21T09:01:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.049+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c005330 con 0x7feec4103fe0 2024-03-21T09:01:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.049+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7feeb402a070 con 0x7feec4103fe0 2024-03-21T09:01:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.050+0000 7feec1600700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feea80396b0 0x7feea803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:01:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.050+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feeb404d5a0 con 0x7feec4103fe0 2024-03-21T09:01:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.051+0000 7feebbe00700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feea80396b0 0x7feea803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:01:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.051+0000 7feebbe00700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feea80396b0 0x7feea803bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feeac009ad0 tx=0x7feeac009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:01:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.055+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feeb4013070 con 0x7feec4103fe0 2024-03-21T09:01:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.295+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fee9c005620 con 0x7feec4103fe0 2024-03-21T09:01:58.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.296+0000 7feec1600700 1 -- 172.21.15.37:0/4292192248 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7feeb4031070 con 0x7feec4103fe0 2024-03-21T09:01:58.290 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:01:58.290 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-03-21T09:01:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:01:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.298+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feea80396b0 msgr2=0x7feea803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.298+0000 7feeba000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feea80396b0 0x7feea803bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feeac009ad0 tx=0x7feeac009210 comp rx=0 tx=0).stop 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.298+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 msgr2=0x7feec419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.298+0000 7feeba000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 0x7feec419f810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feeb400bdc0 tx=0x7feeb400bdf0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 shutdown_connections 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feea80396b0 0x7feea803bb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feec4199f10 0x7feec419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec4103fe0 0x7feec419f810 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 --2- 172.21.15.37:0/4292192248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec419fd50 0x7feec41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 >> 172.21.15.37:0/4292192248 conn(0x7feec40ff870 msgr2=0x7feec41059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 shutdown_connections 2024-03-21T09:01:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:01:58.299+0000 7feeba000700 1 -- 172.21.15.37:0/4292192248 wait complete. 2024-03-21T09:01:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:58.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:58 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4292192248' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:58.918 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:01:58.628+0000 7f45ae200700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:58.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:01:58.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:58.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:58 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4292192248' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:58.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:58 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4292192248' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:59.182 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T09:01:59.183 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:01:59.183 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T09:01:59.287 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:01:59.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:59.435 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T09:01:59.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T09:01:59.475 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T09:01:59.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:59.550 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T09:01:59.550 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T09:01:59.580 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T09:01:59.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:01:59.648 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-03-21T09:01:59.648 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-03-21T09:01:59.648 INFO:tasks.cephadm:Adding mgr.z on smithi071 2024-03-21T09:01:59.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi037=x;smithi053=y;smithi071=z' 2024-03-21T09:01:59.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:59 smithi037 ceph-mon[31830]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:59.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:59 smithi037 ceph-mon[31830]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T09:01:59.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:01:59 smithi037 ceph-mon[31830]: Reconfiguring daemon mon.a on smithi037 2024-03-21T09:01:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:59 smithi071 ceph-mon[36597]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:59 smithi071 ceph-mon[36597]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T09:01:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:01:59 smithi071 ceph-mon[36597]: Reconfiguring daemon mon.a on smithi037 2024-03-21T09:01:59.918 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:01:59.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:59 smithi053 ceph-mon[36169]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:01:59.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:59 smithi053 ceph-mon[36169]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T09:01:59.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:01:59 smithi053 ceph-mon[36169]: Reconfiguring daemon mon.a on smithi037 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 -- 172.21.15.71:0/591766845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 msgr2=0x7f3b78103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 --2- 172.21.15.71:0/591766845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78103810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009910 tx=0x7f3b68009c20 comp rx=0 tx=0).stop 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 -- 172.21.15.71:0/591766845 shutdown_connections 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 --2- 172.21.15.71:0/591766845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78103810 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 -- 172.21.15.71:0/591766845 >> 172.21.15.71:0/591766845 conn(0x7f3b780fd2b0 msgr2=0x7f3b780ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 -- 172.21.15.71:0/591766845 shutdown_connections 2024-03-21T09:02:01.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.256+0000 7f3b7d803700 1 -- 172.21.15.71:0/591766845 wait complete. 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 Processor -- start 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 -- start start 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 0x7f3b78074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78074f40 0x7f3b781a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78107e60 con 0x7f3b781033f0 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78107ce0 con 0x7f3b78074f40 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b7d803700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78107fe0 con 0x7f3b78074580 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b777fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78074f40 0x7f3b781a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b767fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 0x7f3b78074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b777fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78074f40 0x7f3b781a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.71:44214/0 (socket says 172.21.15.71:44214) 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b777fe700 1 -- 172.21.15.71:0/148933672 learned_addr learned my addr 172.21.15.71:0/148933672 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.257+0000 7f3b767fc700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 0x7f3b78074a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39308/0 (socket says 172.21.15.71:39308) 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b76ffd700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b767fc700 1 -- 172.21.15.71:0/148933672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78074f40 msgr2=0x7f3b781a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b767fc700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78074f40 0x7f3b781a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b767fc700 1 -- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 msgr2=0x7f3b78075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b767fc700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b767fc700 1 -- 172.21.15.71:0/148933672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b781a1c50 con 0x7f3b78074580 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b76ffd700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b767fc700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 0x7f3b78074a00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00b550 tx=0x7f3b6c00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:02:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c004230 con 0x7f3b78074580 2024-03-21T09:02:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c0043d0 con 0x7f3b78074580 2024-03-21T09:02:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b781a1f40 con 0x7f3b78074580 2024-03-21T09:02:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c011c20 con 0x7f3b78074580 2024-03-21T09:02:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.258+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b781a24c0 con 0x7f3b78074580 2024-03-21T09:02:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.259+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3b6c01a400 con 0x7f3b78074580 2024-03-21T09:02:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.259+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b780637c0 con 0x7f3b78074580 2024-03-21T09:02:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.260+0000 7f3b7c801700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600396b0 0x7f3b6003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:01.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.260+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b6c04bda0 con 0x7f3b78074580 2024-03-21T09:02:01.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.260+0000 7f3b76ffd700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600396b0 0x7f3b6003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:01.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.261+0000 7f3b76ffd700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600396b0 0x7f3b6003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b68005330 tx=0x7f3b68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:02:01.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.265+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b6c020500 con 0x7f3b78074580 2024-03-21T09:02:01.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.474+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi053=y;smithi071=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b78062720 con 0x7f3b600396b0 2024-03-21T09:02:01.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.487+0000 7f3b7c801700 1 -- 172.21.15.71:0/148933672 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3b78062720 con 0x7f3b600396b0 2024-03-21T09:02:01.481 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mgr update... 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600396b0 msgr2=0x7f3b6003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600396b0 0x7f3b6003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b68005330 tx=0x7f3b68005fb0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 msgr2=0x7f3b78074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 0x7f3b78074a00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00b550 tx=0x7f3b6c00b860 comp rx=0 tx=0).stop 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 shutdown_connections 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600396b0 0x7f3b6003bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78074f40 0x7f3b781a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b781033f0 0x7f3b78075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 --2- 172.21.15.71:0/148933672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b78074580 0x7f3b78074a00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:01.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 >> 172.21.15.71:0/148933672 conn(0x7f3b780fd2b0 msgr2=0x7f3b780fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:02:01.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 shutdown_connections 2024-03-21T09:02:01.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:02:01.489+0000 7f3b7d803700 1 -- 172.21.15.71:0/148933672 wait complete. 2024-03-21T09:02:01.767 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:01.767 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.767 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.768 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:02:01.768 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:02:01.768 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:01.768 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:02:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:02:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:01.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:01.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:02:01.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:02:01.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:01.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.001 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T09:02:02.004 DEBUG:teuthology.orchestra.run.smithi071:mgr.z> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.z.service 2024-03-21T09:02:02.006 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T09:02:02.007 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:02:02.007 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T09:02:02.068 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T09:02:02.068 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-21 09:00:59.028578126 +0000 2024-03-21T09:02:02.128 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-21 08:55:44.894668912 +0000 2024-03-21T09:02:02.129 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-21 08:55:44.894668912 +0000 2024-03-21T09:02:02.129 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-21 08:55:44.894668912 +0000 2024-03-21T09:02:02.129 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T09:02:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-21T09:02:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-21T09:02:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00038049 s, 1.3 MB/s 2024-03-21T09:02:02.229 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T09:02:02.287 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-21 09:00:59.134576512 +0000 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-21 08:55:45.456660576 +0000 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-21 08:55:45.456660576 +0000 2024-03-21T09:02:02.338 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-21 08:55:45.456660576 +0000 2024-03-21T09:02:02.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T09:02:02.428 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-21T09:02:02.428 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-21T09:02:02.428 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000342668 s, 1.5 MB/s 2024-03-21T09:02:02.432 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T09:02:02.492 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-03-21T09:02:02.546 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-21 09:00:59.231575035 +0000 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-21 08:55:46.046651825 +0000 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-21 08:55:46.046651825 +0000 2024-03-21T09:02:02.547 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-21 08:55:46.046651825 +0000 2024-03-21T09:02:02.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T09:02:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-21T09:02:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-21T09:02:02.644 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00034668 s, 1.5 MB/s 2024-03-21T09:02:02.648 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T09:02:02.705 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-03-21T09:02:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T09:02:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:02.755 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-21T09:02:02.756 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:02.756 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:02.756 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-21 09:00:59.324573620 +0000 2024-03-21T09:02:02.756 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-21 08:55:46.660642718 +0000 2024-03-21T09:02:02.756 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-21 08:55:46.660642718 +0000 2024-03-21T09:02:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-21 08:55:46.660642718 +0000 2024-03-21T09:02:02.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T09:02:02.847 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-21T09:02:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-21T09:02:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000355812 s, 1.4 MB/s 2024-03-21T09:02:02.852 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: Reconfiguring mon.b (monmap changed)... 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: Reconfiguring daemon mon.b on smithi053 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi053=y;smithi071=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: Saving service mgr spec with placement smithi037=x;smithi053=y;smithi071=z;count:3 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:02:02.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:02.913 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-21T09:02:02.913 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T09:02:02.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: Reconfiguring mon.b (monmap changed)... 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: Reconfiguring daemon mon.b on smithi053 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi053=y;smithi071=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: Saving service mgr spec with placement smithi037=x;smithi053=y;smithi071=z;count:3 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:02:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:02.933 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T09:02:02.933 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-03-21T09:02:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: Reconfiguring mon.b (monmap changed)... 2024-03-21T09:02:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: Reconfiguring daemon mon.b on smithi053 2024-03-21T09:02:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi053=y;smithi071=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: Saving service mgr spec with placement smithi037=x;smithi053=y;smithi071=z;count:3 2024-03-21T09:02:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:02.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:02:02.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:02:02.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-21 09:01:49.071702805 +0000 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-21 08:54:28.291717412 +0000 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-21 08:54:28.291717412 +0000 2024-03-21T09:02:02.960 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-21 08:54:28.291717412 +0000 2024-03-21T09:02:02.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T09:02:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-21T09:02:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-21T09:02:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000112109 s, 4.6 MB/s 2024-03-21T09:02:03.035 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T09:02:03.094 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-21 09:01:49.071702805 +0000 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-21 08:54:28.633710520 +0000 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-21 08:54:28.633710520 +0000 2024-03-21T09:02:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-21 08:54:28.633710520 +0000 2024-03-21T09:02:03.152 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T09:02:03.222 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-21T09:02:03.222 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-21T09:02:03.222 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.00011885 s, 4.3 MB/s 2024-03-21T09:02:03.223 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T09:02:03.281 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-21 09:01:49.072702785 +0000 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-21 08:54:28.985703427 +0000 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-21 08:54:28.985703427 +0000 2024-03-21T09:02:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-21 08:54:28.985703427 +0000 2024-03-21T09:02:03.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T09:02:03.410 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-21T09:02:03.410 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-21T09:02:03.410 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.5494e-05 s, 5.4 MB/s 2024-03-21T09:02:03.411 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T09:02:03.468 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-21 09:01:49.072702785 +0000 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-21 08:54:29.323696616 +0000 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-21 08:54:29.323696616 +0000 2024-03-21T09:02:03.524 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-21 08:54:29.323696616 +0000 2024-03-21T09:02:03.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T09:02:03.594 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-21T09:02:03.595 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-21T09:02:03.595 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000113419 s, 4.5 MB/s 2024-03-21T09:02:03.596 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T09:02:03.653 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-21T09:02:03.653 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T09:02:03.668 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T09:02:03.669 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-03-21T09:02:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:03 smithi053 ceph-mon[36169]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:03 smithi053 ceph-mon[36169]: Reconfiguring mon.c (monmap changed)... 2024-03-21T09:02:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:03 smithi053 ceph-mon[36169]: Reconfiguring daemon mon.c on smithi071 2024-03-21T09:02:03.723 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-21 09:01:39.789431212 +0000 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-21 08:54:26.192270595 +0000 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-21 08:54:26.192270595 +0000 2024-03-21T09:02:03.724 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-21 08:54:26.192270595 +0000 2024-03-21T09:02:03.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T09:02:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-21T09:02:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-21T09:02:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000217236 s, 2.4 MB/s 2024-03-21T09:02:03.793 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T09:02:03.847 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-03-21T09:02:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:03 smithi071 ceph-mon[36597]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:03 smithi071 ceph-mon[36597]: Reconfiguring mon.c (monmap changed)... 2024-03-21T09:02:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:03 smithi071 ceph-mon[36597]: Reconfiguring daemon mon.c on smithi071 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-21 09:01:39.789431212 +0000 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-21 08:54:26.534264279 +0000 2024-03-21T09:02:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-21 08:54:26.534264279 +0000 2024-03-21T09:02:03.868 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-21 08:54:26.534264279 +0000 2024-03-21T09:02:03.868 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T09:02:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:03 smithi037 ceph-mon[31830]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:03 smithi037 ceph-mon[31830]: Reconfiguring mon.c (monmap changed)... 2024-03-21T09:02:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:03 smithi037 ceph-mon[31830]: Reconfiguring daemon mon.c on smithi071 2024-03-21T09:02:03.932 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-21T09:02:03.932 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-21T09:02:03.932 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000224151 s, 2.3 MB/s 2024-03-21T09:02:03.933 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T09:02:03.989 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-21 09:01:39.790431194 +0000 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-21 08:54:26.882257851 +0000 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-21 08:54:26.882257851 +0000 2024-03-21T09:02:04.045 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-21 08:54:26.882257851 +0000 2024-03-21T09:02:04.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T09:02:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-21T09:02:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-21T09:02:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000203125 s, 2.5 MB/s 2024-03-21T09:02:04.112 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T09:02:04.169 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-03-21T09:02:04.224 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T09:02:04.224 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:02:04.224 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-21T09:02:04.225 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:02:04.225 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:02:04.225 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-21 09:01:39.790431194 +0000 2024-03-21T09:02:04.225 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-21 08:54:27.216251682 +0000 2024-03-21T09:02:04.225 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-21 08:54:27.216251682 +0000 2024-03-21T09:02:04.225 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-21 08:54:27.216251682 +0000 2024-03-21T09:02:04.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T09:02:04.290 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-21T09:02:04.290 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-21T09:02:04.290 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000205906 s, 2.5 MB/s 2024-03-21T09:02:04.291 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T09:02:04.346 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-03-21T09:02:04.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T09:02:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:02:04.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:04.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:04.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:04 smithi071 ceph-mon[36597]: Deploying daemon mgr.z on smithi071 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:04.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:04 smithi037 ceph-mon[31830]: Deploying daemon mgr.z on smithi071 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:02:04.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:04.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:04.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:04 smithi053 ceph-mon[36169]: Deploying daemon mgr.z on smithi071 2024-03-21T09:02:05.854 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:05 smithi071 podman[38464]: 2024-03-21 09:02:05.715152335 +0000 UTC m=+0.017369101 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:02:06.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 podman[38464]: 2024-03-21 09:02:06.006457556 +0000 UTC m=+0.308674308 container init be3dcb7258ecb9af11332d7c4e6f81bdd66e21022b4f7bfa388035ea2d267ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T09:02:06.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 podman[38464]: 2024-03-21 09:02:06.009170196 +0000 UTC m=+0.311386952 container start be3dcb7258ecb9af11332d7c4e6f81bdd66e21022b4f7bfa388035ea2d267ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T09:02:06.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 ceph-mgr[38483]: -- 172.21.15.71:0/1602850253 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56086ab2a5a0 con 0x56086aaff000 2024-03-21T09:02:06.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 bash[38464]: be3dcb7258ecb9af11332d7c4e6f81bdd66e21022b4f7bfa388035ea2d267ddc 2024-03-21T09:02:06.354 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 systemd[1]: Started Ceph mgr.z for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:02:06.632 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:06.546+0000 7f27ba5a4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:02:06.632 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:06.637+0000 7f27ba5a4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:02:07.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:07.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:07.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:02:07.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:07.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:07.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:07 smithi037 ceph-mon[31830]: Deploying daemon mgr.y on smithi053 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:07.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:07.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:07 smithi053 ceph-mon[36169]: Deploying daemon mgr.y on smithi053 2024-03-21T09:02:07.494 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:07.268+0000 7f27ba5a4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:07.358+0000 7f27ba5a4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:07.499+0000 7f27ba5a4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:07.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:07.496 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-mon[36597]: Deploying daemon mgr.y on smithi053 2024-03-21T09:02:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T09:02:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T09:02:07.602 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-21T09:02:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-21T09:02:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0378866 s, 277 MB/s 2024-03-21T09:02:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-21T09:02:07.853 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:07.593+0000 7f27ba5a4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:02:07.853 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:07.675+0000 7f27ba5a4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:02:08.215 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:07.935+0000 7f27ba5a4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:02:08.215 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:08.042+0000 7f27ba5a4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:02:08.215 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:08.126+0000 7f27ba5a4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:02:08.215 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:08.220+0000 7f27ba5a4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:02:08.378 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 systemd[1]: Starting Ceph mgr.y for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:02:08.486 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-03-21T09:02:08.603 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:08.391+0000 7f27ba5a4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:02:08.639 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 podman[37533]: 2024-03-21 09:02:08.28678411 +0000 UTC m=+0.018021559 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:02:08.639 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 podman[37533]: 2024-03-21 09:02:08.389403017 +0000 UTC m=+0.120640442 container create 27f6ceb034e87afc776032c9d0e3ad61e6b95ac0bb4c405674e7c8a5f0d76757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:02:08.639 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 podman[37533]: 2024-03-21 09:02:08.616718389 +0000 UTC m=+0.347955841 container init 27f6ceb034e87afc776032c9d0e3ad61e6b95ac0bb4c405674e7c8a5f0d76757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T09:02:08.639 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 podman[37533]: 2024-03-21 09:02:08.619482255 +0000 UTC m=+0.350719681 container start 27f6ceb034e87afc776032c9d0e3ad61e6b95ac0bb4c405674e7c8a5f0d76757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T09:02:08.639 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 ceph-mgr[37552]: -- 172.21.15.53:0/2682920336 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d87b08a5a0 con 0x55d87b05ec00 2024-03-21T09:02:08.942 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 bash[37533]: 27f6ceb034e87afc776032c9d0e3ad61e6b95ac0bb4c405674e7c8a5f0d76757 2024-03-21T09:02:08.942 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:08 smithi053 systemd[1]: Started Ceph mgr.y for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:02:08.951 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-mon[36169]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:09.199+0000 7fb7cd3cf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:02:09.383 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:09.300+0000 7fb7cd3cf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:02:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:09 smithi037 ceph-mon[31830]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:09.494 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:09.367+0000 7f27ba5a4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:02:09.494 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:09.499+0000 7f27ba5a4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:02:09.494 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-mon[36597]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:09.494 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.494 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.494 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:09.495 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:10.103 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:09.846+0000 7f27ba5a4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:02:10.103 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:09.935+0000 7f27ba5a4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:02:10.304 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.063+0000 7fb7cd3cf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:02:10.304 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.162+0000 7fb7cd3cf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:02:10.598 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:10.140+0000 7f27ba5a4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:02:10.599 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:10.321+0000 7f27ba5a4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:02:10.692 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.309+0000 7fb7cd3cf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:02:10.692 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.420+0000 7fb7cd3cf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:02:10.692 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.510+0000 7fb7cd3cf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:02:10.853 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:10.603+0000 7f27ba5a4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:02:11.078 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.783+0000 7fb7cd3cf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:02:11.078 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.896+0000 7fb7cd3cf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:02:11.078 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:10.986+0000 7fb7cd3cf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:02:11.078 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:11.083+0000 7fb7cd3cf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 -- 172.21.15.37:0/151685559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c074bf0 msgr2=0x7f1f1c075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 --2- 172.21.15.37:0/151685559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c074bf0 0x7f1f1c075070 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009990 tx=0x7f1f04009ca0 comp rx=0 tx=0).stop 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 -- 172.21.15.37:0/151685559 shutdown_connections 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 --2- 172.21.15.37:0/151685559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c10e7b0 0x7f1f1c110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 --2- 172.21.15.37:0/151685559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 --2- 172.21.15.37:0/151685559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c074bf0 0x7f1f1c075070 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.111+0000 7f1f22800700 1 -- 172.21.15.37:0/151685559 >> 172.21.15.37:0/151685559 conn(0x7f1f1c0fd730 msgr2=0x7f1f1c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.112+0000 7f1f22800700 1 -- 172.21.15.37:0/151685559 shutdown_connections 2024-03-21T09:02:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.112+0000 7f1f22800700 1 -- 172.21.15.37:0/151685559 wait complete. 2024-03-21T09:02:11.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 Processor -- start 2024-03-21T09:02:11.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 -- start start 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c074bf0 0x7f1f1c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f1c10e7b0 0x7f1f1c1a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c114420 con 0x7f1f1c10e7b0 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1142a0 con 0x7f1f1c0761c0 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f22800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1145a0 con 0x7f1f1c074bf0 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f13e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.113+0000 7f1f13e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40516/0 (socket says 172.21.15.37:40516) 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f1b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c074bf0 0x7f1f1c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 -- 172.21.15.37:0/363607816 learned_addr learned my addr 172.21.15.37:0/363607816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f1be00700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f1c10e7b0 0x7f1f1c1a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 -- 172.21.15.37:0/363607816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c074bf0 msgr2=0x7f1f1c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c074bf0 0x7f1f1c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 -- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f1c10e7b0 msgr2=0x7f1f1c1a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f1c10e7b0 0x7f1f1c1a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 -- 172.21.15.37:0/363607816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c1a66c0 con 0x7f1f1c0761c0 2024-03-21T09:02:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f13e00700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c19efa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400b7b0 tx=0x7f1f0400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.114+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04026070 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.115+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f0400f460 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.115+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c1a6950 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.115+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04021670 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.115+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1a6e00 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.116+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f1f04029070 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.116+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c068e50 con 0x7f1f1c0761c0 2024-03-21T09:02:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.117+0000 7f1f18c00700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f00039460 0x7f1f0003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.117+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f0404d300 con 0x7f1f1c0761c0 2024-03-21T09:02:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.117+0000 7f1f1b400700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f00039460 0x7f1f0003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.118+0000 7f1f1b400700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f00039460 0x7f1f0003b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0765a0 tx=0x7f1f0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:02:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.122+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f04013070 con 0x7f1f1c0761c0 2024-03-21T09:02:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:11.358+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f1c0625d0 con 0x7f1f00039460 2024-03-21T09:02:11.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:10.859+0000 7f27ba5a4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:02:11.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:11.087+0000 7f27ba5a4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:02:11.410 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:11.265+0000 7fb7cd3cf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:02:11.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:11.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:02:11.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:11 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:02:11.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:11 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:11.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:02:11.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:12.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:11.862+0000 7f27ba5a4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:02:12.624 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:12 smithi037 ceph-mon[31830]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T09:02:12.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:12 smithi037 ceph-mon[31830]: Reconfiguring daemon mgr.x on smithi037 2024-03-21T09:02:12.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:12 smithi037 ceph-mon[31830]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:12.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-mon[36169]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T09:02:12.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-mon[36169]: Reconfiguring daemon mgr.x on smithi037 2024-03-21T09:02:12.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-mon[36169]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:12.692 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:12.288+0000 7fb7cd3cf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:02:12.693 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:12.422+0000 7fb7cd3cf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:02:12.712 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:12 smithi071 ceph-mon[36597]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T09:02:12.712 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:12 smithi071 ceph-mon[36597]: Reconfiguring daemon mgr.x on smithi037 2024-03-21T09:02:12.712 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:12 smithi071 ceph-mon[36597]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:12.713 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:12.415+0000 7f27ba5a4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:02:12.713 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:12.510+0000 7f27ba5a4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:02:12.713 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:02:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:02:12.717+0000 7f27ba5a4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:02:13.055 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:12.767+0000 7fb7cd3cf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:02:13.055 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:12 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:12.859+0000 7fb7cd3cf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:02:13.055 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:13.060+0000 7fb7cd3cf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:02:13.421 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:13.250+0000 7fb7cd3cf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:02:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:13.445+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 14) v1 ==== 92074+0+0 (secure 0 0 0) 0x7f1f04009d10 con 0x7f1f1c0761c0 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: Standby manager daemon z started 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:13.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:13.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:13.544+0000 7fb7cd3cf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: Standby manager daemon z started 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:13.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:13.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:13.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:13.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: Standby manager daemon z started 2024-03-21T09:02:13.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/2871711512' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:13.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:14.192 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:13 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:13.812+0000 7fb7cd3cf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:02:14.192 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:14.054+0000 7fb7cd3cf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:02:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:14 smithi037 ceph-mon[31830]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-21T09:02:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T09:02:14.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:14 smithi071 ceph-mon[36597]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-21T09:02:14.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T09:02:14.883 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:14 smithi053 ceph-mon[36169]: mgrmap e14: x(active, since 87s), standbys: z 2024-03-21T09:02:14.883 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T09:02:15.192 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:14.889+0000 7fb7cd3cf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:02:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:15 smithi053 ceph-mon[36169]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:15.800 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:15 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:15.481+0000 7fb7cd3cf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:02:15.800 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:15 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:15.581+0000 7fb7cd3cf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:02:15.800 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:02:15 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:02:15.805+0000 7fb7cd3cf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:02:15.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:15 smithi071 ceph-mon[36597]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:15.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:15 smithi037 ceph-mon[31830]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:16.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:16.474+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mon.2 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f1f0402bcc0 con 0x7f1f1c0761c0 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: Standby manager daemon y started 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/562605612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]: dispatch 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]: dispatch 2024-03-21T09:02:16.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]': finished 2024-03-21T09:02:16.473 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T09:02:16.473 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:16.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T09:02:16.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: Standby manager daemon y started 2024-03-21T09:02:16.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:02:16.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T09:02:16.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:02:16.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/562605612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]: dispatch 2024-03-21T09:02:16.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]: dispatch 2024-03-21T09:02:16.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]': finished 2024-03-21T09:02:16.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T09:02:16.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T09:02:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: Standby manager daemon y started 2024-03-21T09:02:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:02:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T09:02:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/1230093605' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:02:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/562605612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]: dispatch 2024-03-21T09:02:16.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]: dispatch 2024-03-21T09:02:16.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8ca1c1e-a53f-43bb-b2d7-0df467be85b4"}]': finished 2024-03-21T09:02:16.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T09:02:16.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:17.801 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:17 smithi037 ceph-mon[31830]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:17.802 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:17 smithi037 ceph-mon[31830]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-03-21T09:02:17.802 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T09:02:17.802 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:17 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/39448072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:02:17.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:17 smithi071 ceph-mon[36597]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:17.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:17 smithi071 ceph-mon[36597]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-03-21T09:02:17.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T09:02:17.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:17 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/39448072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:02:17.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:17 smithi053 ceph-mon[36169]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:17.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:17 smithi053 ceph-mon[36169]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-03-21T09:02:17.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T09:02:17.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:17 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/39448072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:02:19.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:19 smithi071 ceph-mon[36597]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:19.870 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:19 smithi053 ceph-mon[36169]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:19.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:19 smithi037 ceph-mon[31830]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:21.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:21 smithi071 ceph-mon[36597]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:21.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:21 smithi037 ceph-mon[31830]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:21.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:21 smithi053 ceph-mon[36169]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:23.628 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:23 smithi037 ceph-mon[31830]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:23 smithi071 ceph-mon[36597]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:23 smithi053 ceph-mon[36169]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:25 smithi071 ceph-mon[36597]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T09:02:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:25.886 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:25 smithi037 ceph-mon[31830]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:25.887 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T09:02:25.887 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:25 smithi053 ceph-mon[36169]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T09:02:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:26 smithi071 ceph-mon[36597]: Deploying daemon osd.0 on smithi037 2024-03-21T09:02:26.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:26 smithi037 ceph-mon[31830]: Deploying daemon osd.0 on smithi037 2024-03-21T09:02:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:26 smithi053 ceph-mon[36169]: Deploying daemon osd.0 on smithi037 2024-03-21T09:02:27.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:27 smithi037 ceph-mon[31830]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:27 smithi071 ceph-mon[36597]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:27 smithi053 ceph-mon[36169]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:29.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:29 smithi037 ceph-mon[31830]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:29.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:29 smithi071 ceph-mon[36597]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:29 smithi053 ceph-mon[36169]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:31.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:31 smithi037 ceph-mon[31830]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:31.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:31 smithi071 ceph-mon[36597]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:31.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:31 smithi053 ceph-mon[36169]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:33 smithi037 ceph-mon[31830]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:33.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:33 smithi071 ceph-mon[36597]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:33.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:33.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:33.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:33 smithi053 ceph-mon[36169]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:35.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:35 smithi071 ceph-mon[36597]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:35.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:35 smithi037 ceph-mon[31830]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:35.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:35 smithi053 ceph-mon[36169]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:37 smithi037 ceph-mon[31830]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:37 smithi053 ceph-mon[36169]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T09:02:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.960+0000 7f1f18c00700 1 -- 172.21.15.37:0/363607816 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f1c0625d0 con 0x7f1f00039460 2024-03-21T09:02:37.955 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.963+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f00039460 msgr2=0x7f1f0003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.963+0000 7f1f22800700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f00039460 0x7f1f0003b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0765a0 tx=0x7f1f0c009260 comp rx=0 tx=0).stop 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.963+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 msgr2=0x7f1f1c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.963+0000 7f1f22800700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c19efa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400b7b0 tx=0x7f1f0400bbb0 comp rx=0 tx=0).stop 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 shutdown_connections 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f1c10e7b0 0x7f1f1c1a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c074bf0 0x7f1f1c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f00039460 0x7f1f0003b920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 --2- 172.21.15.37:0/363607816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f1c0761c0 0x7f1f1c19efa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:37.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 >> 172.21.15.37:0/363607816 conn(0x7f1f1c0fd730 msgr2=0x7f1f1c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:02:37.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 shutdown_connections 2024-03-21T09:02:37.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:02:37.964+0000 7f1f22800700 1 -- 172.21.15.37:0/363607816 wait complete. 2024-03-21T09:02:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:37 smithi071 ceph-mon[36597]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T09:02:38.725 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T09:02:38.729 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_4... 2024-03-21T09:02:38.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T09:02:38.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T09:02:38.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T09:02:38.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:02:38.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:38.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:38 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:38.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:38 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:38.990 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:02:39.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T09:02:39.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T09:02:39.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:02:39.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:39.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:39.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:39.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:38 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:39.910 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:39 smithi053 ceph-mon[36169]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:39.911 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:39 smithi053 ceph-mon[36169]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:02:39.911 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:39 smithi053 ceph-mon[36169]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T09:02:39.911 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:39.911 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:39 smithi037 ceph-mon[31830]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:39 smithi037 ceph-mon[31830]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:02:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:39 smithi037 ceph-mon[31830]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T09:02:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:39.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:40.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:39 smithi071 ceph-mon[36597]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:40.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:39 smithi071 ceph-mon[36597]: from='osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:02:40.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:39 smithi071 ceph-mon[36597]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T09:02:40.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:40.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:40.757 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T09:02:40.757 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T09:02:40.757 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-03-21T09:02:40.757 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-03-21T09:02:40.757 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852224 s, 1.2 GB/s 2024-03-21T09:02:40.757 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-03-21T09:02:40.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:40 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:02:40.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:40 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:02:40.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:40 smithi037 ceph-mon[31830]: osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] boot 2024-03-21T09:02:40.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:40 smithi037 ceph-mon[31830]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T09:02:40.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:40 smithi037 ceph-mon[31830]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:40.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:40 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:40.916 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:40 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:02:40.916 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:40 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:02:40.917 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:40 smithi071 ceph-mon[36597]: osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] boot 2024-03-21T09:02:40.917 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:40 smithi071 ceph-mon[36597]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T09:02:40.917 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:40 smithi071 ceph-mon[36597]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:40.917 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:40.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:40 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:02:40.980 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:40 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:02:40.981 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:40 smithi053 ceph-mon[36169]: osd.0 [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] boot 2024-03-21T09:02:40.981 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:40 smithi053 ceph-mon[36169]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T09:02:40.981 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:40 smithi053 ceph-mon[36169]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:02:40.981 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:40 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:02:41.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-03-21T09:02:41.458 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:02:41.912 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:41 smithi053 ceph-mon[36169]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T09:02:41.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:41 smithi037 ceph-mon[31830]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T09:02:42.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:41 smithi071 ceph-mon[36597]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T09:02:42.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:42 smithi037 ceph-mon[31830]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:42.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:42 smithi053 ceph-mon[36169]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:42.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 -- 172.21.15.53:0/771171082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 msgr2=0x7fd4941027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/771171082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd4941027c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4880099c0 tx=0x7fd488009cd0 comp rx=0 tx=0).stop 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 -- 172.21.15.53:0/771171082 shutdown_connections 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/771171082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 0x7fd4941095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/771171082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd494101160 0x7fd494101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/771171082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd4941027c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 -- 172.21.15.53:0/771171082 >> 172.21.15.53:0/771171082 conn(0x7fd4940fc720 msgr2=0x7fd4940feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 -- 172.21.15.53:0/771171082 shutdown_connections 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:42.999+0000 7fd49b3b4700 1 -- 172.21.15.53:0/771171082 wait complete. 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 Processor -- start 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 -- start start 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd494101160 0x7fd494105240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd494103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 0x7fd494103dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49410cd50 con 0x7fd494101160 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49410cbd0 con 0x7fd494102360 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49b3b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49410ced0 con 0x7fd494102d00 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd494103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd494103890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43772/0 (socket says 172.21.15.53:43772) 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 -- 172.21.15.53:0/2370984692 learned_addr learned my addr 172.21.15.53:0/2370984692 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 -- 172.21.15.53:0/2370984692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 msgr2=0x7fd494103dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd499951700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 0x7fd494103dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 0x7fd494103dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 -- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd494101160 msgr2=0x7fd494105240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd494101160 0x7fd494105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 -- 172.21.15.53:0/2370984692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494104680 con 0x7fd494102360 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd499951700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 0x7fd494103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.000+0000 7fd49894f700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd494103890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd488003890 tx=0x7fd488003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.001+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488027070 con 0x7fd494102360 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.001+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd488022470 con 0x7fd494102360 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.001+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4941a25b0 con 0x7fd494102360 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.001+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800f6c0 con 0x7fd494102360 2024-03-21T09:02:42.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.001+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4941a2a80 con 0x7fd494102360 2024-03-21T09:02:43.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.002+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd494068e50 con 0x7fd494102360 2024-03-21T09:02:43.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.003+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7fd48802a070 con 0x7fd494102360 2024-03-21T09:02:43.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.003+0000 7fd4927fc700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:02:43.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.003+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd4880ce980 con 0x7fd494102360 2024-03-21T09:02:43.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.004+0000 7fd499150700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:02:43.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.005+0000 7fd499150700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4941021c0 tx=0x7fd484009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:02:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.008+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd48805e740 con 0x7fd494102360 2024-03-21T09:02:43.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:42 smithi071 ceph-mon[36597]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:43.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:02:43.224+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd49406afa0 con 0x7fd47c0a2fa0 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi037 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:43.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:43.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi037 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:43.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:44.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:02:44.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:02:44.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:02:44.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi037 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:44.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:44.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:44 smithi053 ceph-mon[36169]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:44.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:44 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi037 to 17095M 2024-03-21T09:02:44.733 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:44 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:45.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:44 smithi071 ceph-mon[36597]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:45.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:44 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi037 to 17095M 2024-03-21T09:02:45.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:44 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:44 smithi037 ceph-mon[31830]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:44 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi037 to 17095M 2024-03-21T09:02:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:44 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/884568858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]: dispatch 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]: dispatch 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]': finished 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:02:46.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:46 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/245429611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:02:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/884568858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]: dispatch 2024-03-21T09:02:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]: dispatch 2024-03-21T09:02:47.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]': finished 2024-03-21T09:02:47.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T09:02:47.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:02:47.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:46 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/245429611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:02:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/884568858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]: dispatch 2024-03-21T09:02:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]: dispatch 2024-03-21T09:02:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97aea20a-8dd3-4a5d-a5a5-341cd7508568"}]': finished 2024-03-21T09:02:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T09:02:47.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:02:47.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:46 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/245429611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:02:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:48 smithi071 ceph-mon[36597]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:48 smithi037 ceph-mon[31830]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:48 smithi053 ceph-mon[36169]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:51.036 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:50 smithi053 ceph-mon[36169]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:51.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:50 smithi071 ceph-mon[36597]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:51.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:50 smithi037 ceph-mon[31830]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:52.871 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:52 smithi053 ceph-mon[36169]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:52 smithi071 ceph-mon[36597]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:53.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:52 smithi037 ceph-mon[31830]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:53.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T09:02:53.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:53.964 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:53 smithi053 ceph-mon[36169]: Deploying daemon osd.1 on smithi053 2024-03-21T09:02:54.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T09:02:54.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:54.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:53 smithi071 ceph-mon[36597]: Deploying daemon osd.1 on smithi053 2024-03-21T09:02:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T09:02:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:53 smithi037 ceph-mon[31830]: Deploying daemon osd.1 on smithi053 2024-03-21T09:02:55.054 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:54 smithi053 ceph-mon[36169]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:55.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:54 smithi071 ceph-mon[36597]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:55.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:54 smithi037 ceph-mon[31830]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:56.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:56 smithi053 ceph-mon[36169]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:57.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:56 smithi071 ceph-mon[36597]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:57.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:56 smithi037 ceph-mon[31830]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:59.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:58 smithi037 ceph-mon[31830]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:59.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:58 smithi071 ceph-mon[36597]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:02:59.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:58 smithi053 ceph-mon[36169]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:00.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:00.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:00.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:02:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:00.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:00.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:00.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:02:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:00.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:00.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:00.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:02:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:01.125 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:00 smithi053 ceph-mon[36169]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:01.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:00 smithi037 ceph-mon[31830]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:00 smithi071 ceph-mon[36597]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:02.317 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-03-21T09:03:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.321+0000 7fd4927fc700 1 -- 172.21.15.53:0/2370984692 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd49406afa0 con 0x7fd47c0a2fa0 2024-03-21T09:03:02.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd47c0a2fa0 msgr2=0x7fd47c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:02.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4941021c0 tx=0x7fd484009250 comp rx=0 tx=0).stop 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 msgr2=0x7fd494103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd494103890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd488003890 tx=0x7fd488003730 comp rx=0 tx=0).stop 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 shutdown_connections 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd494102d00 0x7fd494103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd494101160 0x7fd494105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 --2- 172.21.15.53:0/2370984692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd494102360 0x7fd494103890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 >> 172.21.15.53:0/2370984692 conn(0x7fd4940fc720 msgr2=0x7fd49406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 shutdown_connections 2024-03-21T09:03:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:03:02.324+0000 7fd49b3b4700 1 -- 172.21.15.53:0/2370984692 wait complete. 2024-03-21T09:03:02.951 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T09:03:02.954 INFO:tasks.cephadm:Deploying osd.2 on smithi071 with /dev/vg_nvme/lv_4... 2024-03-21T09:03:02.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T09:03:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:03:03.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:03.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.505 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: from='osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T09:03:03.505 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:03 smithi071 ceph-mon[36597]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T09:03:03.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:03.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: from='osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T09:03:03.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:03 smithi037 ceph-mon[31830]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: from='osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T09:03:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:03 smithi053 ceph-mon[36169]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T09:03:04.392 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-mon[36169]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T09:03:04.392 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-mon[36169]: from='osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:03:04.393 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-mon[36169]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T09:03:04.393 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:04.393 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-mon[36169]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:03:04.393 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1[40766]: 2024-03-21T09:03:04.366+0000 7f5b13dee700 -1 osd.1 0 waiting for initial osdmap 2024-03-21T09:03:04.393 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1[40766]: 2024-03-21T09:03:04.369+0000 7f5b0fbe1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:03:04.578 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:04 smithi071 ceph-mon[36597]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T09:03:04.579 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:04 smithi071 ceph-mon[36597]: from='osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:03:04.579 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:04 smithi071 ceph-mon[36597]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T09:03:04.579 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:04 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:04.579 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:04 smithi071 ceph-mon[36597]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:03:04.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:04 smithi037 ceph-mon[31830]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T09:03:04.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:04 smithi037 ceph-mon[31830]: from='osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:03:04.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:04 smithi037 ceph-mon[31830]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T09:03:04.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:04 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:04.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:04 smithi037 ceph-mon[31830]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:03:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T09:03:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T09:03:05.096 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-21T09:03:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-21T09:03:05.096 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112597 s, 931 MB/s 2024-03-21T09:03:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-21T09:03:05.640 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:05 smithi071 ceph-mon[36597]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:05.640 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:05 smithi071 ceph-mon[36597]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:03:05.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:05 smithi071 ceph-mon[36597]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T09:03:05.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:05.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:05.665 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:05 smithi053 ceph-mon[36169]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:05.665 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:05 smithi053 ceph-mon[36169]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:03:05.665 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:05 smithi053 ceph-mon[36169]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T09:03:05.665 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:05.665 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:05.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-03-21T09:03:05.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:05 smithi037 ceph-mon[31830]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:05.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:05 smithi037 ceph-mon[31830]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:03:05.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:05 smithi037 ceph-mon[31830]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T09:03:05.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:05.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:05.911 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:03:06.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] boot 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:06.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] boot 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: osd.1 [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] boot 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 -- 172.21.15.71:0/1317478875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24102c50 msgr2=0x7f8b24105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 --2- 172.21.15.71:0/1317478875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24102c50 0x7f8b24105040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009990 tx=0x7f8b20009ca0 comp rx=0 tx=0).stop 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 -- 172.21.15.71:0/1317478875 shutdown_connections 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 --2- 172.21.15.71:0/1317478875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b24107eb0 0x7f8b2410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 --2- 172.21.15.71:0/1317478875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24102c50 0x7f8b24105040 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 --2- 172.21.15.71:0/1317478875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b24105580 0x7f8b24107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 -- 172.21.15.71:0/1317478875 >> 172.21.15.71:0/1317478875 conn(0x7f8b240fc660 msgr2=0x7f8b240feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 -- 172.21.15.71:0/1317478875 shutdown_connections 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 -- 172.21.15.71:0/1317478875 wait complete. 2024-03-21T09:03:07.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 Processor -- start 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.482+0000 7f8b2b777700 1 -- start start 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b24102c50 0x7f8b2419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 0x7f8b2419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b24107eb0 0x7f8b241a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2410de10 con 0x7f8b24107eb0 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2410dc90 con 0x7f8b24102c50 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2410df90 con 0x7f8b24105580 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 0x7f8b2419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 0x7f8b2419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36934/0 (socket says 172.21.15.71:36934) 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 -- 172.21.15.71:0/587275972 learned_addr learned my addr 172.21.15.71:0/587275972 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b29513700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b24102c50 0x7f8b2419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 -- 172.21.15.71:0/587275972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b24102c50 msgr2=0x7f8b2419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b24102c50 0x7f8b2419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 -- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b24107eb0 msgr2=0x7f8b241a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b24107eb0 0x7f8b241a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 -- 172.21.15.71:0/587275972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b241a2310 con 0x7f8b24105580 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b28d12700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 0x7f8b2419abf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b560 tx=0x7f8b1400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14015020 con 0x7f8b24105580 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b241a2600 con 0x7f8b24105580 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b14011d10 con 0x7f8b24105580 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.483+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400f2e0 con 0x7f8b24105580 2024-03-21T09:03:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.484+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b241a2b40 con 0x7f8b24105580 2024-03-21T09:03:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.485+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b24068e50 con 0x7f8b24105580 2024-03-21T09:03:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.485+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f8b14018020 con 0x7f8b24105580 2024-03-21T09:03:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.486+0000 7f8b1a7fc700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b100a2f20 0x7f8b100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.486+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f8b140cd3a0 con 0x7f8b24105580 2024-03-21T09:03:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.486+0000 7f8b29513700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b100a2f20 0x7f8b100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:07.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.487+0000 7f8b29513700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b100a2f20 0x7f8b100a53e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b200039a0 tx=0x7f8b20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:03:07.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.491+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b1401f020 con 0x7f8b24105580 2024-03-21T09:03:07.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:07 smithi071 ceph-mon[36597]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:07.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:07 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi053 2024-03-21T09:03:07.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:07 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi053 to 17084M 2024-03-21T09:03:07.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:07 smithi071 ceph-mon[36597]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T09:03:07.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:07 smithi037 ceph-mon[31830]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:07.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:07 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi053 2024-03-21T09:03:07.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:07 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi053 to 17084M 2024-03-21T09:03:07.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:07 smithi037 ceph-mon[31830]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T09:03:07.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:07.694+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b240625d0 con 0x7f8b100a2f20 2024-03-21T09:03:07.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:07 smithi053 ceph-mon[36169]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T09:03:07.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:07 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi053 2024-03-21T09:03:07.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:07 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi053 to 17084M 2024-03-21T09:03:07.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:07 smithi053 ceph-mon[36169]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T09:03:08.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:08 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:03:08.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:08 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:03:08.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:08 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:08.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:08 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:03:08.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:08 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:03:08.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:08 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:08 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:03:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:08 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:03:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:08 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:09.467 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:09 smithi071 ceph-mon[36597]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:09.467 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:09 smithi071 ceph-mon[36597]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:03:09.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:09 smithi037 ceph-mon[31830]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:09.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:09 smithi037 ceph-mon[31830]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:03:09.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:09 smithi053 ceph-mon[36169]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:09.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:09 smithi053 ceph-mon[36169]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:03:10.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:10 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/187672943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]: dispatch 2024-03-21T09:03:10.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:10 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]: dispatch 2024-03-21T09:03:10.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:10 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]': finished 2024-03-21T09:03:10.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:10 smithi037 ceph-mon[31830]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T09:03:10.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:10 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:10 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/187672943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]: dispatch 2024-03-21T09:03:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:10 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]: dispatch 2024-03-21T09:03:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:10 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]': finished 2024-03-21T09:03:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:10 smithi053 ceph-mon[36169]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T09:03:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:10 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:10.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:10 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/187672943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]: dispatch 2024-03-21T09:03:10.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:10 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]: dispatch 2024-03-21T09:03:10.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:10 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d5b09e0-acc1-488c-8e3d-34c05699a51e"}]': finished 2024-03-21T09:03:10.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:10 smithi071 ceph-mon[36597]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T09:03:10.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:10 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:11.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:11 smithi037 ceph-mon[31830]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:11.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:11 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/1728808106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:03:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:11 smithi053 ceph-mon[36169]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:11.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:11 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/1728808106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:03:11.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:11 smithi071 ceph-mon[36597]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:11.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:11 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/1728808106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:03:13.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:13 smithi037 ceph-mon[31830]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:13 smithi053 ceph-mon[36169]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:13.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:13 smithi071 ceph-mon[36597]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:15 smithi053 ceph-mon[36169]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:15.776 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:15 smithi071 ceph-mon[36597]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:15.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:15 smithi037 ceph-mon[31830]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:17.691 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:17 smithi071 ceph-mon[36597]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:17.691 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T09:03:17.691 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:17.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:17 smithi053 ceph-mon[36169]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:17.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T09:03:17.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:17.726 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:17 smithi037 ceph-mon[31830]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:17.726 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T09:03:17.726 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:18.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:18 smithi053 ceph-mon[36169]: Deploying daemon osd.2 on smithi071 2024-03-21T09:03:18.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:18 smithi071 ceph-mon[36597]: Deploying daemon osd.2 on smithi071 2024-03-21T09:03:18.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:18 smithi037 ceph-mon[31830]: Deploying daemon osd.2 on smithi071 2024-03-21T09:03:19.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:19 smithi053 ceph-mon[36169]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:19.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:19 smithi071 ceph-mon[36597]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:19.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:19 smithi037 ceph-mon[31830]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:21.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:21 smithi071 ceph-mon[36597]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:21.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:21 smithi037 ceph-mon[31830]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:22.026 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:21 smithi053 ceph-mon[36169]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:23.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:23 smithi053 ceph-mon[36169]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:23.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:23.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:23.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:23.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:23 smithi071 ceph-mon[36597]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:23.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:23.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:23.585 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:23.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:23 smithi037 ceph-mon[31830]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:23.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:23.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:23.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:25.285 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:25 smithi071 ceph-mon[36597]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:25.286 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.286 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.286 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:25.286 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:25.286 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:25 smithi053 ceph-mon[36169]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.629+0000 7f8b1a7fc700 1 -- 172.21.15.71:0/587275972 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8b240625d0 con 0x7f8b100a2f20 2024-03-21T09:03:25.625 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 2 on host 'smithi071' 2024-03-21T09:03:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b100a2f20 msgr2=0x7f8b100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b100a2f20 0x7f8b100a53e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b200039a0 tx=0x7f8b20005fd0 comp rx=0 tx=0).stop 2024-03-21T09:03:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 msgr2=0x7f8b2419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 0x7f8b2419abf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b560 tx=0x7f8b1400b920 comp rx=0 tx=0).stop 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 shutdown_connections 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b24107eb0 0x7f8b241a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b100a2f20 0x7f8b100a53e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b24102c50 0x7f8b2419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 --2- 172.21.15.71:0/587275972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b24105580 0x7f8b2419abf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 >> 172.21.15.71:0/587275972 conn(0x7f8b240fc660 msgr2=0x7f8b24102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.631+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 shutdown_connections 2024-03-21T09:03:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:03:25.632+0000 7f8b2b777700 1 -- 172.21.15.71:0/587275972 wait complete. 2024-03-21T09:03:25.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:25 smithi037 ceph-mon[31830]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:25.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:25.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:25.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:25.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.224 DEBUG:teuthology.orchestra.run.smithi071:osd.2> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T09:03:26.226 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-03-21T09:03:26.227 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T09:03:26.686 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:03:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:26 smithi071 ceph-mon[36597]: from='osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T09:03:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:26 smithi071 ceph-mon[36597]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T09:03:26.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:26.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:26 smithi037 ceph-mon[31830]: from='osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T09:03:26.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:26 smithi037 ceph-mon[31830]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T09:03:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:26 smithi053 ceph-mon[36169]: from='osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T09:03:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:26 smithi053 ceph-mon[36169]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T09:03:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:27 smithi071 ceph-mon[36597]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:27 smithi071 ceph-mon[36597]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T09:03:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:27 smithi071 ceph-mon[36597]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T09:03:27.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:27 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:27.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:27 smithi071 ceph-mon[36597]: from='osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:03:27.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:27 smithi071 ceph-mon[36597]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:03:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:27 smithi037 ceph-mon[31830]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:27 smithi037 ceph-mon[31830]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T09:03:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:27 smithi037 ceph-mon[31830]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T09:03:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:27 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:27 smithi037 ceph-mon[31830]: from='osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:03:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:27 smithi037 ceph-mon[31830]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:03:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:27 smithi053 ceph-mon[36169]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:27 smithi053 ceph-mon[36169]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T09:03:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:27 smithi053 ceph-mon[36169]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T09:03:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:27 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:27 smithi053 ceph-mon[36169]: from='osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:03:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:27 smithi053 ceph-mon[36169]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:03:28.353 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2[41771]: 2024-03-21T09:03:28.085+0000 7f5bccc37700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T09:03:28.353 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2[41771]: 2024-03-21T09:03:28.088+0000 7f5bc4a22700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:03:28.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:28 smithi037 ceph-mon[31830]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:03:28.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:28 smithi037 ceph-mon[31830]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T09:03:28.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:28.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:28.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:28 smithi037 ceph-mon[31830]: from='osd.2 ' entity='osd.2' 2024-03-21T09:03:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:28 smithi053 ceph-mon[36169]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:03:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:28 smithi053 ceph-mon[36169]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T09:03:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:28 smithi053 ceph-mon[36169]: from='osd.2 ' entity='osd.2' 2024-03-21T09:03:29.001 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-mon[36597]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:03:29.002 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-mon[36597]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T09:03:29.002 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:29.002 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:29.002 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:28 smithi071 ceph-mon[36597]: from='osd.2 ' entity='osd.2' 2024-03-21T09:03:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T09:03:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T09:03:29.546 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-21T09:03:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-21T09:03:29.546 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0522379 s, 201 MB/s 2024-03-21T09:03:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-21T09:03:29.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:03:29.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:03:29.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:29.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] boot 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:29.842 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] boot 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:29.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:29.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:29.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:29.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:30.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: osd.2 [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] boot 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:30.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:30.387 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-03-21T09:03:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:03:30.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi071 2024-03-21T09:03:30.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi071 to 17080M 2024-03-21T09:03:30.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:30.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:03:30.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T09:03:30.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T09:03:30.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:30 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi071 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi071 to 17080M 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T09:03:30.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:30 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi071 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi071 to 17080M 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T09:03:31.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:30 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:03:32.174 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T09:03:32.174 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:32.174 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48530]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:32.174 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48530]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:32.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:03:32.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:32 smithi037 ceph-mon[31830]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T09:03:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:03:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:32 smithi071 ceph-mon[36597]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T09:03:32.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:03:32.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:32 smithi053 ceph-mon[36169]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T09:03:32.750 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48557]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T09:03:32.751 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48557]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:32.751 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48557]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:32.751 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:03:32 smithi037 sudo[48557]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.926+0000 7f4ef3a00700 1 -- 172.21.15.37:0/1176541735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102c50 msgr2=0x7f4eec105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.926+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/1176541735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102c50 0x7f4eec105040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009f90 tx=0x7f4ee000b330 comp rx=0 tx=0).stop 2024-03-21T09:03:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 -- 172.21.15.37:0/1176541735 shutdown_connections 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/1176541735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec107eb0 0x7f4eec10a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/1176541735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102c50 0x7f4eec105040 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/1176541735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec105580 0x7f4eec107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 -- 172.21.15.37:0/1176541735 >> 172.21.15.37:0/1176541735 conn(0x7f4eec0fc660 msgr2=0x7f4eec0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 -- 172.21.15.37:0/1176541735 shutdown_connections 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.927+0000 7f4ef3a00700 1 -- 172.21.15.37:0/1176541735 wait complete. 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.928+0000 7f4ef3a00700 1 Processor -- start 2024-03-21T09:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.928+0000 7f4ef3a00700 1 -- start start 2024-03-21T09:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef3a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec102c50 0x7f4eec19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef3a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 0x7f4eec19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef3a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec107eb0 0x7f4eec1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef3a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec10de20 con 0x7f4eec107eb0 2024-03-21T09:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef3a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec10dca0 con 0x7f4eec105580 2024-03-21T09:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef3a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec10dfa0 con 0x7f4eec102c50 2024-03-21T09:03:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 0x7f4eec19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 0x7f4eec19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:43788/0 (socket says 172.21.15.37:43788) 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 -- 172.21.15.37:0/4139724026 learned_addr learned my addr 172.21.15.37:0/4139724026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef1000700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec102c50 0x7f4eec19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4ef1a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec107eb0 0x7f4eec1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 -- 172.21.15.37:0/4139724026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec102c50 msgr2=0x7f4eec19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec102c50 0x7f4eec19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 -- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec107eb0 msgr2=0x7f4eec1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.929+0000 7f4eebe00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec107eb0 0x7f4eec1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4eebe00700 1 -- 172.21.15.37:0/4139724026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec1a2320 con 0x7f4eec105580 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4ef1000700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec102c50 0x7f4eec19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4eebe00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 0x7f4eec19ac00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800e9c0 tx=0x7f4ed800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed800cbd0 con 0x7f4eec105580 2024-03-21T09:03:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec1a2610 con 0x7f4eec105580 2024-03-21T09:03:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ed800ee60 con 0x7f4eec105580 2024-03-21T09:03:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8005680 con 0x7f4eec105580 2024-03-21T09:03:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.930+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec1a2b00 con 0x7f4eec105580 2024-03-21T09:03:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.932+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec068e50 con 0x7f4eec105580 2024-03-21T09:03:32.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.934+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4ed8004750 con 0x7f4eec105580 2024-03-21T09:03:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.935+0000 7f4ee9600700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4ed40a2fb0 0x7f4ed40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:03:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.935+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f4ed80ce2b0 con 0x7f4eec105580 2024-03-21T09:03:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.935+0000 7f4ef1000700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4ed40a2fb0 0x7f4ed40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:03:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.936+0000 7f4ef1000700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4ed40a2fb0 0x7f4ed40a5470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0007f50 tx=0x7f4ee00094b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:03:32.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:32.947+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ed809c410 con 0x7f4eec105580 2024-03-21T09:03:33.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:33.184+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eec0625d0 con 0x7f4ed40a2fb0 2024-03-21T09:03:33.442 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44929]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T09:03:33.442 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44929]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:33.442 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44929]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:33.442 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44929]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:33.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: pgmap v93: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:33.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T09:03:33.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T09:03:33.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:33.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:03:33.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:03:33.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:33.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: pgmap v93: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:33.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T09:03:33.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T09:03:33.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:33.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:03:33.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:03:33.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: pgmap v93: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T09:03:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T09:03:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:03:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:03:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:33.942 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44932]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T09:03:33.942 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44932]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:33.942 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44932]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:33.942 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:03:33 smithi053 sudo[44932]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:34.180 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:33 smithi071 sudo[45562]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T09:03:34.180 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:33 smithi071 sudo[45562]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:34.180 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:33 smithi071 sudo[45562]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:34.180 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:33 smithi071 sudo[45562]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:34.603 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:34 smithi071 sudo[45565]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T09:03:34.603 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:34 smithi071 sudo[45565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:34.603 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:34 smithi071 sudo[45565]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:34.603 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:03:34 smithi071 sudo[45565]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:35.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:34 smithi037 sudo[48722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T09:03:35.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:34 smithi037 sudo[48722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:35.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:34 smithi037 sudo[48722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:35.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:34 smithi037 sudo[48722]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:35.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 sudo[48729]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T09:03:35.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 sudo[48729]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:35.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 sudo[48729]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:35.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 sudo[48729]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:35.644 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 ceph-mon[31830]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:03:35.644 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:35.644 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:35.644 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:35 smithi037 ceph-mon[31830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:35.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 ceph-mon[36597]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:03:35.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:35.645 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:35.645 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:35.645 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45568]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T09:03:35.645 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:35.645 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45568]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:35.645 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45568]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:35 smithi053 ceph-mon[36169]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:03:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:35 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:35 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:35 smithi053 ceph-mon[36169]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:36.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T09:03:36.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:36.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45571]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:36.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:35 smithi071 sudo[45571]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:36.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44935]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T09:03:36.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44935]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:36.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44935]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:36.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44935]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:37.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: pgmap v95: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:37.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:37.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: pgmap v95: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:37.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44938]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44938]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 sudo[44938]: pam_unix(sudo:session): session closed for user root 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: pgmap v95: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:37.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:37.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:37.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:03:37.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:03:37.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:03:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: pgmap v96: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/897661555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]: dispatch 2024-03-21T09:03:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]: dispatch 2024-03-21T09:03:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]': finished 2024-03-21T09:03:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T09:03:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: pgmap v96: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/897661555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]: dispatch 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]: dispatch 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]': finished 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T09:03:38.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: pgmap v96: 1 pgs: 1 active+clean; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/897661555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]: dispatch 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]: dispatch 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "879c9049-bfc3-41fa-be8e-354edcc68a14"}]': finished 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T09:03:38.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:03:39.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:38 smithi071 ceph-mon[36597]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 16 KiB/s rd, 446 KiB/s wr, 43 op/s 2024-03-21T09:03:39.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2662963938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:03:39.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:38 smithi037 ceph-mon[31830]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 16 KiB/s rd, 446 KiB/s wr, 43 op/s 2024-03-21T09:03:39.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2662963938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:03:39.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:38 smithi053 ceph-mon[36169]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 16 KiB/s rd, 446 KiB/s wr, 43 op/s 2024-03-21T09:03:39.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2662963938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:03:41.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:40 smithi071 ceph-mon[36597]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 334 KiB/s wr, 32 op/s 2024-03-21T09:03:41.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:40 smithi037 ceph-mon[31830]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 334 KiB/s wr, 32 op/s 2024-03-21T09:03:41.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:40 smithi053 ceph-mon[36169]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 334 KiB/s wr, 32 op/s 2024-03-21T09:03:43.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:42 smithi071 ceph-mon[36597]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 315 KiB/s wr, 30 op/s 2024-03-21T09:03:43.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:42 smithi037 ceph-mon[31830]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 315 KiB/s wr, 30 op/s 2024-03-21T09:03:43.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:42 smithi053 ceph-mon[36169]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 315 KiB/s wr, 30 op/s 2024-03-21T09:03:45.062 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:44 smithi037 ceph-mon[31830]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-03-21T09:03:45.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:44 smithi071 ceph-mon[36597]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-03-21T09:03:45.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:44 smithi053 ceph-mon[36169]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-03-21T09:03:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:46 smithi071 ceph-mon[36597]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-03-21T09:03:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T09:03:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:46 smithi071 ceph-mon[36597]: Deploying daemon osd.3 on smithi037 2024-03-21T09:03:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:46 smithi037 ceph-mon[31830]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-03-21T09:03:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T09:03:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:46 smithi037 ceph-mon[31830]: Deploying daemon osd.3 on smithi037 2024-03-21T09:03:47.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:46 smithi053 ceph-mon[36169]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-03-21T09:03:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T09:03:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:46 smithi053 ceph-mon[36169]: Deploying daemon osd.3 on smithi037 2024-03-21T09:03:48.849 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:48 smithi037 ceph-mon[31830]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 975 B/s rd, 25 KiB/s wr, 2 op/s 2024-03-21T09:03:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:48 smithi071 ceph-mon[36597]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 975 B/s rd, 25 KiB/s wr, 2 op/s 2024-03-21T09:03:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:48 smithi053 ceph-mon[36169]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 975 B/s rd, 25 KiB/s wr, 2 op/s 2024-03-21T09:03:51.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:50 smithi071 ceph-mon[36597]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T09:03:51.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:50 smithi037 ceph-mon[31830]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T09:03:51.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:50 smithi053 ceph-mon[36169]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T09:03:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:52 smithi071 ceph-mon[36597]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T09:03:53.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:52 smithi037 ceph-mon[31830]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T09:03:53.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:52 smithi053 ceph-mon[36169]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T09:03:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:54 smithi037 ceph-mon[31830]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:55.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:54 smithi071 ceph-mon[36597]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:55.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:54 smithi053 ceph-mon[36169]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:56.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:56.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:56.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:56.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:55 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:56.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:55 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:56.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:55 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:56.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:56.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:56.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:57.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:56 smithi037 ceph-mon[31830]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:57.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:56 smithi053 ceph-mon[36169]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:57.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:56 smithi071 ceph-mon[36597]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:59.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:58 smithi071 ceph-mon[36597]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:59.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:59.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:59.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:58 smithi037 ceph-mon[31830]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:59.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:59.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:59.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:58 smithi053 ceph-mon[36169]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:03:59.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:59.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:59.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:03:59.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.938+0000 7f4ee9600700 1 -- 172.21.15.37:0/4139724026 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4eec0625d0 con 0x7f4ed40a2fb0 2024-03-21T09:03:59.935 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-03-21T09:03:59.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4ed40a2fb0 msgr2=0x7f4ed40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4ed40a2fb0 0x7f4ed40a5470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0007f50 tx=0x7f4ee00094b0 comp rx=0 tx=0).stop 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 msgr2=0x7f4eec19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 0x7f4eec19ac00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800e9c0 tx=0x7f4ed800ecd0 comp rx=0 tx=0).stop 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 shutdown_connections 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec107eb0 0x7f4eec1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eec102c50 0x7f4eec19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4ed40a2fb0 0x7f4ed40a5470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 --2- 172.21.15.37:0/4139724026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eec105580 0x7f4eec19ac00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 >> 172.21.15.37:0/4139724026 conn(0x7f4eec0fc660 msgr2=0x7f4eec102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 shutdown_connections 2024-03-21T09:03:59.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:03:59.942+0000 7f4ef3a00700 1 -- 172.21.15.37:0/4139724026 wait complete. 2024-03-21T09:04:00.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:59 smithi071 ceph-mon[36597]: from='osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T09:04:00.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:03:59 smithi071 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T09:04:00.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:59 smithi037 ceph-mon[31830]: from='osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T09:04:00.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:03:59 smithi037 ceph-mon[31830]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T09:04:00.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:59 smithi053 ceph-mon[36169]: from='osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T09:04:00.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:03:59 smithi053 ceph-mon[36169]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T09:04:00.779 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T09:04:00.785 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-03-21T09:04:00.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T09:04:00.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:04:00.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T09:04:00.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:04:00.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T09:04:00.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:04:00.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:00.990 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:00.990 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:00.990 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:01.045 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:04:01.167 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:04:01.167 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T09:04:01.167 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:04:01.168 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T09:04:01.168 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:04:01.168 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:01.168 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:01.168 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:01.168 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:01.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:01.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:01.674 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:04:01.497+0000 7f678e800700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T09:04:01.675 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:04:01.530+0000 7f6783c00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:04:02.080 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:04:02.080 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:04:02.081 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:04:02.081 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T09:04:02.081 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.081 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.081 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:01 smithi037 ceph-mon[31830]: from='osd.3 ' entity='osd.3' 2024-03-21T09:04:02.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:04:02.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:04:02.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:04:02.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T09:04:02.153 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.154 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.154 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:01 smithi053 ceph-mon[36169]: from='osd.3 ' entity='osd.3' 2024-03-21T09:04:02.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:04:02.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:04:02.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:04:02.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T09:04:02.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:01 smithi071 ceph-mon[36597]: from='osd.3 ' entity='osd.3' 2024-03-21T09:04:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:02 smithi037 ceph-mon[31830]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:04:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:02 smithi037 ceph-mon[31830]: osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] boot 2024-03-21T09:04:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:02 smithi037 ceph-mon[31830]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T09:04:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:02.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:02 smithi037 ceph-mon[31830]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T09:04:03.024 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T09:04:03.024 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T09:04:03.024 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-03-21T09:04:03.025 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-03-21T09:04:03.025 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00854368 s, 1.2 GB/s 2024-03-21T09:04:03.025 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-03-21T09:04:03.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:02 smithi053 ceph-mon[36169]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:04:03.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:02 smithi053 ceph-mon[36169]: osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] boot 2024-03-21T09:04:03.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:02 smithi053 ceph-mon[36169]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T09:04:03.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:03.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:02 smithi053 ceph-mon[36169]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T09:04:03.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:02 smithi071 ceph-mon[36597]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T09:04:03.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:02 smithi071 ceph-mon[36597]: osd.3 [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] boot 2024-03-21T09:04:03.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:02 smithi071 ceph-mon[36597]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T09:04:03.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:04:03.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:02 smithi071 ceph-mon[36597]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T09:04:03.663 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-03-21T09:04:03.916 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:04:05.010 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:04 smithi053 ceph-mon[36169]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:05.010 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:04 smithi053 ceph-mon[36169]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T09:04:05.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:04 smithi037 ceph-mon[31830]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:05.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:04 smithi037 ceph-mon[31830]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T09:04:05.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:04 smithi071 ceph-mon[36597]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:05.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:04 smithi071 ceph-mon[36597]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T09:04:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 -- 172.21.15.53:0/4244583425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c103d60 msgr2=0x7fbf7c10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 --2- 172.21.15.53:0/4244583425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c103d60 0x7fbf7c10a610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400b0d0 tx=0x7fbf7400b3e0 comp rx=0 tx=0).stop 2024-03-21T09:04:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 -- 172.21.15.53:0/4244583425 shutdown_connections 2024-03-21T09:04:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 --2- 172.21.15.53:0/4244583425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c103d60 0x7fbf7c10a610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 --2- 172.21.15.53:0/4244583425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c1021c0 0x7fbf7c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 --2- 172.21.15.53:0/4244583425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.553+0000 7fbf846ba700 1 -- 172.21.15.53:0/4244583425 >> 172.21.15.53:0/4244583425 conn(0x7fbf7c0fd7a0 msgr2=0x7fbf7c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 -- 172.21.15.53:0/4244583425 shutdown_connections 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 -- 172.21.15.53:0/4244583425 wait complete. 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 Processor -- start 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 -- start start 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c1021c0 0x7fbf7c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7c103d60 0x7fbf7c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c10de30 con 0x7fbf7c103d60 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c10dcb0 con 0x7fbf7c1033c0 2024-03-21T09:04:05.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf846ba700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c10dfb0 con 0x7fbf7c1021c0 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf81c55700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf81c55700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43702/0 (socket says 172.21.15.53:43702) 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.554+0000 7fbf81c55700 1 -- 172.21.15.53:0/2976369551 learned_addr learned my addr 172.21.15.53:0/2976369551 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf82456700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c1021c0 0x7fbf7c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf81c55700 1 -- 172.21.15.53:0/2976369551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c1021c0 msgr2=0x7fbf7c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf81c55700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c1021c0 0x7fbf7c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf81c55700 1 -- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7c103d60 msgr2=0x7fbf7c074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf81c55700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7c103d60 0x7fbf7c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf81c55700 1 -- 172.21.15.53:0/2976369551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c075370 con 0x7fbf7c1033c0 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf81c55700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c074580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b530 tx=0x7fbf6c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c015020 con 0x7fbf7c1033c0 2024-03-21T09:04:05.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf6c011d10 con 0x7fbf7c1033c0 2024-03-21T09:04:05.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c010e50 con 0x7fbf7c1033c0 2024-03-21T09:04:05.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c1a2550 con 0x7fbf7c1033c0 2024-03-21T09:04:05.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.555+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c1a2b10 con 0x7fbf7c1033c0 2024-03-21T09:04:05.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.556+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c068e50 con 0x7fbf7c1033c0 2024-03-21T09:04:05.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.557+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbf6c018020 con 0x7fbf7c1033c0 2024-03-21T09:04:05.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.558+0000 7fbf737fe700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf680a3250 0x7fbf680a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:05.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.558+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3669+0+0 (secure 0 0 0) 0x7fbf6c0cda20 con 0x7fbf7c1033c0 2024-03-21T09:04:05.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.558+0000 7fbf82456700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf680a3250 0x7fbf680a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:05.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.559+0000 7fbf82456700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf680a3250 0x7fbf680a5710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005950 tx=0x7fbf780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:04:05.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.562+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf6c01f020 con 0x7fbf7c1033c0 2024-03-21T09:04:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:05.777+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf7c0625d0 con 0x7fbf680a3250 2024-03-21T09:04:06.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T09:04:06.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi037 2024-03-21T09:04:06.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi037 to 8547M 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:06.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:06.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:05 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:06.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T09:04:06.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi037 2024-03-21T09:04:06.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi037 to 8547M 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:06.194 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:06.194 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:05 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:06.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T09:04:06.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi037 2024-03-21T09:04:06.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi037 to 8547M 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:06.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:05 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:07.111 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:06 smithi053 ceph-mon[36169]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:07.111 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:06 smithi053 ceph-mon[36169]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:07.112 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:06 smithi053 ceph-mon[36169]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T09:04:07.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:06 smithi037 ceph-mon[31830]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:07.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:06 smithi037 ceph-mon[31830]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:07.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:06 smithi037 ceph-mon[31830]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T09:04:07.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:06 smithi071 ceph-mon[36597]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:07.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:06 smithi071 ceph-mon[36597]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:07.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:06 smithi071 ceph-mon[36597]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T09:04:09.040 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:08 smithi053 ceph-mon[36169]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-03-21T09:04:09.040 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:08 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/2216463959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]: dispatch 2024-03-21T09:04:09.040 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:08 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]: dispatch 2024-03-21T09:04:09.040 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:08 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]': finished 2024-03-21T09:04:09.040 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:08 smithi053 ceph-mon[36169]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T09:04:09.040 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:08 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:09.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:08 smithi037 ceph-mon[31830]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-03-21T09:04:09.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:08 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/2216463959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]: dispatch 2024-03-21T09:04:09.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:08 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]: dispatch 2024-03-21T09:04:09.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:08 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]': finished 2024-03-21T09:04:09.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:08 smithi037 ceph-mon[31830]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T09:04:09.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:08 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:08 smithi071 ceph-mon[36597]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-03-21T09:04:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:08 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/2216463959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]: dispatch 2024-03-21T09:04:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:08 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]: dispatch 2024-03-21T09:04:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:08 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "721729f1-98cf-4ce4-b0e6-3a0d9736d2d0"}]': finished 2024-03-21T09:04:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:08 smithi071 ceph-mon[36597]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T09:04:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:08 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:09 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/342655087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:04:10.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:09 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/342655087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:04:10.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:09 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/342655087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:04:11.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:10 smithi037 ceph-mon[31830]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-21T09:04:11.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:10 smithi053 ceph-mon[36169]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-21T09:04:11.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:10 smithi071 ceph-mon[36597]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-21T09:04:13.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:12 smithi053 ceph-mon[36169]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-21T09:04:13.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:12 smithi037 ceph-mon[31830]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-21T09:04:13.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:12 smithi071 ceph-mon[36597]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-21T09:04:15.025 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:14 smithi053 ceph-mon[36169]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T09:04:15.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:14 smithi071 ceph-mon[36597]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T09:04:15.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:14 smithi037 ceph-mon[31830]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T09:04:17.147 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:16 smithi053 ceph-mon[36169]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-21T09:04:17.147 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T09:04:17.147 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:17.147 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:16 smithi053 ceph-mon[36169]: Deploying daemon osd.4 on smithi053 2024-03-21T09:04:17.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:16 smithi037 ceph-mon[31830]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-21T09:04:17.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T09:04:17.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:17.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:16 smithi037 ceph-mon[31830]: Deploying daemon osd.4 on smithi053 2024-03-21T09:04:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:16 smithi071 ceph-mon[36597]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-21T09:04:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T09:04:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:16 smithi071 ceph-mon[36597]: Deploying daemon osd.4 on smithi053 2024-03-21T09:04:19.041 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:18 smithi053 ceph-mon[36169]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:19.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:18 smithi071 ceph-mon[36597]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:18 smithi037 ceph-mon[31830]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:21.186 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:20 smithi053 ceph-mon[36169]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:20 smithi071 ceph-mon[36597]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:21.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:20 smithi037 ceph-mon[31830]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:22.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:22 smithi053 ceph-mon[36169]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:22.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:22 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:22.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:22 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:22.781 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:22 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:23.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:22 smithi071 ceph-mon[36597]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:23.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:22 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:23.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:22 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:23.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:22 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:23.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:22 smithi037 ceph-mon[31830]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:23.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:23.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:23.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:24.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:24 smithi053 ceph-mon[36169]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:25.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:24 smithi071 ceph-mon[36597]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:25.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:24 smithi037 ceph-mon[31830]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:25.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.716+0000 7fbf737fe700 1 -- 172.21.15.53:0/2976369551 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbf7c0625d0 con 0x7fbf680a3250 2024-03-21T09:04:25.711 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf680a3250 msgr2=0x7fbf680a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf680a3250 0x7fbf680a5710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005950 tx=0x7fbf780058e0 comp rx=0 tx=0).stop 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 msgr2=0x7fbf7c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c074580 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b530 tx=0x7fbf6c00b8f0 comp rx=0 tx=0).stop 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 shutdown_connections 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf7c103d60 0x7fbf7c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c1021c0 0x7fbf7c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:25.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf680a3250 0x7fbf680a5710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:25.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 --2- 172.21.15.53:0/2976369551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7c1033c0 0x7fbf7c074580 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:25.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 >> 172.21.15.53:0/2976369551 conn(0x7fbf7c0fd7a0 msgr2=0x7fbf7c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:04:25.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 shutdown_connections 2024-03-21T09:04:25.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:04:25.719+0000 7fbf846ba700 1 -- 172.21.15.53:0/2976369551 wait complete. 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T09:04:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:26.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:26.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:26.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T09:04:26.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T09:04:26.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:26.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T09:04:26.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:26.370 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T09:04:26.373 INFO:tasks.cephadm:Deploying osd.5 on smithi071 with /dev/vg_nvme/lv_3... 2024-03-21T09:04:26.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T09:04:26.617 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:04:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:26 smithi053 ceph-mon[36169]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:26 smithi053 ceph-mon[36169]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T09:04:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:26 smithi053 ceph-mon[36169]: from='osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:04:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:26 smithi053 ceph-mon[36169]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T09:04:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:26 smithi053 ceph-mon[36169]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:04:27.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:26 smithi071 ceph-mon[36597]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:27.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:26 smithi071 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T09:04:27.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:26 smithi071 ceph-mon[36597]: from='osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:04:27.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:26 smithi071 ceph-mon[36597]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T09:04:27.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:27.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:26 smithi071 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:04:27.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:26 smithi037 ceph-mon[31830]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:27.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:26 smithi037 ceph-mon[31830]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T09:04:27.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:26 smithi037 ceph-mon[31830]: from='osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:04:27.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:26 smithi037 ceph-mon[31830]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T09:04:27.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:27.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:26 smithi037 ceph-mon[31830]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:04:28.442 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:04:28.179+0000 7fc04228b700 -1 osd.4 0 waiting for initial osdmap 2024-03-21T09:04:28.442 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:04:28.185+0000 7fc039875700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:04:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T09:04:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T09:04:28.560 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-21T09:04:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-21T09:04:28.560 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111523 s, 940 MB/s 2024-03-21T09:04:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:28.890 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:28 smithi071 ceph-mon[36597]: from='osd.4 ' entity='osd.4' 2024-03-21T09:04:28.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:04:28.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:04:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:04:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T09:04:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:28.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:28 smithi053 ceph-mon[36169]: from='osd.4 ' entity='osd.4' 2024-03-21T09:04:29.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:04:29.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:04:29.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:04:29.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T09:04:29.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T09:04:29.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:28 smithi037 ceph-mon[31830]: from='osd.4 ' entity='osd.4' 2024-03-21T09:04:29.184 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-03-21T09:04:29.424 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:04:29.810 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.810 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] boot 2024-03-21T09:04:29.810 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi053 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi053 to 8542M 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:29.811 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] boot 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi053 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi053 to 8542M 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:29.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:29.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: osd.4 [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] boot 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi053 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi053 to 8542M 2024-03-21T09:04:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:29.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:29.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:29.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:31.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 -- 172.21.15.71:0/3118499543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102360 msgr2=0x7f30b81027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 --2- 172.21.15.71:0/3118499543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102360 0x7f30b81027c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009990 tx=0x7f30a8009ca0 comp rx=0 tx=0).stop 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 -- 172.21.15.71:0/3118499543 shutdown_connections 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 --2- 172.21.15.71:0/3118499543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30b8102d00 0x7f30b81095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 --2- 172.21.15.71:0/3118499543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30b8101160 0x7f30b8101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 --2- 172.21.15.71:0/3118499543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102360 0x7f30b81027c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 -- 172.21.15.71:0/3118499543 >> 172.21.15.71:0/3118499543 conn(0x7f30b80fc720 msgr2=0x7f30b80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 -- 172.21.15.71:0/3118499543 shutdown_connections 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.014+0000 7f30bca8f700 1 -- 172.21.15.71:0/3118499543 wait complete. 2024-03-21T09:04:31.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 Processor -- start 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 -- start start 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30b8101160 0x7f30b8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30b8102360 0x7f30b81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 0x7f30b819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b810cdd0 con 0x7f30b8101160 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b810cc50 con 0x7f30b8102360 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30bca8f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b810cf50 con 0x7f30b8102d00 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30b6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 0x7f30b819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30b6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 0x7f30b819ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60524/0 (socket says 172.21.15.71:60524) 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.015+0000 7f30b6d9d700 1 -- 172.21.15.71:0/1855197645 learned_addr learned my addr 172.21.15.71:0/1855197645 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b5d9b700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30b8102360 0x7f30b81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:31.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b6d9d700 1 -- 172.21.15.71:0/1855197645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30b8102360 msgr2=0x7f30b81979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b6d9d700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30b8102360 0x7f30b81979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b6d9d700 1 -- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30b8101160 msgr2=0x7f30b8197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b6d9d700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30b8101160 0x7f30b8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b6d9d700 1 -- 172.21.15.71:0/1855197645 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b819f0c0 con 0x7f30b8102d00 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30b6d9d700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 0x7f30b819ea20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b9a0 tx=0x7f30b000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0019070 con 0x7f30b8102d00 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30b0014440 con 0x7f30b8102d00 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b819f3b0 con 0x7f30b8102d00 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b00075d0 con 0x7f30b8102d00 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.016+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b819f920 con 0x7f30b8102d00 2024-03-21T09:04:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.017+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8068e50 con 0x7f30b8102d00 2024-03-21T09:04:31.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.018+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f30b001c070 con 0x7f30b8102d00 2024-03-21T09:04:31.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.019+0000 7f30a77fe700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f30a00a2fb0 0x7f30a00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:31.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.019+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f30b00cdca0 con 0x7f30b8102d00 2024-03-21T09:04:31.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.019+0000 7f30b659c700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f30a00a2fb0 0x7f30a00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:31.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.020+0000 7f30b659c700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f30a00a2fb0 0x7f30a00a5470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30b81021c0 tx=0x7f30ac006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:04:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.023+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f30b009bbd0 con 0x7f30b8102d00 2024-03-21T09:04:31.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:31.230+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f30b80625d0 con 0x7f30a00a2fb0 2024-03-21T09:04:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:31 smithi071 ceph-mon[36597]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:31 smithi071 ceph-mon[36597]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T09:04:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:31 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:31 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:31 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:31 smithi037 ceph-mon[31830]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:31 smithi037 ceph-mon[31830]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T09:04:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:31 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:31 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:31 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:31 smithi053 ceph-mon[36169]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:31 smithi053 ceph-mon[36169]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T09:04:31.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:31 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:31.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:31 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:31.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:31 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:32.547 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:32 smithi071 ceph-mon[36597]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:32.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:32 smithi053 ceph-mon[36169]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:32 smithi037 ceph-mon[31830]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:33.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:33 smithi053 ceph-mon[36169]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:33.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:33 smithi071 ceph-mon[36597]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:33.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:33 smithi037 ceph-mon[31830]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:34 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/3720796497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]: dispatch 2024-03-21T09:04:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:34 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]: dispatch 2024-03-21T09:04:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:34 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]': finished 2024-03-21T09:04:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:34 smithi071 ceph-mon[36597]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T09:04:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:34 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:34 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/3720796497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]: dispatch 2024-03-21T09:04:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:34 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]: dispatch 2024-03-21T09:04:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:34 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]': finished 2024-03-21T09:04:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:34 smithi053 ceph-mon[36169]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T09:04:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:34 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:34.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:34 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/3720796497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]: dispatch 2024-03-21T09:04:34.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:34 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]: dispatch 2024-03-21T09:04:34.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:34 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee6f4f1f-25a9-4e57-a211-178500d9cf88"}]': finished 2024-03-21T09:04:34.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:34 smithi037 ceph-mon[31830]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T09:04:34.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:34 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:35.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:35 smithi071 ceph-mon[36597]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:35.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:35 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/3597269126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:04:35.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:35 smithi037 ceph-mon[31830]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:35.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:35 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/3597269126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:04:35.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:35 smithi053 ceph-mon[36169]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:35.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:35 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/3597269126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:04:37.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:37 smithi053 ceph-mon[36169]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:37.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:37 smithi071 ceph-mon[36597]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:37 smithi037 ceph-mon[31830]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:39.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:39 smithi071 ceph-mon[36597]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:39 smithi037 ceph-mon[31830]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:39.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:39 smithi053 ceph-mon[36169]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:41.596 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:41 smithi071 ceph-mon[36597]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:41.597 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T09:04:41.597 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:41 smithi037 ceph-mon[31830]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T09:04:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:41 smithi053 ceph-mon[36169]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T09:04:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:42.737 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:42 smithi071 ceph-mon[36597]: Deploying daemon osd.5 on smithi071 2024-03-21T09:04:42.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:42 smithi037 ceph-mon[31830]: Deploying daemon osd.5 on smithi071 2024-03-21T09:04:42.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:42 smithi053 ceph-mon[36169]: Deploying daemon osd.5 on smithi071 2024-03-21T09:04:43.742 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:43 smithi071 ceph-mon[36597]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:43.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:43 smithi037 ceph-mon[31830]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:43.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:43 smithi053 ceph-mon[36169]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:45.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:45 smithi071 ceph-mon[36597]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:45.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:45 smithi037 ceph-mon[31830]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:45.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:45 smithi053 ceph-mon[36169]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:47.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:47 smithi053 ceph-mon[36169]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:47.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:47 smithi071 ceph-mon[36597]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:47.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:47.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:47.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:47 smithi037 ceph-mon[31830]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:47.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:47.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:49 smithi053 ceph-mon[36169]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:49.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:49 smithi071 ceph-mon[36597]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:49.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:49.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:49.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:49 smithi037 ceph-mon[31830]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:49.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:49.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:49.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:49.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:50.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.014+0000 7f30a77fe700 1 -- 172.21.15.71:0/1855197645 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f30b80625d0 con 0x7f30a00a2fb0 2024-03-21T09:04:50.009 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 5 on host 'smithi071' 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f30a00a2fb0 msgr2=0x7f30a00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f30a00a2fb0 0x7f30a00a5470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30b81021c0 tx=0x7f30ac006d20 comp rx=0 tx=0).stop 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 msgr2=0x7f30b819ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 0x7f30b819ea20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30b000b9a0 tx=0x7f30b000bd60 comp rx=0 tx=0).stop 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 shutdown_connections 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b8102d00 0x7f30b819ea20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30b8101160 0x7f30b8197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f30a00a2fb0 0x7f30a00a5470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:50.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 --2- 172.21.15.71:0/1855197645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30b8102360 0x7f30b81979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:50.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.016+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 >> 172.21.15.71:0/1855197645 conn(0x7f30b80fc720 msgr2=0x7f30b8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:04:50.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.017+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 shutdown_connections 2024-03-21T09:04:50.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:04:50.017+0000 7f30bca8f700 1 -- 172.21.15.71:0/1855197645 wait complete. 2024-03-21T09:04:50.570 DEBUG:teuthology.orchestra.run.smithi071:osd.5> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T09:04:50.572 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-03-21T09:04:50.572 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T09:04:51.055 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:04:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:50 smithi037 ceph-mon[31830]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:50 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:50 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:50 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:50 smithi037 ceph-mon[31830]: from='osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T09:04:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:50 smithi037 ceph-mon[31830]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T09:04:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:50 smithi071 ceph-mon[36597]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:50 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:50 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:50 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:50 smithi071 ceph-mon[36597]: from='osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T09:04:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:50 smithi071 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T09:04:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:50 smithi053 ceph-mon[36169]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:50 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:50 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:50 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:50 smithi053 ceph-mon[36169]: from='osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T09:04:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:50 smithi053 ceph-mon[36169]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T09:04:52.169 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T09:04:52.169 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-mon[36597]: from='osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:04:52.169 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-mon[36597]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T09:04:52.169 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:52.169 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:04:52.169 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:04:51.972+0000 7fd1da3db700 -1 osd.5 0 waiting for initial osdmap 2024-03-21T09:04:52.169 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:04:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:04:51.979+0000 7fd1d61ce700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:04:52.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:51 smithi037 ceph-mon[31830]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T09:04:52.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:51 smithi037 ceph-mon[31830]: from='osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:04:52.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:51 smithi037 ceph-mon[31830]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T09:04:52.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:51 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:52.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:51 smithi037 ceph-mon[31830]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:04:52.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:51 smithi053 ceph-mon[36169]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T09:04:52.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:51 smithi053 ceph-mon[36169]: from='osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:04:52.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:51 smithi053 ceph-mon[36169]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T09:04:52.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:51 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:52.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:51 smithi053 ceph-mon[36169]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:04:53.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:53.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:04:53.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T09:04:53.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:53.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] boot 2024-03-21T09:04:53.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: osdmap e38: 6 total, 6 up, 6 in 2024-03-21T09:04:53.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] boot 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: osdmap e38: 6 total, 6 up, 6 in 2024-03-21T09:04:53.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:53.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-21T09:04:53.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:04:53.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T09:04:53.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:53.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: osd.5 [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] boot 2024-03-21T09:04:53.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: osdmap e38: 6 total, 6 up, 6 in 2024-03-21T09:04:53.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:04:54.035 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T09:04:54.035 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T09:04:54.035 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-21T09:04:54.035 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-21T09:04:54.035 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0344413 s, 304 MB/s 2024-03-21T09:04:54.036 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi071 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:54.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi071 to 8540M 2024-03-21T09:04:54.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:54.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:54.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:53 smithi071 ceph-mon[36597]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi071 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi071 to 8540M 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:54.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:53 smithi037 ceph-mon[31830]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T09:04:54.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:04:54.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi071 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi071 to 8540M 2024-03-21T09:04:54.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:54.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:54.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:04:54.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:53 smithi053 ceph-mon[36169]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T09:04:54.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-03-21T09:04:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:04:55.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:55 smithi037 ceph-mon[31830]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:04:55.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:55 smithi037 ceph-mon[31830]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T09:04:55.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:55 smithi037 ceph-mon[31830]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T09:04:55.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:55 smithi053 ceph-mon[36169]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:04:55.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:55 smithi053 ceph-mon[36169]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T09:04:55.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:55 smithi053 ceph-mon[36169]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T09:04:56.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:55 smithi071 ceph-mon[36597]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:04:56.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:55 smithi071 ceph-mon[36597]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T09:04:56.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:55 smithi071 ceph-mon[36597]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T09:04:57.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:57 smithi037 ceph-mon[31830]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:04:57.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:57 smithi037 ceph-mon[31830]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T09:04:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.692+0000 7f1658400700 1 -- 172.21.15.37:0/658588944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105610 msgr2=0x7f1650105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.692+0000 7f1658400700 1 --2- 172.21.15.37:0/658588944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105610 0x7f1650105a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f163c0099c0 tx=0x7f163c009cd0 comp rx=0 tx=0).stop 2024-03-21T09:04:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.692+0000 7f1658400700 1 -- 172.21.15.37:0/658588944 shutdown_connections 2024-03-21T09:04:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.693+0000 7f1658400700 1 --2- 172.21.15.37:0/658588944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105fb0 0x7f165010c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.693+0000 7f1658400700 1 --2- 172.21.15.37:0/658588944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 0x7f1650104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.693+0000 7f1658400700 1 --2- 172.21.15.37:0/658588944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105610 0x7f1650105a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:57.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.693+0000 7f1658400700 1 -- 172.21.15.37:0/658588944 >> 172.21.15.37:0/658588944 conn(0x7f16500ff9b0 msgr2=0x7f1650101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:04:57.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.693+0000 7f1658400700 1 -- 172.21.15.37:0/658588944 shutdown_connections 2024-03-21T09:04:57.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.693+0000 7f1658400700 1 -- 172.21.15.37:0/658588944 wait complete. 2024-03-21T09:04:57.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.694+0000 7f1658400700 1 Processor -- start 2024-03-21T09:04:57.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.694+0000 7f1658400700 1 -- start start 2024-03-21T09:04:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1658400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 0x7f165019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1658400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 0x7f165019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1658400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105fb0 0x7f16501a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1658400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165010ffe0 con 0x7f1650105fb0 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1658400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165010fe60 con 0x7f1650104410 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1658400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1650110160 con 0x7f1650105610 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1655000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 0x7f165019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1656400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105fb0 0x7f16501a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1655000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 0x7f165019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54972/0 (socket says 172.21.15.37:54972) 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1655000700 1 -- 172.21.15.37:0/471465757 learned_addr learned my addr 172.21.15.37:0/471465757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1655a00700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 0x7f165019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.695+0000 7f1655000700 1 -- 172.21.15.37:0/471465757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 msgr2=0x7f165019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1655000700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 0x7f165019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1655000700 1 -- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105fb0 msgr2=0x7f16501a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:04:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1655000700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105fb0 0x7f16501a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:04:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1655000700 1 -- 172.21.15.37:0/471465757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16501a2270 con 0x7f1650105610 2024-03-21T09:04:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1656400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105fb0 0x7f16501a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:04:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1655a00700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 0x7f165019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:04:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1655000700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 0x7f165019ab50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b580 tx=0x7f163c00bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:04:57.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:57 smithi053 ceph-mon[36169]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:04:57.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:57 smithi053 ceph-mon[36169]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T09:04:57.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c009e60 con 0x7f1650105610 2024-03-21T09:04:57.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16501a2500 con 0x7f1650105610 2024-03-21T09:04:57.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f163c00f460 con 0x7f1650105610 2024-03-21T09:04:57.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.696+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c017630 con 0x7f1650105610 2024-03-21T09:04:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.697+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16501a29b0 con 0x7f1650105610 2024-03-21T09:04:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.698+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1650068e50 con 0x7f1650105610 2024-03-21T09:04:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.700+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f163c00f610 con 0x7f1650105610 2024-03-21T09:04:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.701+0000 7f164e000700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16380a3030 0x7f16380a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:04:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.702+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f163c0cf4b0 con 0x7f1650105610 2024-03-21T09:04:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.702+0000 7f1655a00700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16380a3030 0x7f16380a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:04:57.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.703+0000 7f1655a00700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16380a3030 0x7f16380a54f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1650105470 tx=0x7f1644009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:04:57.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.704+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f163c013070 con 0x7f1650105610 2024-03-21T09:04:57.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:57 smithi071 ceph-mon[36597]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:04:57.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:57 smithi071 ceph-mon[36597]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T09:04:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:04:57.944+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1650062690 con 0x7f16380a3030 2024-03-21T09:04:58.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:58.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:58.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:58 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:58.746 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:58.746 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:58.747 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:58 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:04:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:04:58.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:58 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:59 smithi037 ceph-mon[31830]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T09:04:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:04:59 smithi037 ceph-mon[31830]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:59 smithi053 ceph-mon[36169]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T09:04:59.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:04:59 smithi053 ceph-mon[36169]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:04:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:59 smithi071 ceph-mon[36597]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T09:04:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:04:59 smithi071 ceph-mon[36597]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:01.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:01 smithi053 ceph-mon[36169]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-21T09:05:01.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:01 smithi071 ceph-mon[36597]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-21T09:05:01.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:01 smithi037 ceph-mon[31830]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-21T09:05:02.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2230807682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de23659-fb28-4c49-988a-84c26500a853"}]: dispatch 2024-03-21T09:05:02.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2230807682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de23659-fb28-4c49-988a-84c26500a853"}]': finished 2024-03-21T09:05:02.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:02 smithi037 ceph-mon[31830]: osdmap e43: 7 total, 6 up, 7 in 2024-03-21T09:05:02.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:02.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:02 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2230807682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de23659-fb28-4c49-988a-84c26500a853"}]: dispatch 2024-03-21T09:05:02.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:02 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2230807682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de23659-fb28-4c49-988a-84c26500a853"}]': finished 2024-03-21T09:05:02.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:02 smithi071 ceph-mon[36597]: osdmap e43: 7 total, 6 up, 7 in 2024-03-21T09:05:02.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:02.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:02 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2230807682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de23659-fb28-4c49-988a-84c26500a853"}]: dispatch 2024-03-21T09:05:02.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:02 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2230807682' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de23659-fb28-4c49-988a-84c26500a853"}]': finished 2024-03-21T09:05:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:02 smithi053 ceph-mon[36169]: osdmap e43: 7 total, 6 up, 7 in 2024-03-21T09:05:02.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:03 smithi071 ceph-mon[36597]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-21T09:05:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:03 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/344534969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:05:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:03 smithi037 ceph-mon[31830]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-21T09:05:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:03 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/344534969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:05:03.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:03 smithi053 ceph-mon[36169]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-21T09:05:03.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:03 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/344534969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:05:05.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:05 smithi071 ceph-mon[36597]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T09:05:05.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:05 smithi037 ceph-mon[31830]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T09:05:05.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:05 smithi053 ceph-mon[36169]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T09:05:07.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:07 smithi037 ceph-mon[31830]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:07.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:07 smithi053 ceph-mon[36169]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:07.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:07 smithi071 ceph-mon[36597]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:09.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:09 smithi037 ceph-mon[31830]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:09.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:09 smithi071 ceph-mon[36597]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:09.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:09 smithi053 ceph-mon[36169]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:11.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:11 smithi071 ceph-mon[36597]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:11.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:11 smithi037 ceph-mon[31830]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:11.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:11 smithi053 ceph-mon[36169]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:12.783 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:12 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T09:05:12.783 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:12 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:12.783 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:12 smithi037 ceph-mon[31830]: Deploying daemon osd.6 on smithi037 2024-03-21T09:05:12.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:12 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T09:05:12.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:12 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:12.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:12 smithi071 ceph-mon[36597]: Deploying daemon osd.6 on smithi037 2024-03-21T09:05:12.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:12 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T09:05:12.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:12 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:12.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:12 smithi053 ceph-mon[36169]: Deploying daemon osd.6 on smithi037 2024-03-21T09:05:13.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:13 smithi037 ceph-mon[31830]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:13.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:13 smithi071 ceph-mon[36597]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:13.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:13 smithi053 ceph-mon[36169]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:15 smithi037 ceph-mon[31830]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:15.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:15 smithi071 ceph-mon[36597]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:15.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:15 smithi053 ceph-mon[36169]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:17.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:17 smithi037 ceph-mon[31830]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:17.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:17 smithi071 ceph-mon[36597]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:17.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:17 smithi053 ceph-mon[36169]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:19.589 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:19 smithi037 ceph-mon[31830]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:19.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:19 smithi071 ceph-mon[36597]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:19.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:19 smithi053 ceph-mon[36169]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:21 smithi037 ceph-mon[31830]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:21 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:21 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:21 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:21.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:21 smithi071 ceph-mon[36597]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:21.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:21 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:21.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:21 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:21.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:21 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:21.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:21 smithi053 ceph-mon[36169]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:21.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:21 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:21.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:21 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:21.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:21 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:23.726 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:23 smithi037 ceph-mon[31830]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:23 smithi071 ceph-mon[36597]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:23.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:23 smithi053 ceph-mon[36169]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:25.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:25 smithi037 ceph-mon[31830]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:25.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:25.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:25.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:25 smithi037 ceph-mon[31830]: from='osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T09:05:25.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:25 smithi037 ceph-mon[31830]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T09:05:25.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:25 smithi053 ceph-mon[36169]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:25.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:25 smithi053 ceph-mon[36169]: from='osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T09:05:25.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:25 smithi053 ceph-mon[36169]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T09:05:25.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:25 smithi071 ceph-mon[36597]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:25.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:25.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:25.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:25 smithi071 ceph-mon[36597]: from='osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T09:05:25.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:25 smithi071 ceph-mon[36597]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T09:05:25.661 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-03-21T09:05:25.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.666+0000 7f164e000700 1 -- 172.21.15.37:0/471465757 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1650062690 con 0x7f16380a3030 2024-03-21T09:05:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.670+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16380a3030 msgr2=0x7f16380a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.670+0000 7f1658400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16380a3030 0x7f16380a54f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1650105470 tx=0x7f1644009040 comp rx=0 tx=0).stop 2024-03-21T09:05:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.670+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 msgr2=0x7f165019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.670+0000 7f1658400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 0x7f165019ab50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b580 tx=0x7f163c00bee0 comp rx=0 tx=0).stop 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 shutdown_connections 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650105fb0 0x7f16501a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16380a3030 0x7f16380a54f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1650104410 0x7f165019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 --2- 172.21.15.37:0/471465757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1650105610 0x7f165019ab50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 >> 172.21.15.37:0/471465757 conn(0x7f16500ff9b0 msgr2=0x7f16501088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 shutdown_connections 2024-03-21T09:05:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:05:25.671+0000 7f1658400700 1 -- 172.21.15.37:0/471465757 wait complete. 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:26.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:26.512 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T09:05:26.517 INFO:tasks.cephadm:Deploying osd.7 on smithi053 with /dev/vg_nvme/lv_2... 2024-03-21T09:05:26.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:26.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:05:27.123 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:05:26.787+0000 7f26e0800700 -1 osd.6 0 waiting for initial osdmap 2024-03-21T09:05:27.123 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:05:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:05:26.823+0000 7f26d5c00700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:05:27.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:27 smithi037 ceph-mon[31830]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:27.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:27 smithi037 ceph-mon[31830]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:05:27.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:27 smithi037 ceph-mon[31830]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T09:05:27.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:27 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:27.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:27 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:27.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:27 smithi037 ceph-mon[31830]: from='osd.6 ' entity='osd.6' 2024-03-21T09:05:27.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:27 smithi053 ceph-mon[36169]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:27.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:27 smithi053 ceph-mon[36169]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:05:27.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:27 smithi053 ceph-mon[36169]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T09:05:27.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:27 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:27.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:27 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:27.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:27 smithi053 ceph-mon[36169]: from='osd.6 ' entity='osd.6' 2024-03-21T09:05:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:27 smithi071 ceph-mon[36597]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:27 smithi071 ceph-mon[36597]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:05:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:27 smithi071 ceph-mon[36597]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T09:05:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:27 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:27 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:27 smithi071 ceph-mon[36597]: from='osd.6 ' entity='osd.6' 2024-03-21T09:05:28.235 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:28 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:05:28.235 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:28 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:05:28.235 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:28.235 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:28 smithi053 ceph-mon[36169]: osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] boot 2024-03-21T09:05:28.235 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:28 smithi053 ceph-mon[36169]: osdmap e46: 7 total, 7 up, 7 in 2024-03-21T09:05:28.235 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:28.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:28 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:05:28.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:28 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:05:28.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:28.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:28 smithi037 ceph-mon[31830]: osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] boot 2024-03-21T09:05:28.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:28 smithi037 ceph-mon[31830]: osdmap e46: 7 total, 7 up, 7 in 2024-03-21T09:05:28.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:28 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:05:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:28 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:05:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:28 smithi071 ceph-mon[36597]: osd.6 [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] boot 2024-03-21T09:05:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:28 smithi071 ceph-mon[36597]: osdmap e46: 7 total, 7 up, 7 in 2024-03-21T09:05:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00816082 s, 1.3 GB/s 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 2024-03-21T09:05:28.789 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-03-21T09:05:29.366 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:29 smithi053 ceph-mon[36169]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:29.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-03-21T09:05:29.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:29 smithi037 ceph-mon[31830]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:29 smithi071 ceph-mon[36597]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T09:05:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:05:30.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:30 smithi053 ceph-mon[36169]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T09:05:30.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:30 smithi071 ceph-mon[36597]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T09:05:30.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:30 smithi037 ceph-mon[31830]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 -- 172.21.15.53:0/491614758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 msgr2=0x7f7bd8103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 --2- 172.21.15.53:0/491614758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8103820 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0099c0 tx=0x7f7bcc009cd0 comp rx=0 tx=0).stop 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 -- 172.21.15.53:0/491614758 shutdown_connections 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 --2- 172.21.15.53:0/491614758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bd8103d60 0x7f7bd810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 --2- 172.21.15.53:0/491614758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd81021c0 0x7f7bd81025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 --2- 172.21.15.53:0/491614758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8103820 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 -- 172.21.15.53:0/491614758 >> 172.21.15.53:0/491614758 conn(0x7f7bd80fd7a0 msgr2=0x7f7bd80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 -- 172.21.15.53:0/491614758 shutdown_connections 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.277+0000 7f7bdf740700 1 -- 172.21.15.53:0/491614758 wait complete. 2024-03-21T09:05:31.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 Processor -- start 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 -- start start 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd81021c0 0x7f7bd8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bd8103d60 0x7f7bd8074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd810de50 con 0x7f7bd81021c0 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd810dcd0 con 0x7f7bd81033c0 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdf740700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd810dfd0 con 0x7f7bd8103d60 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38866/0 (socket says 172.21.15.53:38866) 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 -- 172.21.15.53:0/77199426 learned_addr learned my addr 172.21.15.53:0/77199426 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 -- 172.21.15.53:0/77199426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bd8103d60 msgr2=0x7f7bd8074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bd8103d60 0x7f7bd8074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 -- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd81021c0 msgr2=0x7f7bd8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:31.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd81021c0 0x7f7bd8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 -- 172.21.15.53:0/77199426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd8075370 con 0x7f7bd81033c0 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bdccdb700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8074580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c010 tx=0x7f7bcc0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.278+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc027070 con 0x7f7bd81033c0 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.279+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc0041b0 con 0x7f7bd81033c0 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.279+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd81a2530 con 0x7f7bd81033c0 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.279+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00f460 con 0x7f7bd81033c0 2024-03-21T09:05:31.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.279+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd81a29e0 con 0x7f7bd81033c0 2024-03-21T09:05:31.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.279+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd8068e50 con 0x7f7bd81033c0 2024-03-21T09:05:31.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.281+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7bcc02a070 con 0x7f7bd81033c0 2024-03-21T09:05:31.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.281+0000 7f7bd67fc700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7bc00a2fb0 0x7f7bc00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:31.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.282+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f7bcc0cfb80 con 0x7f7bd81033c0 2024-03-21T09:05:31.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.282+0000 7f7bdd4dc700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7bc00a2fb0 0x7f7bc00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:05:31.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.283+0000 7f7bdd4dc700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7bc00a2fb0 0x7f7bc00a5470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8103220 tx=0x7f7bd0009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:05:31.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.285+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7bcc031070 con 0x7f7bd81033c0 2024-03-21T09:05:31.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:31 smithi037 ceph-mon[31830]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:31.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:31 smithi053 ceph-mon[36169]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:31.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:31.500+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd80625d0 con 0x7f7bc00a2fb0 2024-03-21T09:05:31.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:31 smithi071 ceph-mon[36597]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:32.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:32.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi037 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:32.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:05:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:05:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi037 2024-03-21T09:05:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi037 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:33.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:33 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi037 to 5698M 2024-03-21T09:05:33.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:33 smithi053 ceph-mon[36169]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:33.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:33 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi037 to 5698M 2024-03-21T09:05:33.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:33 smithi071 ceph-mon[36597]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:33 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi037 to 5698M 2024-03-21T09:05:33.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:33 smithi037 ceph-mon[31830]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/3171907462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]: dispatch 2024-03-21T09:05:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]: dispatch 2024-03-21T09:05:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]': finished 2024-03-21T09:05:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: osdmap e48: 8 total, 7 up, 8 in 2024-03-21T09:05:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:35.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:35 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/2907817431' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:05:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/3171907462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]: dispatch 2024-03-21T09:05:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]: dispatch 2024-03-21T09:05:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]': finished 2024-03-21T09:05:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: osdmap e48: 8 total, 7 up, 8 in 2024-03-21T09:05:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:35.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:35 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/2907817431' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/3171907462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]: dispatch 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]: dispatch 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "344d4a58-75bc-4741-b385-e590feb5fcd8"}]': finished 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: osdmap e48: 8 total, 7 up, 8 in 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:35.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:35 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/2907817431' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:05:37.494 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:37 smithi053 ceph-mon[36169]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:37.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:37 smithi071 ceph-mon[36597]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:37.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:37 smithi037 ceph-mon[31830]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:39.477 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:39 smithi053 ceph-mon[36169]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:39.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:39 smithi071 ceph-mon[36597]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:39.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:39 smithi037 ceph-mon[31830]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:41.417 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:41 smithi053 ceph-mon[36169]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:41.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:41 smithi071 ceph-mon[36597]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:41.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:41 smithi037 ceph-mon[31830]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:42.476 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:42 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T09:05:42.476 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:42 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:42.476 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:42 smithi053 ceph-mon[36169]: Deploying daemon osd.7 on smithi053 2024-03-21T09:05:42.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:42 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T09:05:42.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:42 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:42.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:42 smithi071 ceph-mon[36597]: Deploying daemon osd.7 on smithi053 2024-03-21T09:05:42.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:42 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T09:05:42.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:42 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:42.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:42 smithi037 ceph-mon[31830]: Deploying daemon osd.7 on smithi053 2024-03-21T09:05:43.294 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:43 smithi053 ceph-mon[36169]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:43.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:43 smithi071 ceph-mon[36597]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:43.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:43 smithi037 ceph-mon[31830]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:45.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:45 smithi053 ceph-mon[36169]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:45.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:45 smithi071 ceph-mon[36597]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:45.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:45 smithi037 ceph-mon[31830]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:47 smithi053 ceph-mon[36169]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:47.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:47 smithi071 ceph-mon[36597]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:47 smithi037 ceph-mon[31830]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:48 smithi037 ceph-mon[31830]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:48 smithi053 ceph-mon[36169]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:48 smithi071 ceph-mon[36597]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:51.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:50 smithi037 ceph-mon[31830]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:51.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:50 smithi053 ceph-mon[36169]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:51.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.346+0000 7f7bd67fc700 1 -- 172.21.15.53:0/77199426 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7bd80625d0 con 0x7f7bc00a2fb0 2024-03-21T09:05:51.341 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 7 on host 'smithi053' 2024-03-21T09:05:51.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.347+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7bc00a2fb0 msgr2=0x7f7bc00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.347+0000 7f7bdf740700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7bc00a2fb0 0x7f7bc00a5470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8103220 tx=0x7f7bd0009250 comp rx=0 tx=0).stop 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.347+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 msgr2=0x7f7bd8074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.347+0000 7f7bdf740700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8074580 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c010 tx=0x7f7bcc0058e0 comp rx=0 tx=0).stop 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 shutdown_connections 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bd8103d60 0x7f7bd8074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd81021c0 0x7f7bd8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7bc00a2fb0 0x7f7bc00a5470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 --2- 172.21.15.53:0/77199426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bd81033c0 0x7f7bd8074580 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 >> 172.21.15.53:0/77199426 conn(0x7f7bd80fd7a0 msgr2=0x7f7bd81066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 shutdown_connections 2024-03-21T09:05:51.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:05:51.348+0000 7f7bdf740700 1 -- 172.21.15.53:0/77199426 wait complete. 2024-03-21T09:05:51.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:50 smithi071 ceph-mon[36597]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:51.951 DEBUG:teuthology.orchestra.run.smithi053:osd.7> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T09:05:51.954 INFO:tasks.cephadm:Deploying osd.8 on smithi071 with /dev/vg_nvme/lv_2... 2024-03-21T09:05:51.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T09:05:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:05:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:52 smithi071 ceph-mon[36597]: from='osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T09:05:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:52 smithi071 ceph-mon[36597]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T09:05:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:52 smithi037 ceph-mon[31830]: from='osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T09:05:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:52 smithi037 ceph-mon[31830]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T09:05:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:52 smithi053 ceph-mon[36169]: from='osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T09:05:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:52 smithi053 ceph-mon[36169]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T09:05:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:53.546 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:53 smithi071 ceph-mon[36597]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:53.546 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:53 smithi071 ceph-mon[36597]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T09:05:53.546 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:53 smithi071 ceph-mon[36597]: from='osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:05:53.547 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:53 smithi071 ceph-mon[36597]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T09:05:53.547 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:53.547 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:53 smithi071 ceph-mon[36597]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:05:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:53 smithi037 ceph-mon[31830]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:53 smithi037 ceph-mon[31830]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T09:05:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:53 smithi037 ceph-mon[31830]: from='osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:05:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:53 smithi037 ceph-mon[31830]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T09:05:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:53 smithi037 ceph-mon[31830]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:05:53.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-mon[36169]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:53.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-mon[36169]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T09:05:53.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-mon[36169]: from='osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:05:53.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-mon[36169]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T09:05:53.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:53.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-mon[36169]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:05:54.102 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T09:05:54.102 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T09:05:54.103 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-21T09:05:54.103 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-21T09:05:54.103 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112764 s, 930 MB/s 2024-03-21T09:05:54.103 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-21T09:05:54.120 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:05:53.805+0000 7f94291ca700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T09:05:54.120 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:05:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:05:53.817+0000 7f9420fb5700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:05:54.399 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:54 smithi071 ceph-mon[36597]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:05:54.400 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:54 smithi071 ceph-mon[36597]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T09:05:54.400 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:54.400 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:54 smithi071 ceph-mon[36597]: from='osd.7 ' entity='osd.7' 2024-03-21T09:05:54.400 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:54 smithi053 ceph-mon[36169]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:05:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:54 smithi053 ceph-mon[36169]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T09:05:54.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:54 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:54.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:54 smithi053 ceph-mon[36169]: from='osd.7 ' entity='osd.7' 2024-03-21T09:05:54.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:54 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:54 smithi037 ceph-mon[31830]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:05:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:54 smithi037 ceph-mon[31830]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T09:05:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:54 smithi037 ceph-mon[31830]: from='osd.7 ' entity='osd.7' 2024-03-21T09:05:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:54.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-03-21T09:05:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:05:55.455 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:55 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:05:55.456 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:55 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:05:55.456 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:55 smithi071 ceph-mon[36597]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:55.456 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:55 smithi071 ceph-mon[36597]: osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] boot 2024-03-21T09:05:55.456 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:55 smithi071 ceph-mon[36597]: osdmap e51: 8 total, 8 up, 8 in 2024-03-21T09:05:55.456 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:55 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:55.662 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:55 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:05:55.662 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:55 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:05:55.662 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:55 smithi053 ceph-mon[36169]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:55.662 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:55 smithi053 ceph-mon[36169]: osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] boot 2024-03-21T09:05:55.662 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:55 smithi053 ceph-mon[36169]: osdmap e51: 8 total, 8 up, 8 in 2024-03-21T09:05:55.662 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:55 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:55 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:05:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:55 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:05:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:55 smithi037 ceph-mon[31830]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T09:05:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:55 smithi037 ceph-mon[31830]: osd.7 [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] boot 2024-03-21T09:05:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:55 smithi037 ceph-mon[31830]: osdmap e51: 8 total, 8 up, 8 in 2024-03-21T09:05:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:55 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:05:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.628+0000 7fd400de4700 1 -- 172.21.15.71:0/480540500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 msgr2=0x7fd3fc102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 --2- 172.21.15.71:0/480540500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc102720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009990 tx=0x7fd3f0009ca0 comp rx=0 tx=0).stop 2024-03-21T09:05:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 -- 172.21.15.71:0/480540500 shutdown_connections 2024-03-21T09:05:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 --2- 172.21.15.71:0/480540500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc102c60 0x7fd3fc109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 --2- 172.21.15.71:0/480540500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3fc1010c0 0x7fd3fc1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 --2- 172.21.15.71:0/480540500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc102720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 -- 172.21.15.71:0/480540500 >> 172.21.15.71:0/480540500 conn(0x7fd3fc0fc680 msgr2=0x7fd3fc0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 -- 172.21.15.71:0/480540500 shutdown_connections 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 -- 172.21.15.71:0/480540500 wait complete. 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 Processor -- start 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.629+0000 7fd400de4700 1 -- start start 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3fc1010c0 0x7fd3fc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc102c60 0x7fd3fc19fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc10de20 con 0x7fd3fc1010c0 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc10dca0 con 0x7fd3fc102c60 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc10dfa0 con 0x7fd3fc1022c0 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:05:56.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38624/0 (socket says 172.21.15.71:38624) 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 -- 172.21.15.71:0/3810085136 learned_addr learned my addr 172.21.15.71:0/3810085136 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3fad9d700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc102c60 0x7fd3fc19fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 -- 172.21.15.71:0/3810085136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc102c60 msgr2=0x7fd3fc19fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc102c60 0x7fd3fc19fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 -- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3fc1010c0 msgr2=0x7fd3fc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3fc1010c0 0x7fd3fc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 -- 172.21.15.71:0/3810085136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc1a00a0 con 0x7fd3fc1022c0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3f9d9b700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc198980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0004790 tx=0x7fd3f0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0027070 con 0x7fd3fc1022c0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc1a0330 con 0x7fd3fc1022c0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0004150 con 0x7fd3fc1022c0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.630+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000f460 con 0x7fd3fc1022c0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.631+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3fc1a0810 con 0x7fd3fc1022c0 2024-03-21T09:05:56.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.631+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3fc068e50 con 0x7fd3fc1022c0 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.629 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.630 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:56.630 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.630 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.630 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.630 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:56 smithi071 ceph-mon[36597]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T09:05:56.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.633+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd3f002a070 con 0x7fd3fc1022c0 2024-03-21T09:05:56.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.634+0000 7fd3eb7fe700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3e40a2fb0 0x7fd3e40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:05:56.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.634+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fd3f00cfed0 con 0x7fd3fc1022c0 2024-03-21T09:05:56.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.635+0000 7fd3fa59c700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3e40a2fb0 0x7fd3e40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:05:56.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.636+0000 7fd3fa59c700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3e40a2fb0 0x7fd3e40a5470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc102120 tx=0x7fd3ec006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:05:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.637+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd3f0031070 con 0x7fd3fc1022c0 2024-03-21T09:05:56.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T09:05:56.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:56 smithi037 ceph-mon[31830]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:05:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:56 smithi053 ceph-mon[36169]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T09:05:56.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:05:56.843+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3fc0625d0 con 0x7fd3e40a2fb0 2024-03-21T09:05:57.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:57 smithi037 ceph-mon[31830]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:05:57.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:57 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi053 2024-03-21T09:05:57.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:57 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi053 to 5694M 2024-03-21T09:05:57.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:05:57.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:05:57.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:57 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:57.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:57 smithi053 ceph-mon[36169]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:05:57.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:57 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi053 2024-03-21T09:05:57.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:57 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi053 to 5694M 2024-03-21T09:05:57.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:05:57.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:05:57.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:57 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:57.708 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:57 smithi071 ceph-mon[36597]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:05:57.708 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:57 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi053 2024-03-21T09:05:57.708 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:57 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi053 to 5694M 2024-03-21T09:05:57.708 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:05:57.708 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:05:57.708 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:57 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:58.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:58 smithi071 ceph-mon[36597]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:58.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:58 smithi071 ceph-mon[36597]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T09:05:58.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:58 smithi037 ceph-mon[31830]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:58.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:58 smithi037 ceph-mon[31830]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T09:05:58.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:58 smithi053 ceph-mon[36169]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:05:58.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:58 smithi053 ceph-mon[36169]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T09:05:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:59 smithi037 ceph-mon[31830]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:05:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:05:59 smithi037 ceph-mon[31830]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T09:05:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:59 smithi053 ceph-mon[36169]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:05:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:05:59 smithi053 ceph-mon[36169]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T09:05:59.831 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:59 smithi071 ceph-mon[36597]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:05:59.831 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:05:59 smithi071 ceph-mon[36597]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T09:06:00.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/1155245370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]: dispatch 2024-03-21T09:06:00.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:00 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]: dispatch 2024-03-21T09:06:00.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:00 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]': finished 2024-03-21T09:06:00.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:00 smithi037 ceph-mon[31830]: osdmap e56: 9 total, 8 up, 9 in 2024-03-21T09:06:00.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:00.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/885094984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:06:00.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/1155245370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]: dispatch 2024-03-21T09:06:00.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:00 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]: dispatch 2024-03-21T09:06:00.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:00 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]': finished 2024-03-21T09:06:00.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:00 smithi053 ceph-mon[36169]: osdmap e56: 9 total, 8 up, 9 in 2024-03-21T09:06:00.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:00.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/885094984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:06:00.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/1155245370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]: dispatch 2024-03-21T09:06:00.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:00 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]: dispatch 2024-03-21T09:06:00.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:00 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d7e521-1571-45c7-98d2-3a7b3e876488"}]': finished 2024-03-21T09:06:00.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:00 smithi071 ceph-mon[36597]: osdmap e56: 9 total, 8 up, 9 in 2024-03-21T09:06:00.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:00.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/885094984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:06:01.754 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:01 smithi071 ceph-mon[36597]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:01.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:01 smithi037 ceph-mon[31830]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:01 smithi053 ceph-mon[36169]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:03 smithi071 ceph-mon[36597]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:03 smithi037 ceph-mon[31830]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:03.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:03 smithi053 ceph-mon[36169]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:05.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:05 smithi071 ceph-mon[36597]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:05.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:05 smithi037 ceph-mon[31830]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:05.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:05 smithi053 ceph-mon[36169]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:07.558 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:07 smithi071 ceph-mon[36597]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:07.558 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T09:06:07.558 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:07 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:07.558 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:07 smithi071 ceph-mon[36597]: Deploying daemon osd.8 on smithi071 2024-03-21T09:06:07.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:07 smithi053 ceph-mon[36169]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:07.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T09:06:07.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:07 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:07.744 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:07 smithi053 ceph-mon[36169]: Deploying daemon osd.8 on smithi071 2024-03-21T09:06:07.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:07 smithi037 ceph-mon[31830]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:07.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T09:06:07.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:07 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:07.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:07 smithi037 ceph-mon[31830]: Deploying daemon osd.8 on smithi071 2024-03-21T09:06:09.835 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:09 smithi071 ceph-mon[36597]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:09.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:09 smithi037 ceph-mon[31830]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:09.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:09 smithi053 ceph-mon[36169]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:11.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:11 smithi071 ceph-mon[36597]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:11.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:11 smithi037 ceph-mon[31830]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:11.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:11 smithi053 ceph-mon[36169]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:13.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:13 smithi071 ceph-mon[36597]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:13.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:13.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:13.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:13 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:13 smithi037 ceph-mon[31830]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:13 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:13.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:13 smithi053 ceph-mon[36169]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:13.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:13.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:13.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:13 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:15.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:15 smithi037 ceph-mon[31830]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:15.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:15.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:15.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:15 smithi053 ceph-mon[36169]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:15.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:15.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:15.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:15 smithi071 ceph-mon[36597]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:15.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:15.528 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:15.529 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:15.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.836+0000 7fd3eb7fe700 1 -- 172.21.15.71:0/3810085136 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd3fc0625d0 con 0x7fd3e40a2fb0 2024-03-21T09:06:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 8 on host 'smithi071' 2024-03-21T09:06:15.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3e40a2fb0 msgr2=0x7fd3e40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:15.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3e40a2fb0 0x7fd3e40a5470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc102120 tx=0x7fd3ec006d20 comp rx=0 tx=0).stop 2024-03-21T09:06:15.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 msgr2=0x7fd3fc198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc198980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0004790 tx=0x7fd3f0009580 comp rx=0 tx=0).stop 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 shutdown_connections 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3e40a2fb0 0x7fd3e40a5470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc102c60 0x7fd3fc19fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3fc1010c0 0x7fd3fc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 --2- 172.21.15.71:0/3810085136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3fc1022c0 0x7fd3fc198980 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 >> 172.21.15.71:0/3810085136 conn(0x7fd3fc0fc680 msgr2=0x7fd3fc1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:06:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 shutdown_connections 2024-03-21T09:06:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:06:15.838+0000 7fd400de4700 1 -- 172.21.15.71:0/3810085136 wait complete. 2024-03-21T09:06:16.509 DEBUG:teuthology.orchestra.run.smithi071:osd.8> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.8.service 2024-03-21T09:06:16.512 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-03-21T09:06:16.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T09:06:16.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:16 smithi037 ceph-mon[31830]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:16.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:16.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:16.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:16.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:16 smithi037 ceph-mon[31830]: from='osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T09:06:16.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:16 smithi037 ceph-mon[31830]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T09:06:17.000 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:06:17.071 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:16 smithi071 ceph-mon[36597]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:17.071 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:17.071 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:17.071 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:17.071 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:16 smithi071 ceph-mon[36597]: from='osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T09:06:17.071 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:16 smithi071 ceph-mon[36597]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T09:06:17.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:16 smithi053 ceph-mon[36169]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:17.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:17.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:17.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:17.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:16 smithi053 ceph-mon[36169]: from='osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T09:06:17.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:16 smithi053 ceph-mon[36169]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T09:06:18.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-mon[36597]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T09:06:18.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-mon[36597]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T09:06:18.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:18.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-mon[36597]: from='osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:06:18.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-mon[36597]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:06:18.038 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:06:17.835+0000 7fa9c55f1700 -1 osd.8 0 waiting for initial osdmap 2024-03-21T09:06:18.038 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:06:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:06:17.845+0000 7fa9c13e4700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:06:18.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:17 smithi037 ceph-mon[31830]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T09:06:18.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:17 smithi037 ceph-mon[31830]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T09:06:18.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:18.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:17 smithi037 ceph-mon[31830]: from='osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:06:18.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:17 smithi037 ceph-mon[31830]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:06:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:17 smithi053 ceph-mon[36169]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T09:06:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:17 smithi053 ceph-mon[36169]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T09:06:18.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:18.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:17 smithi053 ceph-mon[36169]: from='osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:06:18.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:17 smithi053 ceph-mon[36169]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:06:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:18 smithi071 ceph-mon[36597]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:18 smithi071 ceph-mon[36597]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:06:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:18 smithi071 ceph-mon[36597]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T09:06:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:18 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:18 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:19.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:18 smithi037 ceph-mon[31830]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:19.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:18 smithi037 ceph-mon[31830]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:06:19.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:18 smithi037 ceph-mon[31830]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T09:06:19.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:18 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:19.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:18 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:19.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:18 smithi053 ceph-mon[36169]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:19.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:18 smithi053 ceph-mon[36169]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:06:19.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:18 smithi053 ceph-mon[36169]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T09:06:19.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:18 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:19.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:18 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] boot 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: osdmap e59: 9 total, 9 up, 9 in 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi071 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi071 to 5693M 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:20.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:06:20.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:06:20.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] boot 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: osdmap e59: 9 total, 9 up, 9 in 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi071 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi071 to 5693M 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:20.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:06:20.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:06:20.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: osd.8 [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] boot 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: osdmap e59: 9 total, 9 up, 9 in 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi071 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi071 to 5693M 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:20.193 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T09:06:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T09:06:20.220 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-21T09:06:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-21T09:06:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350313 s, 299 MB/s 2024-03-21T09:06:20.220 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 2024-03-21T09:06:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-21T09:06:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:20 smithi037 ceph-mon[31830]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:20 smithi037 ceph-mon[31830]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T09:06:21.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:20 smithi071 ceph-mon[36597]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:21.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:20 smithi071 ceph-mon[36597]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T09:06:21.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:20 smithi053 ceph-mon[36169]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-21T09:06:21.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:20 smithi053 ceph-mon[36169]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T09:06:21.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-03-21T09:06:21.654 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:06:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:21 smithi037 ceph-mon[31830]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T09:06:22.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:21 smithi053 ceph-mon[36169]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T09:06:22.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:21 smithi071 ceph-mon[36597]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T09:06:23.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:22 smithi037 ceph-mon[31830]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:23.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:22 smithi037 ceph-mon[31830]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T09:06:23.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:22 smithi053 ceph-mon[36169]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:23.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:22 smithi053 ceph-mon[36169]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T09:06:23.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:22 smithi071 ceph-mon[36597]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:23.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:22 smithi071 ceph-mon[36597]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 -- 172.21.15.37:0/3802360708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0107eb0 msgr2=0x7f0ef010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3802360708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0107eb0 0x7f0ef010a320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800b0a0 tx=0x7f0ee800b3b0 comp rx=0 tx=0).stop 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 -- 172.21.15.37:0/3802360708 shutdown_connections 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3802360708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0107eb0 0x7f0ef010a320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3802360708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 0x7f0ef0105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3802360708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef0105580 0x7f0ef0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:23.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.860+0000 7f0ef6000700 1 -- 172.21.15.37:0/3802360708 >> 172.21.15.37:0/3802360708 conn(0x7f0ef00fc660 msgr2=0x7f0ef00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:06:23.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.861+0000 7f0ef6000700 1 -- 172.21.15.37:0/3802360708 shutdown_connections 2024-03-21T09:06:23.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.861+0000 7f0ef6000700 1 -- 172.21.15.37:0/3802360708 wait complete. 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 Processor -- start 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 -- start start 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 0x7f0ef0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 0x7f0ef0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef0107eb0 0x7f0ef019f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef010de20 con 0x7f0ef0105580 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef010dca0 con 0x7f0ef0102c50 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.862+0000 7f0ef6000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef010dfa0 con 0x7f0ef0107eb0 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 0x7f0ef0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 0x7f0ef0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eef400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 0x7f0ef0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54222/0 (socket says 172.21.15.37:54222) 2024-03-21T09:06:23.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 0x7f0ef0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54048/0 (socket says 172.21.15.37:54048) 2024-03-21T09:06:23.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eef400700 1 -- 172.21.15.37:0/3346673122 learned_addr learned my addr 172.21.15.37:0/3346673122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:06:23.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eefe00700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef0107eb0 0x7f0ef019f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 -- 172.21.15.37:0/3346673122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef0107eb0 msgr2=0x7f0ef019f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef0107eb0 0x7f0ef019f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 -- 172.21.15.37:0/3346673122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 msgr2=0x7f0ef0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 0x7f0ef0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 -- 172.21.15.37:0/3346673122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef01a0090 con 0x7f0ef0105580 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.863+0000 7f0eeea00700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 0x7f0ef0198970 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00ea20 tx=0x7f0edc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.864+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00cbd0 con 0x7f0ef0105580 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.864+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0edc004500 con 0x7f0ef0105580 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.864+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc005020 con 0x7f0ef0105580 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.864+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef01a0380 con 0x7f0ef0105580 2024-03-21T09:06:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.864+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef01a0870 con 0x7f0ef0105580 2024-03-21T09:06:23.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.865+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef0068e50 con 0x7f0ef0105580 2024-03-21T09:06:23.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.867+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0edc0051c0 con 0x7f0ef0105580 2024-03-21T09:06:23.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.869+0000 7f0ee7e00700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0ed40a2fb0 0x7f0ed40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:23.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.869+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f0edc0cea00 con 0x7f0ef0105580 2024-03-21T09:06:23.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.869+0000 7f0eef400700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0ed40a2fb0 0x7f0ed40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:23.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.870+0000 7f0eef400700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0ed40a2fb0 0x7f0ed40a5470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009680 tx=0x7f0ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:06:23.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:23.872+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0edc09c1d0 con 0x7f0ef0105580 2024-03-21T09:06:24.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:24.116+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ef00625d0 con 0x7f0ed40a2fb0 2024-03-21T09:06:24.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:23 smithi037 ceph-mon[31830]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T09:06:24.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:23 smithi053 ceph-mon[36169]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T09:06:24.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:23 smithi071 ceph-mon[36597]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T09:06:25.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:24 smithi037 ceph-mon[31830]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:25.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:24 smithi037 ceph-mon[31830]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:06:25.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:24 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:06:25.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:24 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:06:25.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:24 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:25.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:24 smithi053 ceph-mon[36169]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:25.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:24 smithi053 ceph-mon[36169]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:06:25.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:24 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:06:25.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:24 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:06:25.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:24 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:25.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:24 smithi071 ceph-mon[36597]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:25.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:24 smithi071 ceph-mon[36597]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:06:25.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:24 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:06:25.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:24 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:06:25.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:24 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:27.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:26 smithi037 ceph-mon[31830]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-21T09:06:27.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:26 smithi053 ceph-mon[36169]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-21T09:06:27.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:26 smithi071 ceph-mon[36597]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-21T09:06:28.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:28 smithi037 ceph-mon[31830]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-21T09:06:28.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:28 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/276154656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]: dispatch 2024-03-21T09:06:28.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:28 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]: dispatch 2024-03-21T09:06:29.160 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:28 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]': finished 2024-03-21T09:06:29.160 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:28 smithi037 ceph-mon[31830]: osdmap e64: 10 total, 9 up, 10 in 2024-03-21T09:06:29.160 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:28 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:29.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:28 smithi053 ceph-mon[36169]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-21T09:06:29.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:28 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/276154656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]: dispatch 2024-03-21T09:06:29.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:28 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]: dispatch 2024-03-21T09:06:29.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:28 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]': finished 2024-03-21T09:06:29.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:28 smithi053 ceph-mon[36169]: osdmap e64: 10 total, 9 up, 10 in 2024-03-21T09:06:29.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:28 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:29.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:28 smithi071 ceph-mon[36597]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-21T09:06:29.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:28 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/276154656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]: dispatch 2024-03-21T09:06:29.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:28 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]: dispatch 2024-03-21T09:06:29.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:28 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ee448c-645a-44ee-aa9d-f8e9b5898ce0"}]': finished 2024-03-21T09:06:29.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:28 smithi071 ceph-mon[36597]: osdmap e64: 10 total, 9 up, 10 in 2024-03-21T09:06:29.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:28 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:30.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:29 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2244634841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:06:30.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:29 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2244634841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:06:30.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:29 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2244634841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:06:31.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:30 smithi037 ceph-mon[31830]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T09:06:31.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:30 smithi053 ceph-mon[36169]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T09:06:31.254 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:30 smithi071 ceph-mon[36597]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T09:06:33.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:32 smithi037 ceph-mon[31830]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T09:06:33.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:32 smithi053 ceph-mon[36169]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T09:06:33.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:32 smithi071 ceph-mon[36597]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T09:06:35.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:34 smithi037 ceph-mon[31830]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T09:06:35.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:34 smithi053 ceph-mon[36169]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T09:06:35.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:34 smithi071 ceph-mon[36597]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T09:06:37.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:36 smithi037 ceph-mon[31830]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:37.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:36 smithi053 ceph-mon[36169]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:37.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:36 smithi071 ceph-mon[36597]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:39.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:38 smithi053 ceph-mon[36169]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:39.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:38 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T09:06:39.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:38 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:39.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:38 smithi053 ceph-mon[36169]: Deploying daemon osd.9 on smithi037 2024-03-21T09:06:39.340 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:38 smithi037 ceph-mon[31830]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:39.340 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:38 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T09:06:39.340 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:38 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:39.340 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:38 smithi037 ceph-mon[31830]: Deploying daemon osd.9 on smithi037 2024-03-21T09:06:39.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:38 smithi071 ceph-mon[36597]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:39.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:38 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T09:06:39.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:38 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:39.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:38 smithi071 ceph-mon[36597]: Deploying daemon osd.9 on smithi037 2024-03-21T09:06:41.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:40 smithi053 ceph-mon[36169]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:41.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:40 smithi037 ceph-mon[31830]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:41.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:40 smithi071 ceph-mon[36597]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:43.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:42 smithi037 ceph-mon[31830]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:43.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:42 smithi053 ceph-mon[36169]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:43.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:42 smithi071 ceph-mon[36597]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:45.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:44 smithi053 ceph-mon[36169]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:45.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:44 smithi071 ceph-mon[36597]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:45.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:44 smithi037 ceph-mon[31830]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:47.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:46 smithi053 ceph-mon[36169]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:47.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:46 smithi071 ceph-mon[36597]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:47.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:46 smithi037 ceph-mon[31830]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:48.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:48.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:48.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:49.410 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:49 smithi037 ceph-mon[31830]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:49.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:49 smithi071 ceph-mon[36597]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:49.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:49 smithi053 ceph-mon[36169]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:51.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:51 smithi071 ceph-mon[36597]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:51.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:51 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:51.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:51 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:51.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:51 smithi071 ceph-mon[36597]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T09:06:51.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:51 smithi037 ceph-mon[31830]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:51.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:51 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:51.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:51 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:51.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:51 smithi037 ceph-mon[31830]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T09:06:51.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:51 smithi053 ceph-mon[36169]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:51.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:51 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:51.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:51 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:51.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:51 smithi053 ceph-mon[36169]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T09:06:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.279+0000 7f0ee7e00700 1 -- 172.21.15.37:0/3346673122 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0ef00625d0 con 0x7f0ed40a2fb0 2024-03-21T09:06:52.274 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0ed40a2fb0 msgr2=0x7f0ed40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0ed40a2fb0 0x7f0ed40a5470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009680 tx=0x7f0ee0009260 comp rx=0 tx=0).stop 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 msgr2=0x7f0ef0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 0x7f0ef0198970 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00ea20 tx=0x7f0edc00ed30 comp rx=0 tx=0).stop 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 shutdown_connections 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef0107eb0 0x7f0ef019f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0ed40a2fb0 0x7f0ed40a5470 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ef0102c50 0x7f0ef0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.282+0000 7f0ef6000700 1 --2- 172.21.15.37:0/3346673122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ef0105580 0x7f0ef0198970 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.283+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 >> 172.21.15.37:0/3346673122 conn(0x7f0ef00fc660 msgr2=0x7f0ef0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.283+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 shutdown_connections 2024-03-21T09:06:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:06:52.283+0000 7f0ef6000700 1 -- 172.21.15.37:0/3346673122 wait complete. 2024-03-21T09:06:53.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:53.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T09:06:53.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T09:06:53.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:06:53.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:53.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:53.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:53.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:52 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:53.089 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.9.service 2024-03-21T09:06:53.094 INFO:tasks.cephadm:Deploying osd.10 on smithi053 with /dev/vg_nvme/lv_1... 2024-03-21T09:06:53.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:52 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:53.128 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:53.128 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T09:06:53.128 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T09:06:53.128 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:53.129 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:52 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:53.353 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:06:54.166 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:53 smithi053 ceph-mon[36169]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:06:54.166 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:53 smithi053 ceph-mon[36169]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T09:06:54.166 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:54.166 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:53 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:53 smithi037 ceph-mon[31830]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:06:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:53 smithi037 ceph-mon[31830]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T09:06:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:53 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:53 smithi071 ceph-mon[36597]: from='osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-21T09:06:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:53 smithi071 ceph-mon[36597]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T09:06:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:53 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:54 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:06:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:54 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:06:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:54 smithi037 ceph-mon[31830]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:54 smithi037 ceph-mon[31830]: osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] boot 2024-03-21T09:06:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:54 smithi037 ceph-mon[31830]: osdmap e67: 10 total, 10 up, 10 in 2024-03-21T09:06:54.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:54 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:55.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:54 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:06:55.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:54 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:06:55.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:54 smithi053 ceph-mon[36169]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:55.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:54 smithi053 ceph-mon[36169]: osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] boot 2024-03-21T09:06:55.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:54 smithi053 ceph-mon[36169]: osdmap e67: 10 total, 10 up, 10 in 2024-03-21T09:06:55.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:54 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:54 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:06:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:54 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:06:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:54 smithi071 ceph-mon[36597]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-21T09:06:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:54 smithi071 ceph-mon[36597]: osd.9 [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] boot 2024-03-21T09:06:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:54 smithi071 ceph-mon[36597]: osdmap e67: 10 total, 10 up, 10 in 2024-03-21T09:06:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:54 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:06:55.421 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T09:06:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T09:06:55.422 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-03-21T09:06:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-03-21T09:06:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00821517 s, 1.3 GB/s 2024-03-21T09:06:55.422 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-03-21T09:06:56.117 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:55 smithi053 ceph-mon[36169]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T09:06:56.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-03-21T09:06:56.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:55 smithi037 ceph-mon[31830]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T09:06:56.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:55 smithi071 ceph-mon[36597]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T09:06:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:06:57.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:56 smithi037 ceph-mon[31830]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:06:57.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:56 smithi053 ceph-mon[36169]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:06:57.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:56 smithi071 ceph-mon[36597]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:06:58.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.146+0000 7fe524dcc700 1 -- 172.21.15.53:0/1771563698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 msgr2=0x7fe52010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.146+0000 7fe524dcc700 1 --2- 172.21.15.53:0/1771563698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52010a610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b0d0 tx=0x7fe51800b3e0 comp rx=0 tx=0).stop 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 -- 172.21.15.53:0/1771563698 shutdown_connections 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 --2- 172.21.15.53:0/1771563698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52010a610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 --2- 172.21.15.53:0/1771563698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5201021c0 0x7fe5201025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 --2- 172.21.15.53:0/1771563698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5201033c0 0x7fe520103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 -- 172.21.15.53:0/1771563698 >> 172.21.15.53:0/1771563698 conn(0x7fe5200fd7a0 msgr2=0x7fe5200ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 -- 172.21.15.53:0/1771563698 shutdown_connections 2024-03-21T09:06:58.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 -- 172.21.15.53:0/1771563698 wait complete. 2024-03-21T09:06:58.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 Processor -- start 2024-03-21T09:06:58.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.147+0000 7fe524dcc700 1 -- start start 2024-03-21T09:06:58.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe524dcc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5201021c0 0x7fe520198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:58.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe524dcc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5201033c0 0x7fe520198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:58.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe524dcc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:58.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe524dcc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52010de80 con 0x7fe5201033c0 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe524dcc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52010dd00 con 0x7fe520103d60 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe524dcc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52010e000 con 0x7fe5201021c0 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52019fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52954/0 (socket says 172.21.15.53:52954) 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 -- 172.21.15.53:0/3917880200 learned_addr learned my addr 172.21.15.53:0/3917880200 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51e59c700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5201021c0 0x7fe520198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 -- 172.21.15.53:0/3917880200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5201021c0 msgr2=0x7fe520198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5201021c0 0x7fe520198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 -- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5201033c0 msgr2=0x7fe520198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5201033c0 0x7fe520198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 -- 172.21.15.53:0/3917880200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5201a00a0 con 0x7fe520103d60 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe51ed9d700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52019fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b770 tx=0x7fe5180079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:06:58.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.148+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518003e10 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.149+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe518007c80 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.149+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5201a0330 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.149+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51801bac0 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.149+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5201a0730 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.150+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52002da30 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.151+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe518026400 con 0x7fe520103d60 2024-03-21T09:06:58.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.151+0000 7fe50f7fe700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe5080a2fb0 0x7fe5080a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:06:58.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.151+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fe5180d0300 con 0x7fe520103d60 2024-03-21T09:06:58.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.152+0000 7fe51e59c700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe5080a2fb0 0x7fe5080a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:06:58.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.153+0000 7fe51e59c700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe5080a2fb0 0x7fe5080a5470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe520103220 tx=0x7fe510009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:06:58.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.156+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe51809d9c0 con 0x7fe520103d60 2024-03-21T09:06:58.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:06:58.371+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5200625d0 con 0x7fe5080a2fb0 2024-03-21T09:06:59.285 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi037 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi037 to 4273M 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.286 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:06:59.287 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:06:59.287 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:06:59.287 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:06:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:06:59.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi037 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi037 to 4273M 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:06:59.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:06:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi037 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi037 to 4273M 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:06:59.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:06:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:01.405 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:01 smithi071 ceph-mon[36597]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:01.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:01 smithi037 ceph-mon[31830]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:01.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:01 smithi053 ceph-mon[36169]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/3445644180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]: dispatch 2024-03-21T09:07:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:02 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]: dispatch 2024-03-21T09:07:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:02 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]': finished 2024-03-21T09:07:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:02 smithi037 ceph-mon[31830]: osdmap e69: 11 total, 10 up, 11 in 2024-03-21T09:07:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:02 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.53:0/3809451605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:07:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:02 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/3445644180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]: dispatch 2024-03-21T09:07:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:02 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]: dispatch 2024-03-21T09:07:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:02 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]': finished 2024-03-21T09:07:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:02 smithi053 ceph-mon[36169]: osdmap e69: 11 total, 10 up, 11 in 2024-03-21T09:07:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:02 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:02.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:02 smithi053 ceph-mon[36169]: from='client.? 172.21.15.53:0/3809451605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:07:02.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:02 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/3445644180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]: dispatch 2024-03-21T09:07:02.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:02 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]: dispatch 2024-03-21T09:07:02.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:02 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4c10007-9a20-4448-981e-e97b3650beca"}]': finished 2024-03-21T09:07:02.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:02 smithi071 ceph-mon[36597]: osdmap e69: 11 total, 10 up, 11 in 2024-03-21T09:07:02.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:02 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:02.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:02 smithi071 ceph-mon[36597]: from='client.? 172.21.15.53:0/3809451605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:07:03.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:03 smithi071 ceph-mon[36597]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:03.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:03 smithi037 ceph-mon[31830]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:03.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:03 smithi053 ceph-mon[36169]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:05.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:05 smithi071 ceph-mon[36597]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:05.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:05 smithi037 ceph-mon[31830]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:05.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:05 smithi053 ceph-mon[36169]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:07.460 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:07 smithi053 ceph-mon[36169]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:07.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:07 smithi071 ceph-mon[36597]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:07.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:07 smithi037 ceph-mon[31830]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:09.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:09 smithi053 ceph-mon[36169]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:09.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T09:07:09.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:09 smithi071 ceph-mon[36597]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T09:07:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:09.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:09 smithi037 ceph-mon[31830]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:09.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T09:07:09.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:10.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:10 smithi053 ceph-mon[36169]: Deploying daemon osd.10 on smithi053 2024-03-21T09:07:10.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:10 smithi071 ceph-mon[36597]: Deploying daemon osd.10 on smithi053 2024-03-21T09:07:10.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:10 smithi037 ceph-mon[31830]: Deploying daemon osd.10 on smithi053 2024-03-21T09:07:11.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:11 smithi071 ceph-mon[36597]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:11.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:11 smithi037 ceph-mon[31830]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:11.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:11 smithi053 ceph-mon[36169]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:13.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:13 smithi053 ceph-mon[36169]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:13.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:13 smithi071 ceph-mon[36597]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:13.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:13 smithi037 ceph-mon[31830]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:15.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:15 smithi053 ceph-mon[36169]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:15.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:15.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:15.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:15 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:15.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:15 smithi071 ceph-mon[36597]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:15 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:15 smithi037 ceph-mon[31830]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:15 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:17.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:17 smithi053 ceph-mon[36169]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:17.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:17 smithi071 ceph-mon[36597]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:17.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:17 smithi037 ceph-mon[31830]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:18.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.359+0000 7fe50f7fe700 1 -- 172.21.15.53:0/3917880200 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe5200625d0 con 0x7fe5080a2fb0 2024-03-21T09:07:18.353 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 10 on host 'smithi053' 2024-03-21T09:07:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe5080a2fb0 msgr2=0x7fe5080a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe5080a2fb0 0x7fe5080a5470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe520103220 tx=0x7fe510009260 comp rx=0 tx=0).stop 2024-03-21T09:07:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 msgr2=0x7fe52019fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52019fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b770 tx=0x7fe5180079a0 comp rx=0 tx=0).stop 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 shutdown_connections 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe5080a2fb0 0x7fe5080a5470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520103d60 0x7fe52019fa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5201021c0 0x7fe520198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 --2- 172.21.15.53:0/3917880200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5201033c0 0x7fe520198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 >> 172.21.15.53:0/3917880200 conn(0x7fe5200fd7a0 msgr2=0x7fe5201066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 shutdown_connections 2024-03-21T09:07:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-21T09:07:18.361+0000 7fe524dcc700 1 -- 172.21.15.53:0/3917880200 wait complete. 2024-03-21T09:07:19.070 DEBUG:teuthology.orchestra.run.smithi053:osd.10> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.10.service 2024-03-21T09:07:19.073 INFO:tasks.cephadm:Deploying osd.11 on smithi071 with /dev/vg_nvme/lv_1... 2024-03-21T09:07:19.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T09:07:19.318 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:07:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:19.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: from='osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T09:07:19.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T09:07:19.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:19 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: from='osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.440 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:19 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.623 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:19.623 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.623 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.623 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: from='osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T09:07:19.623 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T09:07:19.624 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:19.624 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:19 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-mon[36169]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T09:07:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-mon[36169]: from='osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:07:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-mon[36169]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T09:07:20.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:20.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-mon[36169]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:07:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:20 smithi071 ceph-mon[36597]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T09:07:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:20 smithi071 ceph-mon[36597]: from='osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:07:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:20 smithi071 ceph-mon[36597]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T09:07:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:20 smithi071 ceph-mon[36597]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:07:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:20 smithi037 ceph-mon[31830]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T09:07:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:20 smithi037 ceph-mon[31830]: from='osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:07:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:20 smithi037 ceph-mon[31830]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T09:07:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:20 smithi037 ceph-mon[31830]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-03-21T09:07:21.132 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:07:20.818+0000 7f77edf29700 -1 osd.10 0 waiting for initial osdmap 2024-03-21T09:07:21.132 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:07:20 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:07:20.834+0000 7f77e5d14700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:07:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: from='osd.10 ' entity='osd.10' 2024-03-21T09:07:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:21 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T09:07:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T09:07:21.536 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-21T09:07:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-21T09:07:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111504 s, 940 MB/s 2024-03-21T09:07:21.537 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 2024-03-21T09:07:21.537 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-21T09:07:21.631 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:21.631 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:07:21.631 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T09:07:21.631 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.631 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T09:07:21.632 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.632 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: from='osd.10 ' entity='osd.10' 2024-03-21T09:07:21.632 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:21 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-03-21T09:07:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T09:07:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T09:07:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:21.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: from='osd.10 ' entity='osd.10' 2024-03-21T09:07:21.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:21 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:22.213 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-03-21T09:07:22.457 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:07:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:22 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:07:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:22 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:07:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:22 smithi053 ceph-mon[36169]: osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] boot 2024-03-21T09:07:22.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:22 smithi053 ceph-mon[36169]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T09:07:22.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:22 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:22.851 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:22 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:07:22.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:22 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:07:22.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:22 smithi071 ceph-mon[36597]: osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] boot 2024-03-21T09:07:22.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:22 smithi071 ceph-mon[36597]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T09:07:22.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:22 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:22.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:22 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:07:22.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:22 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:07:22.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:22 smithi037 ceph-mon[31830]: osd.10 [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] boot 2024-03-21T09:07:22.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:22 smithi037 ceph-mon[31830]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T09:07:22.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:22 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:07:23.720 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:23.720 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T09:07:23.720 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:23.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:23.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T09:07:23.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:23.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T09:07:23.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.306+0000 7f6df2cf7700 1 -- 172.21.15.71:0/2448174027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1033c0 msgr2=0x7f6dec103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.306+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/2448174027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1033c0 0x7f6dec103820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009990 tx=0x7f6dd8009ca0 comp rx=0 tx=0).stop 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.306+0000 7f6df2cf7700 1 -- 172.21.15.71:0/2448174027 shutdown_connections 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/2448174027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dec103d60 0x7f6dec10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/2448174027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dec1021c0 0x7f6dec1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/2448174027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1033c0 0x7f6dec103820 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- 172.21.15.71:0/2448174027 >> 172.21.15.71:0/2448174027 conn(0x7f6dec0fd7a0 msgr2=0x7f6dec0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- 172.21.15.71:0/2448174027 shutdown_connections 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- 172.21.15.71:0/2448174027 wait complete. 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 Processor -- start 2024-03-21T09:07:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- start start 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 0x7f6dec075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dec1033c0 0x7f6dec074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dec103d60 0x7f6dec074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec10def0 con 0x7f6dec103d60 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec10dd70 con 0x7f6dec1033c0 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df2cf7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec10e070 con 0x7f6dec1021c0 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.307+0000 7f6df0a93700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 0x7f6dec075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 0x7f6dec075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41410/0 (socket says 172.21.15.71:41410) 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 -- 172.21.15.71:0/320379730 learned_addr learned my addr 172.21.15.71:0/320379730 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6debfff700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dec1033c0 0x7f6dec074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 -- 172.21.15.71:0/320379730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dec1033c0 msgr2=0x7f6dec074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dec1033c0 0x7f6dec074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 -- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dec103d60 msgr2=0x7f6dec074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dec103d60 0x7f6dec074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 -- 172.21.15.71:0/320379730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec075370 con 0x7f6dec1021c0 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df0a93700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 0x7f6dec075f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b500 tx=0x7f6de000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0015020 con 0x7f6dec1021c0 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6de0011d10 con 0x7f6dec1021c0 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec1a2720 con 0x7f6dec1021c0 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000f2e0 con 0x7f6dec1021c0 2024-03-21T09:07:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.308+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec1a2cb0 con 0x7f6dec1021c0 2024-03-21T09:07:24.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.309+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec068e50 con 0x7f6dec1021c0 2024-03-21T09:07:24.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.310+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6de0018020 con 0x7f6dec1021c0 2024-03-21T09:07:24.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.311+0000 7f6de9ffb700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ddc0a3030 0x7f6ddc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:24.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.311+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f6de00ce580 con 0x7f6dec1021c0 2024-03-21T09:07:24.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.311+0000 7f6debfff700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ddc0a3030 0x7f6ddc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:24.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.312+0000 7f6debfff700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ddc0a3030 0x7f6ddc0a54f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8003a60 tx=0x7f6dd80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:24.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.315+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6de001f020 con 0x7f6dec1021c0 2024-03-21T09:07:24.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:24.520+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dec0625d0 con 0x7f6ddc0a3030 2024-03-21T09:07:24.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:24 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi053 2024-03-21T09:07:24.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:24 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi053 to 4271M 2024-03-21T09:07:24.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:24 smithi071 ceph-mon[36597]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T09:07:24.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:24 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi053 2024-03-21T09:07:24.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:24 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi053 to 4271M 2024-03-21T09:07:24.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:24 smithi037 ceph-mon[31830]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T09:07:24.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:24 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi053 2024-03-21T09:07:24.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:24 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi053 to 4271M 2024-03-21T09:07:24.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:24 smithi053 ceph-mon[36169]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T09:07:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:25 smithi071 ceph-mon[36597]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:25 smithi071 ceph-mon[36597]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:07:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:25 smithi071 ceph-mon[36597]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T09:07:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:07:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:07:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:25 smithi037 ceph-mon[31830]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:25 smithi037 ceph-mon[31830]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:07:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:25 smithi037 ceph-mon[31830]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T09:07:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:07:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:07:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:25.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:25 smithi053 ceph-mon[36169]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:25.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:25 smithi053 ceph-mon[36169]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:07:25.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:25 smithi053 ceph-mon[36169]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T09:07:25.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:07:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:07:25.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:26 smithi071 ceph-mon[36597]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T09:07:26.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:26 smithi037 ceph-mon[31830]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T09:07:26.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:26 smithi053 ceph-mon[36169]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T09:07:27.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:27 smithi071 ceph-mon[36597]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T09:07:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:27 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/2084492503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]: dispatch 2024-03-21T09:07:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:27 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]: dispatch 2024-03-21T09:07:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:27 smithi071 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]': finished 2024-03-21T09:07:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:27 smithi071 ceph-mon[36597]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T09:07:27.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:27 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:27.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:27 smithi037 ceph-mon[31830]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T09:07:27.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:27 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/2084492503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]: dispatch 2024-03-21T09:07:27.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:27 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]: dispatch 2024-03-21T09:07:27.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:27 smithi037 ceph-mon[31830]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]': finished 2024-03-21T09:07:27.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:27 smithi037 ceph-mon[31830]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T09:07:27.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:27 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:27 smithi053 ceph-mon[36169]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T09:07:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:27 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/2084492503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]: dispatch 2024-03-21T09:07:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:27 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]: dispatch 2024-03-21T09:07:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:27 smithi053 ceph-mon[36169]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae4df7bc-7bbd-421b-8031-0599545220d8"}]': finished 2024-03-21T09:07:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:27 smithi053 ceph-mon[36169]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T09:07:27.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:27 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:28.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:28 smithi071 ceph-mon[36597]: from='client.? 172.21.15.71:0/3587588315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:07:28.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:28 smithi037 ceph-mon[31830]: from='client.? 172.21.15.71:0/3587588315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:07:28.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:28 smithi053 ceph-mon[36169]: from='client.? 172.21.15.71:0/3587588315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:07:29.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:29 smithi071 ceph-mon[36597]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-21T09:07:29.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:29 smithi037 ceph-mon[31830]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-21T09:07:29.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:29 smithi053 ceph-mon[36169]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-21T09:07:31.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:31 smithi071 ceph-mon[36597]: pgmap v270: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T09:07:31.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:31 smithi037 ceph-mon[31830]: pgmap v270: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T09:07:31.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:31 smithi053 ceph-mon[36169]: pgmap v270: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T09:07:33.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:33 smithi071 ceph-mon[36597]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T09:07:33.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:33 smithi037 ceph-mon[31830]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T09:07:33.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:33 smithi053 ceph-mon[36169]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T09:07:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:35 smithi071 ceph-mon[36597]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:35 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T09:07:35.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:35 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:35.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:35 smithi037 ceph-mon[31830]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:35.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:35 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T09:07:35.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:35 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:35.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:35 smithi053 ceph-mon[36169]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:35.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:35 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T09:07:35.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:35 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:36.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:36 smithi071 ceph-mon[36597]: Deploying daemon osd.11 on smithi071 2024-03-21T09:07:36.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:36 smithi037 ceph-mon[31830]: Deploying daemon osd.11 on smithi071 2024-03-21T09:07:36.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:36 smithi053 ceph-mon[36169]: Deploying daemon osd.11 on smithi071 2024-03-21T09:07:37.728 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:37 smithi071 ceph-mon[36597]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:37.843 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:37 smithi053 ceph-mon[36169]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:37 smithi037 ceph-mon[31830]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:39.798 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:39 smithi071 ceph-mon[36597]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:39 smithi037 ceph-mon[31830]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:39 smithi053 ceph-mon[36169]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:41.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:41 smithi071 ceph-mon[36597]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:41.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:41.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:41.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:41 smithi037 ceph-mon[31830]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:41 smithi053 ceph-mon[36169]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:43.740 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:43 smithi071 ceph-mon[36597]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:43.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:43 smithi037 ceph-mon[31830]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:43.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:43 smithi053 ceph-mon[36169]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.109+0000 7f6de9ffb700 1 -- 172.21.15.71:0/320379730 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f6dec0625d0 con 0x7f6ddc0a3030 2024-03-21T09:07:44.103 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 11 on host 'smithi071' 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ddc0a3030 msgr2=0x7f6ddc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ddc0a3030 0x7f6ddc0a54f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8003a60 tx=0x7f6dd80058e0 comp rx=0 tx=0).stop 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 msgr2=0x7f6dec075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 0x7f6dec075f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b500 tx=0x7f6de000b8c0 comp rx=0 tx=0).stop 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 shutdown_connections 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dec103d60 0x7f6dec074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec1021c0 0x7f6dec075f30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ddc0a3030 0x7f6ddc0a54f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:44.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 --2- 172.21.15.71:0/320379730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dec1033c0 0x7f6dec074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:44.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 >> 172.21.15.71:0/320379730 conn(0x7f6dec0fd7a0 msgr2=0x7f6dec106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:44.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 shutdown_connections 2024-03-21T09:07:44.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:44.111+0000 7f6df2cf7700 1 -- 172.21.15.71:0/320379730 wait complete. 2024-03-21T09:07:44.774 DEBUG:teuthology.orchestra.run.smithi071:osd.11> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.11.service 2024-03-21T09:07:44.777 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-21T09:07:44.777 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T09:07:45.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: from='osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T09:07:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T09:07:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.246 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:07:45.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:45.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: from='osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T09:07:45.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T09:07:45.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:45.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: from='osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T09:07:45.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T09:07:45.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:45.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:46.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-mon[36597]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T09:07:46.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-mon[36597]: from='osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:07:46.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-mon[36597]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T09:07:46.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:46.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-mon[36597]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:07:46.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:46 smithi037 ceph-mon[31830]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T09:07:46.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:46 smithi037 ceph-mon[31830]: from='osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:07:46.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:46 smithi037 ceph-mon[31830]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T09:07:46.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:46.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:46 smithi037 ceph-mon[31830]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:07:46.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:46 smithi053 ceph-mon[36169]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T09:07:46.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:46 smithi053 ceph-mon[36169]: from='osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:07:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:46 smithi053 ceph-mon[36169]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T09:07:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:46 smithi053 ceph-mon[36169]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-21T09:07:46.831 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:07:46.506+0000 7efe7c005700 -1 osd.11 0 waiting for initial osdmap 2024-03-21T09:07:46.831 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:07:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:07:46.522+0000 7efe73df0700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T09:07:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:47 smithi037 ceph-mon[31830]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:47 smithi037 ceph-mon[31830]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:07:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:47 smithi037 ceph-mon[31830]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T09:07:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:47 smithi037 ceph-mon[31830]: from='osd.11 ' entity='osd.11' 2024-03-21T09:07:47.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:47 smithi071 ceph-mon[36597]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:47.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:47 smithi071 ceph-mon[36597]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:07:47.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:47 smithi071 ceph-mon[36597]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T09:07:47.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:47.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:47 smithi071 ceph-mon[36597]: from='osd.11 ' entity='osd.11' 2024-03-21T09:07:47.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:47.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:47 smithi053 ceph-mon[36169]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:47.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:47 smithi053 ceph-mon[36169]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-21T09:07:47.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:47 smithi053 ceph-mon[36169]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T09:07:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:47 smithi053 ceph-mon[36169]: from='osd.11 ' entity='osd.11' 2024-03-21T09:07:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:47.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 -- 172.21.15.37:0/1890708131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 msgr2=0x7f2804105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 --2- 172.21.15.37:0/1890708131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f2804105ef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27f80099c0 tx=0x7f27f8009cd0 comp rx=0 tx=0).stop 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 -- 172.21.15.37:0/1890708131 shutdown_connections 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 --2- 172.21.15.37:0/1890708131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28041049b0 0x7f280410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 --2- 172.21.15.37:0/1890708131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f2804105ef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 --2- 172.21.15.37:0/1890708131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f2804104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 -- 172.21.15.37:0/1890708131 >> 172.21.15.37:0/1890708131 conn(0x7f28040ff9b0 msgr2=0x7f2804101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 -- 172.21.15.37:0/1890708131 shutdown_connections 2024-03-21T09:07:47.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.797+0000 7f280b600700 1 -- 172.21.15.37:0/1890708131 wait complete. 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.798+0000 7f280b600700 1 Processor -- start 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 -- start start 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f280419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28041049b0 0x7f280419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f28041a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28041100f0 con 0x7f28041049b0 2024-03-21T09:07:47.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280410ff70 con 0x7f2804104010 2024-03-21T09:07:47.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f280b600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804110270 con 0x7f2804105ad0 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f2808c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f280419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f2803e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28041049b0 0x7f280419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.799+0000 7f2809600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f28041a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f280419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:52682/0 (socket says 172.21.15.37:52682) 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 -- 172.21.15.37:0/4224942500 learned_addr learned my addr 172.21.15.37:0/4224942500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 -- 172.21.15.37:0/4224942500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 msgr2=0x7f28041a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f28041a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 -- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28041049b0 msgr2=0x7f280419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28041049b0 0x7f280419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 -- 172.21.15.37:0/4224942500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28041a2310 con 0x7f2804104010 2024-03-21T09:07:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2809600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f28041a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.800+0000 7f2808c00700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f280419a6b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27f800bf60 tx=0x7f27f800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.801+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8009e60 con 0x7f2804104010 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.801+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f8003e30 con 0x7f2804104010 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.801+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28041a25a0 con 0x7f2804104010 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.801+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800fdf0 con 0x7f2804104010 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.801+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28041a2ad0 con 0x7f2804104010 2024-03-21T09:07:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.802+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804068e50 con 0x7f2804104010 2024-03-21T09:07:47.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.804+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f27f800f4f0 con 0x7f2804104010 2024-03-21T09:07:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.805+0000 7f2801600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27ec0a3030 0x7f27ec0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.806+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27f80cfd70 con 0x7f2804104010 2024-03-21T09:07:47.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.806+0000 7f2803e00700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27ec0a3030 0x7f27ec0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:47.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.806+0000 7f2803e00700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27ec0a3030 0x7f27ec0a54f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009c60 tx=0x7f27f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:47.808+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f27f8013070 con 0x7f2804104010 2024-03-21T09:07:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.044+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f280402da30 con 0x7f2804104010 2024-03-21T09:07:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.045+0000 7f2801600700 1 -- 172.21.15.37:0/4224942500 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f27f809d030 con 0x7f2804104010 2024-03-21T09:07:48.039 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:07:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27ec0a3030 msgr2=0x7f27ec0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27ec0a3030 0x7f27ec0a54f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009c60 tx=0x7f27f0009260 comp rx=0 tx=0).stop 2024-03-21T09:07:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 msgr2=0x7f280419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f280419a6b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27f800bf60 tx=0x7f27f800bf90 comp rx=0 tx=0).stop 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 shutdown_connections 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2804105ad0 0x7f28041a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27ec0a3030 0x7f27ec0a54f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2804104010 0x7f280419a6b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 --2- 172.21.15.37:0/4224942500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28041049b0 0x7f280419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 >> 172.21.15.37:0/4224942500 conn(0x7f28040ff9b0 msgr2=0x7f28041088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 shutdown_connections 2024-03-21T09:07:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:48.048+0000 7f280b600700 1 -- 172.21.15.37:0/4224942500 wait complete. 2024-03-21T09:07:48.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:48 smithi071 ceph-mon[36597]: purged_snaps scrub starts 2024-03-21T09:07:48.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:48 smithi071 ceph-mon[36597]: purged_snaps scrub ok 2024-03-21T09:07:48.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:48 smithi071 ceph-mon[36597]: osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] boot 2024-03-21T09:07:48.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:48 smithi071 ceph-mon[36597]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T09:07:48.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:48.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:48 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4224942500' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T09:07:48.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:48 smithi037 ceph-mon[31830]: purged_snaps scrub starts 2024-03-21T09:07:48.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:48 smithi037 ceph-mon[31830]: purged_snaps scrub ok 2024-03-21T09:07:48.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:48 smithi037 ceph-mon[31830]: osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] boot 2024-03-21T09:07:48.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:48 smithi037 ceph-mon[31830]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T09:07:48.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:48.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:48 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4224942500' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T09:07:48.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:48 smithi053 ceph-mon[36169]: purged_snaps scrub starts 2024-03-21T09:07:48.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:48 smithi053 ceph-mon[36169]: purged_snaps scrub ok 2024-03-21T09:07:48.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:48 smithi053 ceph-mon[36169]: osd.11 [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] boot 2024-03-21T09:07:48.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:48 smithi053 ceph-mon[36169]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T09:07:48.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:07:48.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:48 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4224942500' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T09:07:48.841 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711012067,"num_in_osds":12,"osd_in_since":1711012047,"num_remapped_pgs":0} 2024-03-21T09:07:48.842 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T09:07:49.279 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: Detected new or changed devices on smithi071 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi071 to 4270M 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:49.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:49 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: Detected new or changed devices on smithi071 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi071 to 4270M 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:49.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:49 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: Detected new or changed devices on smithi071 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi071 to 4270M 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:49.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T09:07:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:07:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:07:49.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:49 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 -- 172.21.15.37:0/2364257504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d8104fb0 msgr2=0x7fe2d8109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 --2- 172.21.15.37:0/2364257504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d8104fb0 0x7fe2d8109000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00099c0 tx=0x7fe2c0009cd0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 -- 172.21.15.37:0/2364257504 shutdown_connections 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 --2- 172.21.15.37:0/2364257504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d81096e0 0x7fe2d81112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 --2- 172.21.15.37:0/2364257504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104600 0x7fe2d81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 --2- 172.21.15.37:0/2364257504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d8104fb0 0x7fe2d8109000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 -- 172.21.15.37:0/2364257504 >> 172.21.15.37:0/2364257504 conn(0x7fe2d80ffe90 msgr2=0x7fe2d81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 -- 172.21.15.37:0/2364257504 shutdown_connections 2024-03-21T09:07:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.533+0000 7fe2df400700 1 -- 172.21.15.37:0/2364257504 wait complete. 2024-03-21T09:07:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.534+0000 7fe2df400700 1 Processor -- start 2024-03-21T09:07:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.534+0000 7fe2df400700 1 -- start start 2024-03-21T09:07:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2df400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d8104600 0x7fe2d81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 0x7fe2d81a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2df400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d81096e0 0x7fe2d819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2df400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81146f0 con 0x7fe2d81096e0 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2df400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8114570 con 0x7fe2d8104600 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2df400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8114870 con 0x7fe2d8104fb0 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2cfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 0x7fe2d81a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2dca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d8104600 0x7fe2d81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2cfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 0x7fe2d81a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47488/0 (socket says 172.21.15.37:47488) 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2cfe00700 1 -- 172.21.15.37:0/1659377386 learned_addr learned my addr 172.21.15.37:0/1659377386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.535+0000 7fe2dd400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d81096e0 0x7fe2d819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.536+0000 7fe2cfe00700 1 -- 172.21.15.37:0/1659377386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d8104600 msgr2=0x7fe2d81a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.536+0000 7fe2cfe00700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d8104600 0x7fe2d81a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.536+0000 7fe2cfe00700 1 -- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d81096e0 msgr2=0x7fe2d819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.536+0000 7fe2cfe00700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d81096e0 0x7fe2d819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.536+0000 7fe2cfe00700 1 -- 172.21.15.37:0/1659377386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d819b130 con 0x7fe2d8104fb0 2024-03-21T09:07:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.536+0000 7fe2cfe00700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 0x7fe2d81a0d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005f50 tx=0x7fe2c0003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.540+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0003680 con 0x7fe2d8104fb0 2024-03-21T09:07:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.540+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c00055a0 con 0x7fe2d8104fb0 2024-03-21T09:07:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.540+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000fda0 con 0x7fe2d8104fb0 2024-03-21T09:07:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.540+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d819b3c0 con 0x7fe2d8104fb0 2024-03-21T09:07:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.540+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d81a7690 con 0x7fe2d8104fb0 2024-03-21T09:07:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.541+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8068e50 con 0x7fe2d8104fb0 2024-03-21T09:07:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.543+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe2c000f460 con 0x7fe2d8104fb0 2024-03-21T09:07:51.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.544+0000 7fe2d6000700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe2bc0a31d0 0x7fe2bc0a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:51.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.544+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2c00cfe00 con 0x7fe2d8104fb0 2024-03-21T09:07:51.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.544+0000 7fe2dca00700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe2bc0a31d0 0x7fe2bc0a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:51.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.545+0000 7fe2dca00700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe2bc0a31d0 0x7fe2bc0a5690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8100290 tx=0x7fe2c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:51.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.548+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2c0013070 con 0x7fe2d8104fb0 2024-03-21T09:07:51.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:51 smithi037 ceph-mon[31830]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.779+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2d802da30 con 0x7fe2d8104fb0 2024-03-21T09:07:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.781+0000 7fe2d6000700 1 -- 172.21.15.37:0/1659377386 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18370 (secure 0 0 0) 0x7fe2c009d0c0 con 0x7fe2d8104fb0 2024-03-21T09:07:51.776 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:07:51.776 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:07:49.100221+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe2bc0a31d0 msgr2=0x7fe2bc0a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe2bc0a31d0 0x7fe2bc0a5690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8100290 tx=0x7fe2c8009260 comp rx=0 tx=0).stop 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 msgr2=0x7fe2d81a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 0x7fe2d81a0d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005f50 tx=0x7fe2c0003a90 comp rx=0 tx=0).stop 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 shutdown_connections 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2d81096e0 0x7fe2d819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe2bc0a31d0 0x7fe2bc0a5690 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d8104600 0x7fe2d81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 --2- 172.21.15.37:0/1659377386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2d8104fb0 0x7fe2d81a0d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 >> 172.21.15.37:0/1659377386 conn(0x7fe2d80ffe90 msgr2=0x7fe2d8100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 shutdown_connections 2024-03-21T09:07:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:51.784+0000 7fe2df400700 1 -- 172.21.15.37:0/1659377386 wait complete. 2024-03-21T09:07:51.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:51 smithi071 ceph-mon[36597]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:51.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:51 smithi053 ceph-mon[36169]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:52.662 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T09:07:52.662 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T09:07:52.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:52 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1659377386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:07:52.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:52 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1659377386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:07:52.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:52 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1659377386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:07:53.125 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:07:53.819 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:53 smithi037 ceph-mon[31830]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:53.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:53 smithi071 ceph-mon[36597]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:53.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:53 smithi053 ceph-mon[36169]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 -- 172.21.15.37:0/567074420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a10074ce0 msgr2=0x7f4a1010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 --2- 172.21.15.37:0/567074420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a10074ce0 0x7f4a1010e650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098f0 tx=0x7f49fc009c00 comp rx=0 tx=0).stop 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 -- 172.21.15.37:0/567074420 shutdown_connections 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 --2- 172.21.15.37:0/567074420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a1010eb90 0x7f4a10110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 --2- 172.21.15.37:0/567074420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a100743c0 0x7f4a100747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 --2- 172.21.15.37:0/567074420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a10074ce0 0x7f4a1010e650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 -- 172.21.15.37:0/567074420 >> 172.21.15.37:0/567074420 conn(0x7f4a100792f0 msgr2=0x7f4a10079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 -- 172.21.15.37:0/567074420 shutdown_connections 2024-03-21T09:07:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.470+0000 7f4a18000700 1 -- 172.21.15.37:0/567074420 wait complete. 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.471+0000 7f4a18000700 1 Processor -- start 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.471+0000 7f4a18000700 1 -- start start 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a18000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 0x7f4a101022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a18000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a10074ce0 0x7f4a101027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a18000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1010eb90 0x7f4a10102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a18000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10114790 con 0x7f4a100743c0 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a18000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10114610 con 0x7f4a10074ce0 2024-03-21T09:07:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a18000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10114910 con 0x7f4a1010eb90 2024-03-21T09:07:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a15600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 0x7f4a101022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a15600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 0x7f4a101022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51274/0 (socket says 172.21.15.37:51274) 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a15600700 1 -- 172.21.15.37:0/2741495609 learned_addr learned my addr 172.21.15.37:0/2741495609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a16000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1010eb90 0x7f4a10102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.472+0000 7f4a14c00700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a10074ce0 0x7f4a101027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a15600700 1 -- 172.21.15.37:0/2741495609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1010eb90 msgr2=0x7f4a10102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a15600700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1010eb90 0x7f4a10102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a15600700 1 -- 172.21.15.37:0/2741495609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a10074ce0 msgr2=0x7f4a101027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a15600700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a10074ce0 0x7f4a101027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a15600700 1 -- 172.21.15.37:0/2741495609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a101ab2f0 con 0x7f4a100743c0 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a16000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1010eb90 0x7f4a10102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a15600700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 0x7f4a101022b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400e9a0 tx=0x7f4a0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400ca10 con 0x7f4a100743c0 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a101ab5e0 con 0x7f4a100743c0 2024-03-21T09:07:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a0400cbb0 con 0x7f4a100743c0 2024-03-21T09:07:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.473+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04018710 con 0x7f4a100743c0 2024-03-21T09:07:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.474+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a101abb40 con 0x7f4a100743c0 2024-03-21T09:07:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.475+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a10075af0 con 0x7f4a100743c0 2024-03-21T09:07:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.477+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4a040188b0 con 0x7f4a100743c0 2024-03-21T09:07:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.478+0000 7f4a0e000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f80a2fb0 0x7f49f80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.478+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a040cf120 con 0x7f4a100743c0 2024-03-21T09:07:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.478+0000 7f4a14c00700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f80a2fb0 0x7f49f80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.479+0000 7f4a14c00700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f80a2fb0 0x7f49f80a5470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009810 tx=0x7f49fc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.483+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a0409c350 con 0x7f4a100743c0 2024-03-21T09:07:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.710+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a101037e0 con 0x7f4a100743c0 2024-03-21T09:07:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.711+0000 7f4a0e000700 1 -- 172.21.15.37:0/2741495609 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a0409bc50 con 0x7f4a100743c0 2024-03-21T09:07:55.705 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.713+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f80a2fb0 msgr2=0x7f49f80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.713+0000 7f4a18000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f80a2fb0 0x7f49f80a5470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009810 tx=0x7f49fc019040 comp rx=0 tx=0).stop 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.713+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 msgr2=0x7f4a101022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.713+0000 7f4a18000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 0x7f4a101022b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400e9a0 tx=0x7f4a0400ecb0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 shutdown_connections 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1010eb90 0x7f4a10102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a100743c0 0x7f4a101022b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f80a2fb0 0x7f49f80a5470 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 --2- 172.21.15.37:0/2741495609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a10074ce0 0x7f4a101027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 >> 172.21.15.37:0/2741495609 conn(0x7f4a100792f0 msgr2=0x7f4a100ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 shutdown_connections 2024-03-21T09:07:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:55.714+0000 7f4a18000700 1 -- 172.21.15.37:0/2741495609 wait complete. 2024-03-21T09:07:55.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:55 smithi071 ceph-mon[36597]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:55.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:55 smithi037 ceph-mon[31830]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:55.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:55 smithi053 ceph-mon[36169]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:56.504 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.b on smithi053 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.e on smithi053 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.h on smithi053 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.k on smithi053 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.c on smithi071 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.f on smithi071 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.i on smithi071 2024-03-21T09:07:56.505 INFO:tasks.cephadm:Adding mds.l on smithi071 2024-03-21T09:07:56.506 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l' 2024-03-21T09:07:56.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:56 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2741495609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:07:56.754 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:07:56.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:56 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2741495609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:07:56.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:56 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2741495609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:07:57.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:57 smithi071 ceph-mon[36597]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:57.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:57 smithi037 ceph-mon[31830]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:57.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:57 smithi053 ceph-mon[36169]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 -- 172.21.15.71:0/1995513965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6794074bf0 msgr2=0x7f6794075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 --2- 172.21.15.71:0/1995513965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6794074bf0 0x7f6794075070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6788009990 tx=0x7f6788009ca0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 -- 172.21.15.71:0/1995513965 shutdown_connections 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 --2- 172.21.15.71:0/1995513965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f679410e790 0x7f6794110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 --2- 172.21.15.71:0/1995513965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67940761c0 0x7f6794074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 --2- 172.21.15.71:0/1995513965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6794074bf0 0x7f6794075070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 -- 172.21.15.71:0/1995513965 >> 172.21.15.71:0/1995513965 conn(0x7f67940fd770 msgr2=0x7f67940ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 -- 172.21.15.71:0/1995513965 shutdown_connections 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 -- 172.21.15.71:0/1995513965 wait complete. 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.376+0000 7f6798d6e700 1 Processor -- start 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 -- start start 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794074bf0 0x7f679419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 0x7f679419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f679410e790 0x7f67941a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794114490 con 0x7f679410e790 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794114310 con 0x7f6794074bf0 2024-03-21T09:07:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794114610 con 0x7f67940761c0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 0x7f679419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 0x7f679419f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51848/0 (socket says 172.21.15.71:51848) 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 -- 172.21.15.71:0/2532475947 learned_addr learned my addr 172.21.15.71:0/2532475947 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f679259c700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794074bf0 0x7f679419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 -- 172.21.15.71:0/2532475947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794074bf0 msgr2=0x7f679419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794074bf0 0x7f679419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 -- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f679410e790 msgr2=0x7f67941a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f679410e790 0x7f67941a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 -- 172.21.15.71:0/2532475947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67941a67f0 con 0x7f67940761c0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6791d9b700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 0x7f679419f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67880047b0 tx=0x7f6788009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788026070 con 0x7f67940761c0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6788004210 con 0x7f67940761c0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67941a6a80 con 0x7f67940761c0 2024-03-21T09:07:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.377+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880053b0 con 0x7f67940761c0 2024-03-21T09:07:58.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.378+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67941a6fb0 con 0x7f67940761c0 2024-03-21T09:07:58.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.379+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794068e50 con 0x7f67940761c0 2024-03-21T09:07:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.380+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6788029070 con 0x7f67940761c0 2024-03-21T09:07:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.380+0000 7f67837fe700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f677c0a3030 0x7f677c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.380+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67880cfc00 con 0x7f67940761c0 2024-03-21T09:07:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.381+0000 7f679259c700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f677c0a3030 0x7f677c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.382+0000 7f679259c700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f677c0a3030 0x7f677c0a54f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f67940765a0 tx=0x7f6784009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:58.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.385+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f678805d740 con 0x7f67940761c0 2024-03-21T09:07:58.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.588+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f6794062890 con 0x7f677c0a3030 2024-03-21T09:07:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.604+0000 7f67837fe700 1 -- 172.21.15.71:0/2532475947 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6794062890 con 0x7f677c0a3030 2024-03-21T09:07:58.599 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mds.all update... 2024-03-21T09:07:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f677c0a3030 msgr2=0x7f677c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f677c0a3030 0x7f677c0a54f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f67940765a0 tx=0x7f6784009190 comp rx=0 tx=0).stop 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 msgr2=0x7f679419f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 0x7f679419f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67880047b0 tx=0x7f6788009ee0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 shutdown_connections 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f679410e790 0x7f67941a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f677c0a3030 0x7f677c0a54f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794074bf0 0x7f679419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 --2- 172.21.15.71:0/2532475947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67940761c0 0x7f679419f0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 >> 172.21.15.71:0/2532475947 conn(0x7f67940fd770 msgr2=0x7f6794101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 shutdown_connections 2024-03-21T09:07:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-21T09:07:58.606+0000 7f6798d6e700 1 -- 172.21.15.71:0/2532475947 wait complete. 2024-03-21T09:07:59.234 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.a.service 2024-03-21T09:07:59.242 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.d.service 2024-03-21T09:07:59.288 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.g.service 2024-03-21T09:07:59.333 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.j.service 2024-03-21T09:07:59.378 DEBUG:teuthology.orchestra.run.smithi053:mds.b> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.b.service 2024-03-21T09:07:59.381 DEBUG:teuthology.orchestra.run.smithi053:mds.e> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.e.service 2024-03-21T09:07:59.424 DEBUG:teuthology.orchestra.run.smithi053:mds.h> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.h.service 2024-03-21T09:07:59.467 DEBUG:teuthology.orchestra.run.smithi053:mds.k> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.k.service 2024-03-21T09:07:59.510 DEBUG:teuthology.orchestra.run.smithi071:mds.c> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.c.service 2024-03-21T09:07:59.513 DEBUG:teuthology.orchestra.run.smithi071:mds.f> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.f.service 2024-03-21T09:07:59.557 DEBUG:teuthology.orchestra.run.smithi071:mds.i> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.i.service 2024-03-21T09:07:59.600 DEBUG:teuthology.orchestra.run.smithi071:mds.l> sudo journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.l.service 2024-03-21T09:07:59.643 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-21T09:07:59.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:07:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:59 smithi053 ceph-mon[36169]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:59 smithi053 ceph-mon[36169]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:07:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:59 smithi053 ceph-mon[36169]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-03-21T09:07:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:07:59 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:59.723 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:59 smithi037 ceph-mon[31830]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:59.723 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:59 smithi037 ceph-mon[31830]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:07:59.723 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:59 smithi037 ceph-mon[31830]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-03-21T09:07:59.723 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:59.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:07:59 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:59 smithi071 ceph-mon[36597]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:07:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:59 smithi071 ceph-mon[36597]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:07:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:59 smithi071 ceph-mon[36597]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-03-21T09:07:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:07:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:07:59 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 -- 172.21.15.37:0/918801326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 msgr2=0x7fcaf4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/918801326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf4109230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009900 tx=0x7fcae002bc30 comp rx=0 tx=0).stop 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 -- 172.21.15.37:0/918801326 shutdown_connections 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/918801326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4109770 0x7fcaf4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/918801326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf4109230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/918801326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaf4107bc0 0x7fcaf4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 -- 172.21.15.37:0/918801326 >> 172.21.15.37:0/918801326 conn(0x7fcaf4076b60 msgr2=0x7fcaf4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:07:59.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 -- 172.21.15.37:0/918801326 shutdown_connections 2024-03-21T09:07:59.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.876+0000 7fcaf9a00640 1 -- 172.21.15.37:0/918801326 wait complete. 2024-03-21T09:07:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.877+0000 7fcaf9a00640 1 Processor -- start 2024-03-21T09:07:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.877+0000 7fcaf9a00640 1 -- start start 2024-03-21T09:07:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4107bc0 0x7fcaf419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaf4109770 0x7fcaf41a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4113890 con 0x7fcaf4109770 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4113710 con 0x7fcaf4107bc0 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4113a10 con 0x7fcaf4108db0 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf419e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50438/0 (socket says 172.21.15.37:50438) 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 -- 172.21.15.37:0/402496742 learned_addr learned my addr 172.21.15.37:0/402496742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3e00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4107bc0 0x7fcaf419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf9000640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaf4109770 0x7fcaf41a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 -- 172.21.15.37:0/402496742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4107bc0 msgr2=0x7fcaf419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4107bc0 0x7fcaf419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 -- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaf4109770 msgr2=0x7fcaf41a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaf4109770 0x7fcaf41a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3400640 1 -- 172.21.15.37:0/402496742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf41a5a90 con 0x7fcaf4108db0 2024-03-21T09:07:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.878+0000 7fcaf3e00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4107bc0 0x7fcaf419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:07:59.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.879+0000 7fcaf3400640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf419e3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005ab0 tx=0x7fcae002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:59.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.879+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0043020 con 0x7fcaf4108db0 2024-03-21T09:07:59.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.879+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae00054e0 con 0x7fcaf4108db0 2024-03-21T09:07:59.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.879+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf41a5d20 con 0x7fcaf4108db0 2024-03-21T09:07:59.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.879+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0003d70 con 0x7fcaf4108db0 2024-03-21T09:07:59.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.879+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41a6210 con 0x7fcaf4108db0 2024-03-21T09:07:59.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.880+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf410ee80 con 0x7fcaf4108db0 2024-03-21T09:07:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.883+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcae0046020 con 0x7fcaf4108db0 2024-03-21T09:07:59.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.884+0000 7fcaf0c00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcad80a2f10 0x7fcad80a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:07:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.885+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcae00f1240 con 0x7fcaf4108db0 2024-03-21T09:07:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.885+0000 7fcaf3e00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcad80a2f10 0x7fcad80a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:07:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.886+0000 7fcaf3e00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcad80a2f10 0x7fcad80a53d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcae4005d00 tx=0x7fcae4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:07:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:07:59.888+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcae004d020 con 0x7fcaf4108db0 2024-03-21T09:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.098+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaf40622a0 con 0x7fcaf4108db0 2024-03-21T09:08:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.100+0000 7fcaf0c00640 1 -- 172.21.15.37:0/402496742 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18370 (secure 0 0 0) 0x7fcae00be470 con 0x7fcaf4108db0 2024-03-21T09:08:00.094 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:00.094 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:07:49.100221+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:08:00.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.102+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcad80a2f10 msgr2=0x7fcad80a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.102+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcad80a2f10 0x7fcad80a53d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcae4005d00 tx=0x7fcae4005c90 comp rx=0 tx=0).stop 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.102+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 msgr2=0x7fcaf419e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.102+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf419e3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005ab0 tx=0x7fcae002c6c0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 shutdown_connections 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcad80a2f10 0x7fcad80a53d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcaf4109770 0x7fcaf41a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaf4108db0 0x7fcaf419e3b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 --2- 172.21.15.37:0/402496742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4107bc0 0x7fcaf419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 >> 172.21.15.37:0/402496742 conn(0x7fcaf4076b60 msgr2=0x7fcaf410c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 shutdown_connections 2024-03-21T09:08:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.103+0000 7fcaf9a00640 1 -- 172.21.15.37:0/402496742 wait complete. 2024-03-21T09:08:00.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T09:08:00.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:08:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 -- 172.21.15.37:0/855858099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270108cf0 msgr2=0x7f8270109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 --2- 172.21.15.37:0/855858099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270108cf0 0x7f8270109170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f825c009860 tx=0x7f825c02bb70 comp rx=0 tx=0).stop 2024-03-21T09:08:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 -- 172.21.15.37:0/855858099 shutdown_connections 2024-03-21T09:08:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 --2- 172.21.15.37:0/855858099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f827010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 --2- 172.21.15.37:0/855858099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270108cf0 0x7f8270109170 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 --2- 172.21.15.37:0/855858099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270107b00 0x7f8270107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 -- 172.21.15.37:0/855858099 >> 172.21.15.37:0/855858099 conn(0x7f8270076aa0 msgr2=0x7f8270078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:00.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 -- 172.21.15.37:0/855858099 shutdown_connections 2024-03-21T09:08:00.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.354+0000 7f8274a00640 1 -- 172.21.15.37:0/855858099 wait complete. 2024-03-21T09:08:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.355+0000 7f8274a00640 1 Processor -- start 2024-03-21T09:08:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.355+0000 7f8274a00640 1 -- start start 2024-03-21T09:08:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.355+0000 7f8274a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270107b00 0x7f827019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.355+0000 7f8274a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270108cf0 0x7f827019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f8274a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f82701a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f8274a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8270113760 con 0x7f8270108cf0 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f8274a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701135e0 con 0x7f8270107b00 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f8274a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701138e0 con 0x7f82701096b0 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270108cf0 0x7f827019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270107b00 0x7f827019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f82701a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270107b00 0x7f827019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33442/0 (socket says 172.21.15.37:33442) 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826e800640 1 -- 172.21.15.37:0/2441502079 learned_addr learned my addr 172.21.15.37:0/2441502079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f82701a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50454/0 (socket says 172.21.15.37:50454) 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826de00640 1 -- 172.21.15.37:0/2441502079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 msgr2=0x7f82701a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826de00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f82701a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826de00640 1 -- 172.21.15.37:0/2441502079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270107b00 msgr2=0x7f827019de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826de00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270107b00 0x7f827019de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826de00640 1 -- 172.21.15.37:0/2441502079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82701a5a50 con 0x7f8270108cf0 2024-03-21T09:08:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.356+0000 7f826f200640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f82701a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.357+0000 7f826de00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270108cf0 0x7f827019e370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f825c02c0b0 tx=0x7f825c032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.357+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c041020 con 0x7f8270108cf0 2024-03-21T09:08:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.357+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f825c02c9f0 con 0x7f8270108cf0 2024-03-21T09:08:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.357+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82701a5ce0 con 0x7f8270108cf0 2024-03-21T09:08:00.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.357+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c039040 con 0x7f8270108cf0 2024-03-21T09:08:00.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.357+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701a61a0 con 0x7f8270108cf0 2024-03-21T09:08:00.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.358+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827010ee70 con 0x7f8270108cf0 2024-03-21T09:08:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.360+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f825c044020 con 0x7f8270108cf0 2024-03-21T09:08:00.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.361+0000 7f8267400640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f82540a2f20 0x7f82540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.362+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f825c0f0780 con 0x7f8270108cf0 2024-03-21T09:08:00.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.362+0000 7f826e800640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f82540a2f20 0x7f82540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.362+0000 7f826e800640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f82540a2f20 0x7f82540a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8260009c60 tx=0x7f8260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:00.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.366+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f825c04b020 con 0x7f8270108cf0 2024-03-21T09:08:00.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.566+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82700622d0 con 0x7f8270108cf0 2024-03-21T09:08:00.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.567+0000 7f8267400640 1 -- 172.21.15.37:0/2441502079 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f825c0b7e20 con 0x7f8270108cf0 2024-03-21T09:08:00.561 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:08:00.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f82540a2f20 msgr2=0x7f82540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f82540a2f20 0x7f82540a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8260009c60 tx=0x7f8260009260 comp rx=0 tx=0).stop 2024-03-21T09:08:00.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270108cf0 msgr2=0x7f827019e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270108cf0 0x7f827019e370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f825c02c0b0 tx=0x7f825c032430 comp rx=0 tx=0).stop 2024-03-21T09:08:00.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 shutdown_connections 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f82540a2f20 0x7f82540a53e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82701096b0 0x7f82701a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8270108cf0 0x7f827019e370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 --2- 172.21.15.37:0/2441502079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270107b00 0x7f827019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.570+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 >> 172.21.15.37:0/2441502079 conn(0x7f8270076aa0 msgr2=0x7f827010c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.571+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 shutdown_connections 2024-03-21T09:08:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.571+0000 7f8274a00640 1 -- 172.21.15.37:0/2441502079 wait complete. 2024-03-21T09:08:00.586 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:00.745 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:00.745 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:00.746 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:00.746 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:00.746 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:00.746 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:00.746 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/402496742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:00.746 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2441502079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:08:00.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 -- 172.21.15.37:0/1701301407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 msgr2=0x7feed4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 --2- 172.21.15.37:0/1701301407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 0x7feed4104f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec0009900 tx=0x7feec002bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:00.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 -- 172.21.15.37:0/1701301407 shutdown_connections 2024-03-21T09:08:00.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 --2- 172.21.15.37:0/1701301407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 0x7feed410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 --2- 172.21.15.37:0/1701301407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 0x7feed4104f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 --2- 172.21.15.37:0/1701301407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 -- 172.21.15.37:0/1701301407 >> 172.21.15.37:0/1701301407 conn(0x7feed40fee90 msgr2=0x7feed41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 -- 172.21.15.37:0/1701301407 shutdown_connections 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.829+0000 7feedc000640 1 -- 172.21.15.37:0/1701301407 wait complete. 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 Processor -- start 2024-03-21T09:08:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 -- start start 2024-03-21T09:08:00.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 0x7feed4079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 0x7feed4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed410f540 con 0x7feed41038e0 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed410f3c0 con 0x7feed4105490 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.831+0000 7feedc000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed410f6c0 con 0x7feed4104ad0 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feed9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feed8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 0x7feed4079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feed9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51326/0 (socket says 172.21.15.37:51326) 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feed9600640 1 -- 172.21.15.37:0/2478379127 learned_addr learned my addr 172.21.15.37:0/2478379127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 0x7feed4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 -- 172.21.15.37:0/2478379127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 msgr2=0x7feed4079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 0x7feed4079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 -- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 msgr2=0x7feed4079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 -- 172.21.15.37:0/2478379127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4076880 con 0x7feed4105490 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feed9600640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.832+0000 7feeda000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 0x7feed4075fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e940 tx=0x7feecc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.833+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc004d20 con 0x7feed4105490 2024-03-21T09:08:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.833+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feecc005020 con 0x7feed4105490 2024-03-21T09:08:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.833+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed41a6000 con 0x7feed4105490 2024-03-21T09:08:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.833+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0116b0 con 0x7feed4105490 2024-03-21T09:08:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.833+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed41a63b0 con 0x7feed4105490 2024-03-21T09:08:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.834+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed4068970 con 0x7feed4105490 2024-03-21T09:08:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.836+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7feecc0054d0 con 0x7feed4105490 2024-03-21T09:08:00.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.837+0000 7feed2000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feeb80a2f20 0x7feeb80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:00.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.837+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feecc0cefe0 con 0x7feed4105490 2024-03-21T09:08:00.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.837+0000 7feed9600640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feeb80a2f20 0x7feeb80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:00.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.838+0000 7feed9600640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feeb80a2f20 0x7feeb80a53e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feec40058c0 tx=0x7feec4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:00.842+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feecc016030 con 0x7feed4105490 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/402496742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:00.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2441502079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:08:00.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:00.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:00.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:00.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:00.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:00.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:00.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/402496742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:00.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2441502079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:08:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.094+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feed40622a0 con 0x7feed4105490 2024-03-21T09:08:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.095+0000 7feed2000640 1 -- 172.21.15.37:0/2478379127 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7feecc09bbe0 con 0x7feed4105490 2024-03-21T09:08:01.089 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:01.089 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-21T09:08:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 1 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feeb80a2f20 msgr2=0x7feeb80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feeb80a2f20 0x7feeb80a53e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feec40058c0 tx=0x7feec4005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 msgr2=0x7feed4075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 0x7feed4075fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e940 tx=0x7feecc00ee00 comp rx=0 tx=0).stop 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 shutdown_connections 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7feeb80a2f20 0x7feeb80a53e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4105490 0x7feed4075fc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feed4104ad0 0x7feed4079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 --2- 172.21.15.37:0/2478379127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feed41038e0 0x7feed4079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 >> 172.21.15.37:0/2478379127 conn(0x7feed40fee90 msgr2=0x7feed4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 shutdown_connections 2024-03-21T09:08:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:01.098+0000 7feedc000640 1 -- 172.21.15.37:0/2478379127 wait complete. 2024-03-21T09:08:01.832 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:01 smithi071 ceph-mon[36597]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:01.832 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:01 smithi071 ceph-mon[36597]: Deploying daemon mds.l on smithi071 2024-03-21T09:08:01.832 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:01 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2478379127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:01.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:01 smithi037 ceph-mon[31830]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:01.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:01 smithi037 ceph-mon[31830]: Deploying daemon mds.l on smithi071 2024-03-21T09:08:01.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:01 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2478379127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:01 smithi053 ceph-mon[36169]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:01 smithi053 ceph-mon[36169]: Deploying daemon mds.l on smithi071 2024-03-21T09:08:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:01 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2478379127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:02.852 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:08:02 smithi071 systemd[1]: Started Ceph mds.l for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:03.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.354+0000 7fa8d3200640 1 -- 172.21.15.37:0/1222746465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d41049b0 msgr2=0x7fa8d4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.354+0000 7fa8d3200640 1 --2- 172.21.15.37:0/1222746465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d41049b0 0x7fa8d4104e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c002bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 -- 172.21.15.37:0/1222746465 shutdown_connections 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 --2- 172.21.15.37:0/1222746465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d4105370 0x7fa8d410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 --2- 172.21.15.37:0/1222746465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d41049b0 0x7fa8d4104e30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 --2- 172.21.15.37:0/1222746465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d41037c0 0x7fa8d4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 -- 172.21.15.37:0/1222746465 >> 172.21.15.37:0/1222746465 conn(0x7fa8d40fed50 msgr2=0x7fa8d4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 -- 172.21.15.37:0/1222746465 shutdown_connections 2024-03-21T09:08:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.355+0000 7fa8d3200640 1 -- 172.21.15.37:0/1222746465 wait complete. 2024-03-21T09:08:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.356+0000 7fa8d3200640 1 Processor -- start 2024-03-21T09:08:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.356+0000 7fa8d3200640 1 -- start start 2024-03-21T09:08:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 0x7fa8d4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d41049b0 0x7fa8d4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d4105370 0x7fa8d41a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d410f470 con 0x7fa8d41037c0 2024-03-21T09:08:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d410f2f0 con 0x7fa8d4105370 2024-03-21T09:08:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d410f5f0 con 0x7fa8d41049b0 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 0x7fa8d4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 0x7fa8d4199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51344/0 (socket says 172.21.15.37:51344) 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 -- 172.21.15.37:0/2299285859 learned_addr learned my addr 172.21.15.37:0/2299285859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1400640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d41049b0 0x7fa8d4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d2800640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d4105370 0x7fa8d41a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 -- 172.21.15.37:0/2299285859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d41049b0 msgr2=0x7fa8d4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d41049b0 0x7fa8d4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 -- 172.21.15.37:0/2299285859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d4105370 msgr2=0x7fa8d41a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d4105370 0x7fa8d41a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.357+0000 7fa8d1e00640 1 -- 172.21.15.37:0/2299285859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d41a16d0 con 0x7fa8d41037c0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.358+0000 7fa8d1e00640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 0x7fa8d4199ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e9c0 tx=0x7fa8c400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.358+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400ccc0 con 0x7fa8d41037c0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.358+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8c40035f0 con 0x7fa8d41037c0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.358+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d41a19c0 con 0x7fa8d41037c0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.358+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400ce60 con 0x7fa8d41037c0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.358+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d41a1ee0 con 0x7fa8d41037c0 2024-03-21T09:08:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.359+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d404f470 con 0x7fa8d41037c0 2024-03-21T09:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.361+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa8c40037a0 con 0x7fa8d41037c0 2024-03-21T09:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.363+0000 7fa8caa00640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8b80a2f10 0x7fa8b80a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.363+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8c40cf310 con 0x7fa8d41037c0 2024-03-21T09:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.363+0000 7fa8d1400640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8b80a2f10 0x7fa8b80a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.365+0000 7fa8d1400640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8b80a2f10 0x7fa8b80a53d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8c002c240 tx=0x7fa8c002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.367+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa8c409c540 con 0x7fa8d41037c0 2024-03-21T09:08:03.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.616+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8d410aae0 con 0x7fa8d41037c0 2024-03-21T09:08:03.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.617+0000 7fa8caa00640 1 -- 172.21.15.37:0/2299285859 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fa8c409be40 con 0x7fa8d41037c0 2024-03-21T09:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T09:08:03.612 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8b80a2f10 msgr2=0x7fa8b80a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8b80a2f10 0x7fa8b80a53d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8c002c240 tx=0x7fa8c002df90 comp rx=0 tx=0).stop 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 msgr2=0x7fa8d4199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 0x7fa8d4199ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e9c0 tx=0x7fa8c400edc0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 shutdown_connections 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8b80a2f10 0x7fa8b80a53d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d4105370 0x7fa8d41a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d41049b0 0x7fa8d4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 --2- 172.21.15.37:0/2299285859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8d41037c0 0x7fa8d4199ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:03.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 >> 172.21.15.37:0/2299285859 conn(0x7fa8d40fed50 msgr2=0x7fa8d4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:03.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 shutdown_connections 2024-03-21T09:08:03.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:03.621+0000 7fa8d3200640 1 -- 172.21.15.37:0/2299285859 wait complete. 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6832/3927519167,v1:172.21.15.71:6833/3927519167] up:boot 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: fsmap 1 up:standby 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:03.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:03.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:03 smithi053 ceph-mon[36169]: Deploying daemon mds.k on smithi053 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6832/3927519167,v1:172.21.15.71:6833/3927519167] up:boot 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: fsmap 1 up:standby 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:03.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:03 smithi071 ceph-mon[36597]: Deploying daemon mds.k on smithi053 2024-03-21T09:08:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6832/3927519167,v1:172.21.15.71:6833/3927519167] up:boot 2024-03-21T09:08:03.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: fsmap 1 up:standby 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:03.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:03 smithi037 ceph-mon[31830]: Deploying daemon mds.k on smithi053 2024-03-21T09:08:04.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:04 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2299285859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:04.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:04 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2299285859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:04.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:04 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2299285859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:05.191 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 systemd[1]: Starting Ceph mds.k for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:05.519 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 podman[65879]: 2024-03-21 09:08:05.176417072 +0000 UTC m=+0.018520217 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:05.519 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 podman[65879]: 2024-03-21 09:08:05.286631369 +0000 UTC m=+0.128734499 container create f4cb4bb953a1920be476b22f56d7dbe6b968f9ae815cc16fd4ee426931ee891c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T09:08:05.520 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 podman[65879]: 2024-03-21 09:08:05.52313527 +0000 UTC m=+0.365238421 container init f4cb4bb953a1920be476b22f56d7dbe6b968f9ae815cc16fd4ee426931ee891c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:08:05.520 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 podman[65879]: 2024-03-21 09:08:05.525949563 +0000 UTC m=+0.368052693 container start f4cb4bb953a1920be476b22f56d7dbe6b968f9ae815cc16fd4ee426931ee891c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True) 2024-03-21T09:08:05.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:05.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:05 smithi071 ceph-mon[36597]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 -- 172.21.15.37:0/240958479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee80756e0 msgr2=0x7faee8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 --2- 172.21.15.37:0/240958479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee80756e0 0x7faee8073b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faed8009860 tx=0x7faed802bb70 comp rx=0 tx=0).stop 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 -- 172.21.15.37:0/240958479 shutdown_connections 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 --2- 172.21.15.37:0/240958479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8109480 0x7faee810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 --2- 172.21.15.37:0/240958479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee8074100 0x7faee8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 --2- 172.21.15.37:0/240958479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee80756e0 0x7faee8073b30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 -- 172.21.15.37:0/240958479 >> 172.21.15.37:0/240958479 conn(0x7faee80fcb30 msgr2=0x7faee80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 -- 172.21.15.37:0/240958479 shutdown_connections 2024-03-21T09:08:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.879+0000 7faee7e00640 1 -- 172.21.15.37:0/240958479 wait complete. 2024-03-21T09:08:05.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.880+0000 7faee7e00640 1 Processor -- start 2024-03-21T09:08:05.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.880+0000 7faee7e00640 1 -- start start 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.880+0000 7faee7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 0x7faee8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.880+0000 7faee7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 0x7faee819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee8109480 0x7faee81a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810f420 con 0x7faee80756e0 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810f2a0 con 0x7faee8109480 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810f5a0 con 0x7faee8074100 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 0x7faee8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 0x7faee819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee8109480 0x7faee81a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 0x7faee8199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34716/0 (socket says 172.21.15.37:34716) 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6a00640 1 -- 172.21.15.37:0/3016816875 learned_addr learned my addr 172.21.15.37:0/3016816875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 0x7faee819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41440/0 (socket says 172.21.15.37:41440) 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7400640 1 -- 172.21.15.37:0/3016816875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 msgr2=0x7faee8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7400640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 0x7faee8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7400640 1 -- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 msgr2=0x7faee819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7400640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 0x7faee819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee7400640 1 -- 172.21.15.37:0/3016816875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee81a1710 con 0x7faee8109480 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6a00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 0x7faee8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.881+0000 7faee6000640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 0x7faee819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.882+0000 7faee7400640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee8109480 0x7faee81a10a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faee0009fd0 tx=0x7faee000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.882+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000ce20 con 0x7faee8109480 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.882+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faee0005020 con 0x7faee8109480 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.882+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee81a1a00 con 0x7faee8109480 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.882+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0010720 con 0x7faee8109480 2024-03-21T09:08:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.882+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee81a1fa0 con 0x7faee8109480 2024-03-21T09:08:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.883+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee81081b0 con 0x7faee8109480 2024-03-21T09:08:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.885+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faee0005540 con 0x7faee8109480 2024-03-21T09:08:05.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.886+0000 7faedf400640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faecc0a2f90 0x7faecc0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:05.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.886+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faee00cf540 con 0x7faee8109480 2024-03-21T09:08:05.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.886+0000 7faee6a00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faecc0a2f90 0x7faecc0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:05.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.887+0000 7faee6a00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faecc0a2f90 0x7faecc0a5450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faed8035fd0 tx=0x7faed802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:05.890+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faee009c770 con 0x7faee8109480 2024-03-21T09:08:05.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:05 smithi037 ceph-mon[31830]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:05.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:05 smithi053 ceph-mon[36169]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:05.941 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 ceph-mds[65900]: -- 172.21.15.53:0/1743615772 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559617d6da40 con 0x559617d41000 2024-03-21T09:08:05.941 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k[65896]: starting mds.k at 2024-03-21T09:08:05.941 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 bash[65879]: f4cb4bb953a1920be476b22f56d7dbe6b968f9ae815cc16fd4ee426931ee891c 2024-03-21T09:08:05.942 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:08:05 smithi053 systemd[1]: Started Ceph mds.k for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:06.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.154+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faee802d070 con 0x7faee8109480 2024-03-21T09:08:06.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.155+0000 7faedf400640 1 -- 172.21.15.37:0/3016816875 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7faee009c070 con 0x7faee8109480 2024-03-21T09:08:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:06.149 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T09:08:06.149 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-03-21T09:08:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faecc0a2f90 msgr2=0x7faecc0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faecc0a2f90 0x7faecc0a5450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faed8035fd0 tx=0x7faed802e040 comp rx=0 tx=0).stop 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee8109480 msgr2=0x7faee81a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee8109480 0x7faee81a10a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faee0009fd0 tx=0x7faee000ef10 comp rx=0 tx=0).stop 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 shutdown_connections 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faecc0a2f90 0x7faecc0a5450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee8109480 0x7faee81a10a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee80756e0 0x7faee819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 --2- 172.21.15.37:0/3016816875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee8074100 0x7faee8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 >> 172.21.15.37:0/3016816875 conn(0x7faee80fcb30 msgr2=0x7faee8105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 shutdown_connections 2024-03-21T09:08:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:06.158+0000 7faee7e00640 1 -- 172.21.15.37:0/3016816875 wait complete. 2024-03-21T09:08:06.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3016816875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6832/1353662837,v1:172.21.15.53:6833/1353662837] up:boot 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: fsmap 2 up:standby 2024-03-21T09:08:06.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:06 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:08:06.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:06.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3016816875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6832/1353662837,v1:172.21.15.53:6833/1353662837] up:boot 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: fsmap 2 up:standby 2024-03-21T09:08:06.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:06 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3016816875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6832/1353662837,v1:172.21.15.53:6833/1353662837] up:boot 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: fsmap 2 up:standby 2024-03-21T09:08:07.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:06 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:08:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:07 smithi037 ceph-mon[31830]: Deploying daemon mds.j on smithi037 2024-03-21T09:08:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:07 smithi037 ceph-mon[31830]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:07.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:07 smithi053 ceph-mon[36169]: Deploying daemon mds.j on smithi037 2024-03-21T09:08:07.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:07 smithi053 ceph-mon[36169]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:08.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:07 smithi071 ceph-mon[36597]: Deploying daemon mds.j on smithi037 2024-03-21T09:08:08.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:07 smithi071 ceph-mon[36597]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:08.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 -- 172.21.15.37:0/2127773076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 msgr2=0x7f3a280ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 --2- 172.21.15.37:0/2127773076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a280ffa00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009ca0 tx=0x7f3a2002c100 comp rx=0 tx=0).stop 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 -- 172.21.15.37:0/2127773076 shutdown_connections 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 --2- 172.21.15.37:0/2127773076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a281010b0 0x7f3a28105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 --2- 172.21.15.37:0/2127773076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a28100710 0x7f3a28100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 --2- 172.21.15.37:0/2127773076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a280ffa00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 -- 172.21.15.37:0/2127773076 >> 172.21.15.37:0/2127773076 conn(0x7f3a280faec0 msgr2=0x7f3a280fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.472+0000 7f3a2d000640 1 -- 172.21.15.37:0/2127773076 shutdown_connections 2024-03-21T09:08:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.473+0000 7f3a2d000640 1 -- 172.21.15.37:0/2127773076 wait complete. 2024-03-21T09:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.473+0000 7f3a2d000640 1 Processor -- start 2024-03-21T09:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 -- start start 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a2819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a28100710 0x7f3a2819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a281010b0 0x7f3a281a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28113910 con 0x7f3a28100710 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28113790 con 0x7f3a280ff5e0 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a2d000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28113a90 con 0x7f3a281010b0 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a25e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a2819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a25e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a2819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:34336/0 (socket says 172.21.15.37:34336) 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a25e00640 1 -- 172.21.15.37:0/295679035 learned_addr learned my addr 172.21.15.37:0/295679035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.474+0000 7f3a25400640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a28100710 0x7f3a2819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a26800640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a281010b0 0x7f3a281a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25e00640 1 -- 172.21.15.37:0/295679035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a281010b0 msgr2=0x7f3a281a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25e00640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a281010b0 0x7f3a281a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25e00640 1 -- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a28100710 msgr2=0x7f3a2819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25e00640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a28100710 0x7f3a2819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25e00640 1 -- 172.21.15.37:0/295679035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a281a5be0 con 0x7f3a280ff5e0 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25400640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a28100710 0x7f3a2819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a25e00640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a2819e010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a200325f0 tx=0x7f3a20035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20039040 con 0x7f3a280ff5e0 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.475+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a20044070 con 0x7f3a280ff5e0 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.476+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a281a5e70 con 0x7f3a280ff5e0 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.476+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20004610 con 0x7f3a280ff5e0 2024-03-21T09:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.476+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a281a62d0 con 0x7f3a280ff5e0 2024-03-21T09:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.477+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a28104780 con 0x7f3a280ff5e0 2024-03-21T09:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.480+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3a20003970 con 0x7f3a280ff5e0 2024-03-21T09:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.482+0000 7f3a1ea00640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3a0c0a2fa0 0x7f3a0c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.482+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a200f1730 con 0x7f3a280ff5e0 2024-03-21T09:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.482+0000 7f3a25400640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3a0c0a2fa0 0x7f3a0c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.483+0000 7f3a25400640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3a0c0a2fa0 0x7f3a0c0a5460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005d00 tx=0x7f3a18005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.486+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a200be960 con 0x7f3a280ff5e0 2024-03-21T09:08:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.752+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a28062370 con 0x7f3a280ff5e0 2024-03-21T09:08:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.753+0000 7f3a1ea00640 1 -- 172.21.15.37:0/295679035 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f3a200be260 con 0x7f3a280ff5e0 2024-03-21T09:08:08.747 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:08.747 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T09:08:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-03-21T09:08:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3a0c0a2fa0 msgr2=0x7f3a0c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3a0c0a2fa0 0x7f3a0c0a5460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005d00 tx=0x7f3a18005c70 comp rx=0 tx=0).stop 2024-03-21T09:08:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 msgr2=0x7f3a2819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a2819e010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a200325f0 tx=0x7f3a20035d10 comp rx=0 tx=0).stop 2024-03-21T09:08:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 shutdown_connections 2024-03-21T09:08:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3a0c0a2fa0 0x7f3a0c0a5460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a281010b0 0x7f3a281a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a28100710 0x7f3a2819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 --2- 172.21.15.37:0/295679035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a280ff5e0 0x7f3a2819e010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 >> 172.21.15.37:0/295679035 conn(0x7f3a280faec0 msgr2=0x7f3a28101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 shutdown_connections 2024-03-21T09:08:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:08.755+0000 7f3a2d000640 1 -- 172.21.15.37:0/295679035 wait complete. 2024-03-21T09:08:09.329 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 podman[69266]: 2024-03-21 09:08:09.158953703 +0000 UTC m=+0.215051574 container create 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T09:08:09.690 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 podman[69266]: 2024-03-21 09:08:09.441411691 +0000 UTC m=+0.497509572 container init 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-21T09:08:09.690 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 podman[69266]: 2024-03-21 09:08:09.452650699 +0000 UTC m=+0.508748610 container start 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T09:08:09.690 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 ceph-mds[69286]: -- 172.21.15.37:0/886467847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6875e7a40 con 0x55b6875ba800 2024-03-21T09:08:09.690 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j[69282]: starting mds.j at 2024-03-21T09:08:09.690 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 bash[69266]: 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f 2024-03-21T09:08:09.690 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:08:09 smithi037 systemd[1]: Started Ceph mds.j for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:09.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:09 smithi053 ceph-mon[36169]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:09.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:09 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/295679035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:09.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:09.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:09 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:09 smithi071 ceph-mon[36597]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:10.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:09 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/295679035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:10.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:09 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:09 smithi037 ceph-mon[31830]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:09 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/295679035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:09 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.770 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:10.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.37:6834/3418620449,v1:172.21.15.37:6835/3418620449] up:boot 2024-03-21T09:08:10.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: fsmap 3 up:standby 2024-03-21T09:08:10.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T09:08:10.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:10.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:10.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:10 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.37:6834/3418620449,v1:172.21.15.37:6835/3418620449] up:boot 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: fsmap 3 up:standby 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:10.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:10 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.004+0000 7f6449800640 1 -- 172.21.15.37:0/2771687721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6444074100 msgr2=0x7f6444074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.004+0000 7f6449800640 1 --2- 172.21.15.37:0/2771687721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6444074100 0x7f6444074580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f642c009930 tx=0x7f642c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 -- 172.21.15.37:0/2771687721 shutdown_connections 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 --2- 172.21.15.37:0/2771687721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644410db90 0x7f644410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 --2- 172.21.15.37:0/2771687721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6444074100 0x7f6444074580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 --2- 172.21.15.37:0/2771687721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64440756e0 0x7f6444073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 -- 172.21.15.37:0/2771687721 >> 172.21.15.37:0/2771687721 conn(0x7f64440fcb30 msgr2=0x7f64440fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 -- 172.21.15.37:0/2771687721 shutdown_connections 2024-03-21T09:08:10.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.005+0000 7f6449800640 1 -- 172.21.15.37:0/2771687721 wait complete. 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 Processor -- start 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 -- start start 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 0x7f644419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64440756e0 0x7f644419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f644410db90 0x7f64441a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444113880 con 0x7f644410db90 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444113700 con 0x7f6444074100 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.006+0000 7f6449800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444113a00 con 0x7f64440756e0 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6448e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f644410db90 0x7f64441a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6443e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 0x7f644419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6443e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 0x7f644419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:34376/0 (socket says 172.21.15.37:34376) 2024-03-21T09:08:11.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6443e00640 1 -- 172.21.15.37:0/4163995960 learned_addr learned my addr 172.21.15.37:0/4163995960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:11.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6443400640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64440756e0 0x7f644419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6448e00640 1 -- 172.21.15.37:0/4163995960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64440756e0 msgr2=0x7f644419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6448e00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64440756e0 0x7f644419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6448e00640 1 -- 172.21.15.37:0/4163995960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 msgr2=0x7f644419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6448e00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 0x7f644419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6448e00640 1 -- 172.21.15.37:0/4163995960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64441a5b20 con 0x7f644410db90 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.007+0000 7f6443e00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 0x7f644419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.008+0000 7f6448e00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f644410db90 0x7f64441a54b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f643c00e970 tx=0x7f643c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.008+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00ccc0 con 0x7f644410db90 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.008+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f643c005020 con 0x7f644410db90 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.008+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c004a00 con 0x7f644410db90 2024-03-21T09:08:11.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.008+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64441a5e10 con 0x7f644410db90 2024-03-21T09:08:11.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.008+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64441a6220 con 0x7f644410db90 2024-03-21T09:08:11.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.010+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6444103d60 con 0x7f644410db90 2024-03-21T09:08:11.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.012+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f643c0051d0 con 0x7f644410db90 2024-03-21T09:08:11.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.013+0000 7f6440c00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64280a2f10 0x7f64280a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:11.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.013+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f643c0cefd0 con 0x7f644410db90 2024-03-21T09:08:11.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.014+0000 7f6443e00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64280a2f10 0x7f64280a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:11.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.014+0000 7f6443e00640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64280a2f10 0x7f64280a53d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f643400f7d0 tx=0x7f6434009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:11.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.016+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f643c09c290 con 0x7f644410db90 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.37:6834/3418620449,v1:172.21.15.37:6835/3418620449] up:boot 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: fsmap 3 up:standby 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:11.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:10 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.268+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f644404f470 con 0x7f644410db90 2024-03-21T09:08:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.269+0000 7f6440c00640 1 -- 172.21.15.37:0/4163995960 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7f643c09bb90 con 0x7f644410db90 2024-03-21T09:08:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:11.264 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T09:08:11.264 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-03-21T09:08:11.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64280a2f10 msgr2=0x7f64280a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:11.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64280a2f10 0x7f64280a53d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f643400f7d0 tx=0x7f6434009500 comp rx=0 tx=0).stop 2024-03-21T09:08:11.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f644410db90 msgr2=0x7f64441a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:11.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f644410db90 0x7f64441a54b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f643c00e970 tx=0x7f643c00ee30 comp rx=0 tx=0).stop 2024-03-21T09:08:11.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 shutdown_connections 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64280a2f10 0x7f64280a53d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f644410db90 0x7f64441a54b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64440756e0 0x7f644419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 --2- 172.21.15.37:0/4163995960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6444074100 0x7f644419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 >> 172.21.15.37:0/4163995960 conn(0x7f64440fcb30 msgr2=0x7f6444100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.272+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 shutdown_connections 2024-03-21T09:08:11.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:11.273+0000 7f6449800640 1 -- 172.21.15.37:0/4163995960 wait complete. 2024-03-21T09:08:12.044 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:11 smithi071 ceph-mon[36597]: Deploying daemon mds.d on smithi037 2024-03-21T09:08:12.045 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:11 smithi071 ceph-mon[36597]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:12.045 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:11 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4163995960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:12.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:11 smithi037 ceph-mon[31830]: Deploying daemon mds.d on smithi037 2024-03-21T09:08:12.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:11 smithi037 ceph-mon[31830]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:12.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:11 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4163995960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:12.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:11 smithi053 ceph-mon[36169]: Deploying daemon mds.d on smithi037 2024-03-21T09:08:12.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:11 smithi053 ceph-mon[36169]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:12.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:11 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4163995960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:13.173 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:12 smithi037 systemd[1]: Starting Ceph mds.d for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:13.289 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.524+0000 7f595aa00640 1 -- 172.21.15.37:0/1046407240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59540772a0 msgr2=0x7f5954077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.524+0000 7f595aa00640 1 --2- 172.21.15.37:0/1046407240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59540772a0 0x7f5954077700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5940009860 tx=0x7f594002bb90 comp rx=0 tx=0).stop 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 -- 172.21.15.37:0/1046407240 shutdown_connections 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 --2- 172.21.15.37:0/1046407240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5954077c40 0x7f59540784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 --2- 172.21.15.37:0/1046407240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59540772a0 0x7f5954077700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 --2- 172.21.15.37:0/1046407240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f5954076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 -- 172.21.15.37:0/1046407240 >> 172.21.15.37:0/1046407240 conn(0x7f59540fed50 msgr2=0x7f5954101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 -- 172.21.15.37:0/1046407240 shutdown_connections 2024-03-21T09:08:13.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.525+0000 7f595aa00640 1 -- 172.21.15.37:0/1046407240 wait complete. 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.526+0000 7f595aa00640 1 Processor -- start 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.526+0000 7f595aa00640 1 -- start start 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.526+0000 7f595aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f595419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59540772a0 0x7f595419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5954077c40 0x7f59541a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954113810 con 0x7f5954077c40 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954113690 con 0x7f5954076060 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954113990 con 0x7f59540772a0 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f595419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f595419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:34396/0 (socket says 172.21.15.37:34396) 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5954077c40 0x7f59541a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 -- 172.21.15.37:0/853320375 learned_addr learned my addr 172.21.15.37:0/853320375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5958c00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59540772a0 0x7f595419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 -- 172.21.15.37:0/853320375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59540772a0 msgr2=0x7f595419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59540772a0 0x7f595419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 -- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5954077c40 msgr2=0x7f59541a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5954077c40 0x7f59541a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f5959600640 1 -- 172.21.15.37:0/853320375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59541a5ad0 con 0x7f5954076060 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.527+0000 7f595a000640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5954077c40 0x7f59541a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.528+0000 7f5959600640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f595419deb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f594800e9f0 tx=0x7f594800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.528+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800cd80 con 0x7f5954076060 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.528+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5948010910 con 0x7f5954076060 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.528+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948018990 con 0x7f5954076060 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.528+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59541a5dc0 con 0x7f5954076060 2024-03-21T09:08:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.528+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59541a6280 con 0x7f5954076060 2024-03-21T09:08:13.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.529+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954110130 con 0x7f5954076060 2024-03-21T09:08:13.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.531+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5948010ac0 con 0x7f5954076060 2024-03-21T09:08:13.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.532+0000 7f5952000640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f593c0a2f90 0x7f593c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:13.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.533+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59480cf250 con 0x7f5954076060 2024-03-21T09:08:13.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.533+0000 7f5958c00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f593c0a2f90 0x7f593c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:13.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.534+0000 7f5958c00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f593c0a2f90 0x7f593c0a5450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f594002c1a0 tx=0x7f5940034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.537+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f594809be30 con 0x7f5954076060 2024-03-21T09:08:13.548 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 podman[69609]: 2024-03-21 09:08:13.202737689 +0000 UTC m=+0.033966498 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:13.548 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 podman[69609]: 2024-03-21 09:08:13.317164426 +0000 UTC m=+0.148393244 container create 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T09:08:13.548 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 podman[69609]: 2024-03-21 09:08:13.553765289 +0000 UTC m=+0.384994108 container init 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:08:13.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.796+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59540622d0 con 0x7f5954076060 2024-03-21T09:08:13.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.797+0000 7f5952000640 1 -- 172.21.15.37:0/853320375 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3695 (secure 0 0 0) 0x7f594809fed0 con 0x7f5954076060 2024-03-21T09:08:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:13.792 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T09:08:13.793 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-03-21T09:08:13.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f593c0a2f90 msgr2=0x7f593c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:13.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f593c0a2f90 0x7f593c0a5450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f594002c1a0 tx=0x7f5940034ad0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 msgr2=0x7f595419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f595419deb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f594800e9f0 tx=0x7f594800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 shutdown_connections 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f593c0a2f90 0x7f593c0a5450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5954077c40 0x7f59541a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59540772a0 0x7f595419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 --2- 172.21.15.37:0/853320375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5954076060 0x7f595419deb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 >> 172.21.15.37:0/853320375 conn(0x7f59540fed50 msgr2=0x7f5954100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.800+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 shutdown_connections 2024-03-21T09:08:13.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:13.801+0000 7f595aa00640 1 -- 172.21.15.37:0/853320375 wait complete. 2024-03-21T09:08:13.812 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:13 smithi037 ceph-mon[31830]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:13.812 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 podman[69609]: 2024-03-21 09:08:13.561258955 +0000 UTC m=+0.392487764 container start 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T09:08:13.812 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 ceph-mds[69667]: -- 172.21.15.37:0/1296342931 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557650f55a40 con 0x557650f28800 2024-03-21T09:08:13.812 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d[69644]: starting mds.d at 2024-03-21T09:08:13.813 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 bash[69609]: 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 2024-03-21T09:08:13.813 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:08:13 smithi037 systemd[1]: Started Ceph mds.d for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:14.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:13 smithi071 ceph-mon[36597]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:14.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:13 smithi053 ceph-mon[36169]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.37:6836/495294644,v1:172.21.15.37:6837/495294644] up:boot 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: fsmap 4 up:standby 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:14.862 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/853320375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:14.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:14.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:14.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:14.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:14.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:14 smithi053 ceph-mon[36169]: Deploying daemon mds.e on smithi053 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.37:6836/495294644,v1:172.21.15.37:6837/495294644] up:boot 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: fsmap 4 up:standby 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/853320375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:15.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:15.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:15.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:15.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:14 smithi071 ceph-mon[36597]: Deploying daemon mds.e on smithi053 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.37:6836/495294644,v1:172.21.15.37:6837/495294644] up:boot 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: fsmap 4 up:standby 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:15.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/853320375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:15.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:15.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:15.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:15.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:15.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:14 smithi037 ceph-mon[31830]: Deploying daemon mds.e on smithi053 2024-03-21T09:08:15.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 -- 172.21.15.37:0/3427091149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 msgr2=0x7fa100104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/3427091149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa100104e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009900 tx=0x7fa0e802bc40 comp rx=0 tx=0).stop 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 -- 172.21.15.37:0/3427091149 shutdown_connections 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/3427091149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100105370 0x7fa10010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/3427091149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa100104e30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/3427091149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1001037c0 0x7fa100103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 -- 172.21.15.37:0/3427091149 >> 172.21.15.37:0/3427091149 conn(0x7fa1000fed50 msgr2=0x7fa100101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 -- 172.21.15.37:0/3427091149 shutdown_connections 2024-03-21T09:08:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.062+0000 7fa0ffe00640 1 -- 172.21.15.37:0/3427091149 wait complete. 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.063+0000 7fa0ffe00640 1 Processor -- start 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.063+0000 7fa0ffe00640 1 -- start start 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 0x7fa100199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa10019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa100105370 0x7fa1001a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ffe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10010f440 con 0x7fa1001037c0 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ffe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10010f2c0 con 0x7fa1001049b0 2024-03-21T09:08:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ffe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10010f5c0 con 0x7fa100105370 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 0x7fa100199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa10019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 0x7fa100199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45562/0 (socket says 172.21.15.37:45562) 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0fea00640 1 -- 172.21.15.37:0/2836140816 learned_addr learned my addr 172.21.15.37:0/2836140816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0ff400640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa100105370 0x7fa1001a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0f7e00640 1 -- 172.21.15.37:0/2836140816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa100105370 msgr2=0x7fa1001a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0f7e00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa100105370 0x7fa1001a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0f7e00640 1 -- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 msgr2=0x7fa100199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0f7e00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 0x7fa100199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0f7e00640 1 -- 172.21.15.37:0/2836140816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1001a1700 con 0x7fa1001049b0 2024-03-21T09:08:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.064+0000 7fa0fea00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 0x7fa100199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.065+0000 7fa0f7e00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa10019a020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009230 tx=0x7fa0e802c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.065+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8041070 con 0x7fa1001049b0 2024-03-21T09:08:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.065+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0e8004450 con 0x7fa1001049b0 2024-03-21T09:08:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.065+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1001a1990 con 0x7fa1001049b0 2024-03-21T09:08:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.065+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8034440 con 0x7fa1001049b0 2024-03-21T09:08:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.065+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1001a1ed0 con 0x7fa1001049b0 2024-03-21T09:08:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.067+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10010ab30 con 0x7fa1001049b0 2024-03-21T09:08:16.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.069+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa0e8044070 con 0x7fa1001049b0 2024-03-21T09:08:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.070+0000 7fa0f7400640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa0e40a2f90 0x7fa0e40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.070+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0e80f0a80 con 0x7fa1001049b0 2024-03-21T09:08:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.071+0000 7fa0fea00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa0e40a2f90 0x7fa0e40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:16.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.071+0000 7fa0fea00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa0e40a2f90 0x7fa0e40a5450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005fd0 tx=0x7fa0f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.074+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0e80f2e20 con 0x7fa1001049b0 2024-03-21T09:08:16.191 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 systemd[1]: Starting Ceph mds.e for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:16.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.317+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1000622a0 con 0x7fa1001049b0 2024-03-21T09:08:16.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.318+0000 7fa0f7400640 1 -- 172.21.15.37:0/2836140816 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3695 (secure 0 0 0) 0x7fa0e80bd660 con 0x7fa1001049b0 2024-03-21T09:08:16.313 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:16.314 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T09:08:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-03-21T09:08:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa0e40a2f90 msgr2=0x7fa0e40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa0e40a2f90 0x7fa0e40a5450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005fd0 tx=0x7fa0f000b410 comp rx=0 tx=0).stop 2024-03-21T09:08:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 msgr2=0x7fa10019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa10019a020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009230 tx=0x7fa0e802c6d0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 shutdown_connections 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa0e40a2f90 0x7fa0e40a5450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa100105370 0x7fa1001a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1001049b0 0x7fa10019a020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 --2- 172.21.15.37:0/2836140816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1001037c0 0x7fa100199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.321+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 >> 172.21.15.37:0/2836140816 conn(0x7fa1000fed50 msgr2=0x7fa100107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.322+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 shutdown_connections 2024-03-21T09:08:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:16.322+0000 7fa0ffe00640 1 -- 172.21.15.37:0/2836140816 wait complete. 2024-03-21T09:08:16.614 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 podman[66200]: 2024-03-21 09:08:16.257079343 +0000 UTC m=+0.018378999 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:16.614 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 podman[66200]: 2024-03-21 09:08:16.392258788 +0000 UTC m=+0.153558431 container create 3408ef0175f3f1547279c258c924712cb3b4b5e263944707d9f6415ec1c8aa9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T09:08:16.614 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 podman[66200]: 2024-03-21 09:08:16.620269946 +0000 UTC m=+0.381569582 container init 3408ef0175f3f1547279c258c924712cb3b4b5e263944707d9f6415ec1c8aa9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T09:08:16.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:16 smithi053 ceph-mon[36169]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:16 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2836140816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:16 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 podman[66200]: 2024-03-21 09:08:16.623182839 +0000 UTC m=+0.384482484 container start 3408ef0175f3f1547279c258c924712cb3b4b5e263944707d9f6415ec1c8aa9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 ceph-mds[66219]: -- 172.21.15.53:0/2677511579 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643e3a67a40 con 0x5643e3a3b000 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e[66215]: starting mds.e at 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 bash[66200]: 3408ef0175f3f1547279c258c924712cb3b4b5e263944707d9f6415ec1c8aa9d 2024-03-21T09:08:16.942 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:08:16 smithi053 systemd[1]: Started Ceph mds.e for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:17.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:16 smithi071 ceph-mon[36597]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:17.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:16 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2836140816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:17.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:17.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:16 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:17.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:16 smithi037 ceph-mon[31830]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:17.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:16 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2836140816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:17.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:17.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:16 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:18.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6834/1616524011,v1:172.21.15.53:6835/1616524011] up:boot 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: fsmap 5 up:standby 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:18.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:17 smithi071 ceph-mon[36597]: Deploying daemon mds.f on smithi071 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6834/1616524011,v1:172.21.15.53:6835/1616524011] up:boot 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: fsmap 5 up:standby 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:18.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:17 smithi037 ceph-mon[31830]: Deploying daemon mds.f on smithi071 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6834/1616524011,v1:172.21.15.53:6835/1616524011] up:boot 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: fsmap 5 up:standby 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:18.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:18.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:17 smithi053 ceph-mon[36169]: Deploying daemon mds.f on smithi071 2024-03-21T09:08:18.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 -- 172.21.15.37:0/1425855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c105510 msgr2=0x7fa25c10bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 --2- 172.21.15.37:0/1425855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c105510 0x7fa25c10bda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa254009ca0 tx=0x7fa25402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 -- 172.21.15.37:0/1425855829 shutdown_connections 2024-03-21T09:08:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 --2- 172.21.15.37:0/1425855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c105510 0x7fa25c10bda0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 --2- 172.21.15.37:0/1425855829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa25c104b50 0x7fa25c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 --2- 172.21.15.37:0/1425855829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c103960 0x7fa25c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 -- 172.21.15.37:0/1425855829 >> 172.21.15.37:0/1425855829 conn(0x7fa25c0feeb0 msgr2=0x7fa25c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 -- 172.21.15.37:0/1425855829 shutdown_connections 2024-03-21T09:08:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.565+0000 7fa262400640 1 -- 172.21.15.37:0/1425855829 wait complete. 2024-03-21T09:08:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.566+0000 7fa262400640 1 Processor -- start 2024-03-21T09:08:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 -- start start 2024-03-21T09:08:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 0x7fa25c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa25c104b50 0x7fa25c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c105510 0x7fa25c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c10f570 con 0x7fa25c105510 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c10f3f0 con 0x7fa25c104b50 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa262400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c10f6f0 con 0x7fa25c103960 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa25b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 0x7fa25c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa25be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c105510 0x7fa25c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa25b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 0x7fa25c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48274/0 (socket says 172.21.15.37:48274) 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa25b400640 1 -- 172.21.15.37:0/3310516278 learned_addr learned my addr 172.21.15.37:0/3310516278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.567+0000 7fa253e00640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa25c104b50 0x7fa25c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25b400640 1 -- 172.21.15.37:0/3310516278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa25c104b50 msgr2=0x7fa25c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25b400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa25c104b50 0x7fa25c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25b400640 1 -- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c105510 msgr2=0x7fa25c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25b400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c105510 0x7fa25c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25b400640 1 -- 172.21.15.37:0/3310516278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c1a17a0 con 0x7fa25c103960 2024-03-21T09:08:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25be00640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c105510 0x7fa25c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.568+0000 7fa25b400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 0x7fa25c199b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9c0 tx=0x7fa24c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.569+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00cde0 con 0x7fa25c103960 2024-03-21T09:08:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.569+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c1a1a90 con 0x7fa25c103960 2024-03-21T09:08:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.569+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24c010910 con 0x7fa25c103960 2024-03-21T09:08:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.569+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c018a10 con 0x7fa25c103960 2024-03-21T09:08:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.569+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c1a2030 con 0x7fa25c103960 2024-03-21T09:08:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.570+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c10ac80 con 0x7fa25c103960 2024-03-21T09:08:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.573+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa24c01f020 con 0x7fa25c103960 2024-03-21T09:08:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.574+0000 7fa258c00640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2400a2f20 0x7fa2400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.575+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa24c0cf1d0 con 0x7fa25c103960 2024-03-21T09:08:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.575+0000 7fa253e00640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2400a2f20 0x7fa2400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.576+0000 7fa253e00640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2400a2f20 0x7fa2400a53e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa25c19b090 tx=0x7fa244005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.578+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa24c09c400 con 0x7fa25c103960 2024-03-21T09:08:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.830+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa25c02d070 con 0x7fa25c103960 2024-03-21T09:08:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.830+0000 7fa258c00640 1 -- 172.21.15.37:0/3310516278 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4486 (secure 0 0 0) 0x7fa24c09bd00 con 0x7fa25c103960 2024-03-21T09:08:18.826 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:18.826 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T09:08:18.826 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-03-21T09:08:18.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2400a2f20 msgr2=0x7fa2400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2400a2f20 0x7fa2400a53e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa25c19b090 tx=0x7fa244005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 msgr2=0x7fa25c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 0x7fa25c199b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9c0 tx=0x7fa24c00ee80 comp rx=0 tx=0).stop 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 shutdown_connections 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2400a2f20 0x7fa2400a53e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa25c105510 0x7fa25c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa25c104b50 0x7fa25c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 --2- 172.21.15.37:0/3310516278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c103960 0x7fa25c199b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.834+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 >> 172.21.15.37:0/3310516278 conn(0x7fa25c0feeb0 msgr2=0x7fa25c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.835+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 shutdown_connections 2024-03-21T09:08:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:18.835+0000 7fa262400640 1 -- 172.21.15.37:0/3310516278 wait complete. 2024-03-21T09:08:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:18 smithi071 ceph-mon[36597]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:19.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:18 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3310516278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:19.103 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 systemd[1]: Starting Ceph mds.f for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:19.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:18 smithi037 ceph-mon[31830]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:19.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:18 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3310516278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:19.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:18 smithi053 ceph-mon[36169]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:19.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:18 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3310516278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:19.470 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 podman[67040]: 2024-03-21 09:08:19.124471553 +0000 UTC m=+0.018036870 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:19.470 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 podman[67040]: 2024-03-21 09:08:19.223419764 +0000 UTC m=+0.116985067 container create b4766b14ca938b02813a8ad6d5f232a6579e3579578214efc0cbf849890905d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:08:19.470 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 podman[67040]: 2024-03-21 09:08:19.451525252 +0000 UTC m=+0.345090555 container init b4766b14ca938b02813a8ad6d5f232a6579e3579578214efc0cbf849890905d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T09:08:19.470 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 podman[67040]: 2024-03-21 09:08:19.454101787 +0000 UTC m=+0.347667090 container start b4766b14ca938b02813a8ad6d5f232a6579e3579578214efc0cbf849890905d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T09:08:19.852 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 ceph-mds[67060]: -- 172.21.15.71:0/1008777297 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f76463a40 con 0x563f76436800 2024-03-21T09:08:19.853 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f[67056]: starting mds.f at 2024-03-21T09:08:19.853 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 bash[67040]: b4766b14ca938b02813a8ad6d5f232a6579e3579578214efc0cbf849890905d4 2024-03-21T09:08:19.853 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:08:19 smithi071 systemd[1]: Started Ceph mds.f for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: Deploying daemon mds.c on smithi071 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6834/2027751164,v1:172.21.15.71:6835/2027751164] up:boot 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: fsmap 6 up:standby 2024-03-21T09:08:20.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:20 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:08:20.858 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: Deploying daemon mds.c on smithi071 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6834/2027751164,v1:172.21.15.71:6835/2027751164] up:boot 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: fsmap 6 up:standby 2024-03-21T09:08:20.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:20 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:08:20.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: Deploying daemon mds.c on smithi071 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6834/2027751164,v1:172.21.15.71:6835/2027751164] up:boot 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: fsmap 6 up:standby 2024-03-21T09:08:20.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:20 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:08:21.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 -- 172.21.15.37:0/242410869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43881009d0 msgr2=0x7f4388100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:21.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 --2- 172.21.15.37:0/242410869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43881009d0 0x7f4388100e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ca0 tx=0x7f437402c100 comp rx=0 tx=0).stop 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 -- 172.21.15.37:0/242410869 shutdown_connections 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 --2- 172.21.15.37:0/242410869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4388101390 0x7f4388107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 --2- 172.21.15.37:0/242410869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43881009d0 0x7f4388100e50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 --2- 172.21.15.37:0/242410869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 0x7f43880ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 -- 172.21.15.37:0/242410869 >> 172.21.15.37:0/242410869 conn(0x7f43880fad50 msgr2=0x7f43880fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 -- 172.21.15.37:0/242410869 shutdown_connections 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.093+0000 7f438dc00640 1 -- 172.21.15.37:0/242410869 wait complete. 2024-03-21T09:08:21.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 Processor -- start 2024-03-21T09:08:21.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 -- start start 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 0x7f4388103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43881009d0 0x7f4388101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4388101390 0x7f43881022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438810b030 con 0x7f43881009d0 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438810aeb0 con 0x7f43880ff7e0 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.095+0000 7f438dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438810b1b0 con 0x7f4388101390 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f4387e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 0x7f4388103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4388101390 0x7f43881022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f4387e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 0x7f4388103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40836/0 (socket says 172.21.15.37:40836) 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f4387e00640 1 -- 172.21.15.37:0/468730776 learned_addr learned my addr 172.21.15.37:0/468730776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f4387400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43881009d0 0x7f4388101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 -- 172.21.15.37:0/468730776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 msgr2=0x7f4388103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 0x7f4388103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 -- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43881009d0 msgr2=0x7f4388101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43881009d0 0x7f4388101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 -- 172.21.15.37:0/468730776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388102b80 con 0x7f4388101390 2024-03-21T09:08:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f438d200640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4388101390 0x7f43881022f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4380009800 tx=0x7f4380009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.096+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380018030 con 0x7f4388101390 2024-03-21T09:08:21.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.097+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4380005020 con 0x7f4388101390 2024-03-21T09:08:21.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.097+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380004c60 con 0x7f4388101390 2024-03-21T09:08:21.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.097+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43881a1a70 con 0x7f4388101390 2024-03-21T09:08:21.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.097+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43881a1f40 con 0x7f4388101390 2024-03-21T09:08:21.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.098+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438804f470 con 0x7f4388101390 2024-03-21T09:08:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.100+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f43800051d0 con 0x7f4388101390 2024-03-21T09:08:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.101+0000 7f4384c00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f436c0a2f90 0x7f436c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.101+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43800cf100 con 0x7f4388101390 2024-03-21T09:08:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.101+0000 7f4387e00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f436c0a2f90 0x7f436c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.102+0000 7f4387e00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f436c0a2f90 0x7f436c0a5450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4378005fd0 tx=0x7f437800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:21.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.104+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4380011070 con 0x7f4388101390 2024-03-21T09:08:21.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.353+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43881a21d0 con 0x7f4388101390 2024-03-21T09:08:21.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.354+0000 7f4384c00640 1 -- 172.21.15.37:0/468730776 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5277 (secure 0 0 0) 0x7f438009c330 con 0x7f4388101390 2024-03-21T09:08:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:21.350 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-21T09:08:21.350 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-03-21T09:08:21.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f436c0a2f90 msgr2=0x7f436c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:21.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f436c0a2f90 0x7f436c0a5450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4378005fd0 tx=0x7f437800b410 comp rx=0 tx=0).stop 2024-03-21T09:08:21.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4388101390 msgr2=0x7f43881022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:21.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4388101390 0x7f43881022f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4380009800 tx=0x7f4380009cc0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 shutdown_connections 2024-03-21T09:08:21.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f436c0a2f90 0x7f436c0a5450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4388101390 0x7f43881022f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43881009d0 0x7f4388101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 --2- 172.21.15.37:0/468730776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43880ff7e0 0x7f4388103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:21.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 >> 172.21.15.37:0/468730776 conn(0x7f43880fad50 msgr2=0x7f4388105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:21.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 shutdown_connections 2024-03-21T09:08:21.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:21.358+0000 7f438dc00640 1 -- 172.21.15.37:0/468730776 wait complete. 2024-03-21T09:08:21.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:21 smithi053 ceph-mon[36169]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:21.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:21 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/468730776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:22.095 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:21 smithi071 ceph-mon[36597]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:22.095 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:21 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/468730776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:22.096 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:21 smithi071 systemd[1]: Starting Ceph mds.c for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:22.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:21 smithi037 ceph-mon[31830]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:22.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:21 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/468730776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:22.353 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 podman[67352]: 2024-03-21 09:08:22.004754077 +0000 UTC m=+0.017218521 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:22.353 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 podman[67352]: 2024-03-21 09:08:22.11409882 +0000 UTC m=+0.126563248 container create 7250c6777bcb59dbce6aa43271a41a0824992c4a86f881638ea3b9d5d1a3f051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T09:08:22.853 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 podman[67352]: 2024-03-21 09:08:22.367373387 +0000 UTC m=+0.379837824 container init 7250c6777bcb59dbce6aa43271a41a0824992c4a86f881638ea3b9d5d1a3f051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T09:08:22.853 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 podman[67352]: 2024-03-21 09:08:22.370119934 +0000 UTC m=+0.382584377 container start 7250c6777bcb59dbce6aa43271a41a0824992c4a86f881638ea3b9d5d1a3f051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T09:08:22.853 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 ceph-mds[67371]: -- 172.21.15.71:0/2648464951 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd168dfa40 con 0x55fd168b2c00 2024-03-21T09:08:22.853 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c[67367]: starting mds.c at 2024-03-21T09:08:22.853 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 bash[67352]: 7250c6777bcb59dbce6aa43271a41a0824992c4a86f881638ea3b9d5d1a3f051 2024-03-21T09:08:22.853 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:08:22 smithi071 systemd[1]: Started Ceph mds.c for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:23.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:23.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 -- 172.21.15.37:0/3002673607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730107bc0 msgr2=0x7f4730107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 --2- 172.21.15.37:0/3002673607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730107bc0 0x7f4730107fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4720009860 tx=0x7f472002bb70 comp rx=0 tx=0).stop 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 -- 172.21.15.37:0/3002673607 shutdown_connections 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 --2- 172.21.15.37:0/3002673607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4730109770 0x7f4730110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 --2- 172.21.15.37:0/3002673607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f4730109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 --2- 172.21.15.37:0/3002673607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730107bc0 0x7f4730107fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.604+0000 7f472fe00640 1 -- 172.21.15.37:0/3002673607 >> 172.21.15.37:0/3002673607 conn(0x7f4730076b60 msgr2=0x7f4730078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.605+0000 7f472fe00640 1 -- 172.21.15.37:0/3002673607 shutdown_connections 2024-03-21T09:08:23.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.605+0000 7f472fe00640 1 -- 172.21.15.37:0/3002673607 wait complete. 2024-03-21T09:08:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.605+0000 7f472fe00640 1 Processor -- start 2024-03-21T09:08:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 -- start start 2024-03-21T09:08:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f473019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730109770 0x7f473019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301a3670 0x7f47301a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730113810 con 0x7f4730108db0 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730113690 con 0x7f47301a3670 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730113990 con 0x7f4730109770 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f473019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f473019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45610/0 (socket says 172.21.15.37:45610) 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472ea00640 1 -- 172.21.15.37:0/1551455665 learned_addr learned my addr 172.21.15.37:0/1551455665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f4726000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730109770 0x7f473019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.606+0000 7f472f400640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301a3670 0x7f47301a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472ea00640 1 -- 172.21.15.37:0/1551455665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730109770 msgr2=0x7f473019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472ea00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730109770 0x7f473019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472ea00640 1 -- 172.21.15.37:0/1551455665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301a3670 msgr2=0x7f47301a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472ea00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301a3670 0x7f47301a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472ea00640 1 -- 172.21.15.37:0/1551455665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47301a60d0 con 0x7f4730108db0 2024-03-21T09:08:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472f400640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301a3670 0x7f47301a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f4726000640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730109770 0x7f473019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472ea00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f473019e0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47200349e0 tx=0x7f4720032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720041020 con 0x7f4730108db0 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f472002c9f0 con 0x7f4730108db0 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720039040 con 0x7f4730108db0 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47301a6360 con 0x7f4730108db0 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.607+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301a68a0 con 0x7f4730108db0 2024-03-21T09:08:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.608+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473010f000 con 0x7f4730108db0 2024-03-21T09:08:23.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.610+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f472002c510 con 0x7f4730108db0 2024-03-21T09:08:23.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.612+0000 7f4727e00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f47140a2f90 0x7f47140a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:23.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.612+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47200f0130 con 0x7f4730108db0 2024-03-21T09:08:23.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.612+0000 7f4726000640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f47140a2f90 0x7f47140a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:23.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.613+0000 7f4726000640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f47140a2f90 0x7f47140a5450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f47180096c0 tx=0x7f4718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.617+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f472004b020 con 0x7f4730108db0 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: Deploying daemon mds.h on smithi053 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6836/2290408711,v1:172.21.15.71:6837/2290408711] up:boot 2024-03-21T09:08:23.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: fsmap 7 up:standby 2024-03-21T09:08:23.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:23 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:08:23.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.865+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f473002d070 con 0x7f4730108db0 2024-03-21T09:08:23.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.866+0000 7f4727e00640 1 -- 172.21.15.37:0/1551455665 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6068 (secure 0 0 0) 0x7f47200bd3f0 con 0x7f4730108db0 2024-03-21T09:08:23.861 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:23.861 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-21T09:08:23.862 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f47140a2f90 msgr2=0x7f47140a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f47140a2f90 0x7f47140a5450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f47180096c0 tx=0x7f4718009260 comp rx=0 tx=0).stop 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 msgr2=0x7f473019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f473019e0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47200349e0 tx=0x7f4720032470 comp rx=0 tx=0).stop 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 shutdown_connections 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f47140a2f90 0x7f47140a5450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47301a3670 0x7f47301a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4730109770 0x7f473019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 --2- 172.21.15.37:0/1551455665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4730108db0 0x7f473019e0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:23.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 >> 172.21.15.37:0/1551455665 conn(0x7f4730076b60 msgr2=0x7f473010c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:23.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 shutdown_connections 2024-03-21T09:08:23.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:23.869+0000 7f472fe00640 1 -- 172.21.15.37:0/1551455665 wait complete. 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: Deploying daemon mds.h on smithi053 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6836/2290408711,v1:172.21.15.71:6837/2290408711] up:boot 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: fsmap 7 up:standby 2024-03-21T09:08:23.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:23 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:08:23.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: Deploying daemon mds.h on smithi053 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6836/2290408711,v1:172.21.15.71:6837/2290408711] up:boot 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: fsmap 7 up:standby 2024-03-21T09:08:23.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:23 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:08:24.619 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:24 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1551455665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:24.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:24 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1551455665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:24.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:24 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1551455665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:24.941 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:24 smithi053 systemd[1]: Starting Ceph mds.h for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:25.310 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 podman[66515]: 2024-03-21 09:08:24.945827725 +0000 UTC m=+0.018404327 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:25.310 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 podman[66515]: 2024-03-21 09:08:25.064261203 +0000 UTC m=+0.136837824 container create ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:08:25.310 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 podman[66515]: 2024-03-21 09:08:25.3169228 +0000 UTC m=+0.389499411 container init ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T09:08:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:25.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:25.692 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 podman[66515]: 2024-03-21 09:08:25.319656535 +0000 UTC m=+0.392233129 container start ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T09:08:25.692 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-mds[66534]: -- 172.21.15.53:0/3088160122 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd7bb2da40 con 0x55cd7bb00800 2024-03-21T09:08:25.692 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h[66530]: starting mds.h at 2024-03-21T09:08:25.692 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 bash[66515]: ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 2024-03-21T09:08:25.692 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:08:25 smithi053 systemd[1]: Started Ceph mds.h for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:25.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:25.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:25 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:25.883 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:25.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:25.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:25 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.124+0000 7f7fc3200640 1 -- 172.21.15.37:0/1432989421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41015c0 msgr2=0x7f7fc41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.124+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1432989421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41015c0 0x7f7fc41019e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009be0 tx=0x7f7fb402bef0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 -- 172.21.15.37:0/1432989421 shutdown_connections 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1432989421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc4103170 0x7f7fc4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1432989421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fc41027b0 0x7f7fc4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1432989421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41015c0 0x7f7fc41019e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 -- 172.21.15.37:0/1432989421 >> 172.21.15.37:0/1432989421 conn(0x7f7fc40fcb30 msgr2=0x7f7fc40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:26.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 -- 172.21.15.37:0/1432989421 shutdown_connections 2024-03-21T09:08:26.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.125+0000 7f7fc3200640 1 -- 172.21.15.37:0/1432989421 wait complete. 2024-03-21T09:08:26.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.126+0000 7f7fc3200640 1 Processor -- start 2024-03-21T09:08:26.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.126+0000 7f7fc3200640 1 -- start start 2024-03-21T09:08:26.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.126+0000 7f7fc3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 0x7f7fc4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:26.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41027b0 0x7f7fc4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fc4103170 0x7f7fc4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc410d290 con 0x7f7fc41015c0 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc410d110 con 0x7f7fc4103170 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc410d410 con 0x7f7fc41027b0 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 0x7f7fc4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 0x7f7fc4075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60824/0 (socket says 172.21.15.37:60824) 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 -- 172.21.15.37:0/1814882605 learned_addr learned my addr 172.21.15.37:0/1814882605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc2800640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fc4103170 0x7f7fc4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1400640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41027b0 0x7f7fc4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 -- 172.21.15.37:0/1814882605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41027b0 msgr2=0x7f7fc4073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:26.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41027b0 0x7f7fc4073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 -- 172.21.15.37:0/1814882605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fc4103170 msgr2=0x7f7fc4073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:26.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fc4103170 0x7f7fc4073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.127+0000 7f7fc1e00640 1 -- 172.21.15.37:0/1814882605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4074860 con 0x7f7fc41015c0 2024-03-21T09:08:26.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.128+0000 7f7fc1e00640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 0x7f7fc4075450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7fb402c500 tx=0x7f7fb4033580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.128+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4043020 con 0x7f7fc41015c0 2024-03-21T09:08:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.128+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fb4003e50 con 0x7f7fc41015c0 2024-03-21T09:08:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.128+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc41a1a00 con 0x7f7fc41015c0 2024-03-21T09:08:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.128+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4034690 con 0x7f7fc41015c0 2024-03-21T09:08:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.128+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc41a1fa0 con 0x7f7fc41015c0 2024-03-21T09:08:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.129+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc41088e0 con 0x7f7fc41015c0 2024-03-21T09:08:26.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.131+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7fb4046020 con 0x7f7fc41015c0 2024-03-21T09:08:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.133+0000 7f7fbaa00640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7fa80a2f90 0x7f7fa80a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.133+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7fb40f1e70 con 0x7f7fc41015c0 2024-03-21T09:08:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.133+0000 7f7fc1400640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7fa80a2f90 0x7f7fa80a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.134+0000 7f7fc1400640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7fa80a2f90 0x7f7fa80a5450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40751c0 tx=0x7f7fac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:26.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.136+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7fb404d020 con 0x7f7fc41015c0 2024-03-21T09:08:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.379+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fc402d070 con 0x7f7fc41015c0 2024-03-21T09:08:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.380+0000 7f7fbaa00640 1 -- 172.21.15.37:0/1814882605 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6859 (secure 0 0 0) 0x7f7fb40bf0a0 con 0x7f7fc41015c0 2024-03-21T09:08:26.375 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:26.375 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-21T09:08:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7fa80a2f90 msgr2=0x7f7fa80a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7fa80a2f90 0x7f7fa80a5450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40751c0 tx=0x7f7fac005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 msgr2=0x7f7fc4075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 0x7f7fc4075450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7fb402c500 tx=0x7f7fb4033580 comp rx=0 tx=0).stop 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 shutdown_connections 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7fa80a2f90 0x7f7fa80a5450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fc4103170 0x7f7fc4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc41027b0 0x7f7fc4073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 --2- 172.21.15.37:0/1814882605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc41015c0 0x7f7fc4075450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 >> 172.21.15.37:0/1814882605 conn(0x7f7fc40fcb30 msgr2=0x7f7fc4105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 shutdown_connections 2024-03-21T09:08:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:26.383+0000 7f7fc3200640 1 -- 172.21.15.37:0/1814882605 wait complete. 2024-03-21T09:08:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:26 smithi037 ceph-mon[31830]: Deploying daemon mds.i on smithi071 2024-03-21T09:08:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:26 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6836/3665426355,v1:172.21.15.53:6837/3665426355] up:boot 2024-03-21T09:08:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:26 smithi037 ceph-mon[31830]: fsmap 8 up:standby 2024-03-21T09:08:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:26 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:08:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:26 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1814882605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:26 smithi071 ceph-mon[36597]: Deploying daemon mds.i on smithi071 2024-03-21T09:08:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:26 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6836/3665426355,v1:172.21.15.53:6837/3665426355] up:boot 2024-03-21T09:08:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:26 smithi071 ceph-mon[36597]: fsmap 8 up:standby 2024-03-21T09:08:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:26 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:08:26.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:26 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1814882605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:26.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:26 smithi053 ceph-mon[36169]: Deploying daemon mds.i on smithi071 2024-03-21T09:08:26.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:26 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6836/3665426355,v1:172.21.15.53:6837/3665426355] up:boot 2024-03-21T09:08:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:26 smithi053 ceph-mon[36169]: fsmap 8 up:standby 2024-03-21T09:08:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:26 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:08:26.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:26 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1814882605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:27.844 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:27 smithi071 ceph-mon[36597]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:27.844 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:27 smithi071 systemd[1]: Starting Ceph mds.i for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:27.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:27 smithi037 ceph-mon[31830]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:27 smithi053 ceph-mon[36169]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:28.103 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:27 smithi071 podman[67658]: 2024-03-21 09:08:27.752716083 +0000 UTC m=+0.017540943 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:28.103 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:27 smithi071 podman[67658]: 2024-03-21 09:08:27.871738942 +0000 UTC m=+0.136563808 container create d9b59ea58aaf572c0e8ab697f9ce4d5a15a2b28f14378d0c7d17fe3f38acd552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T09:08:28.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:28.603 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:28 smithi071 podman[67658]: 2024-03-21 09:08:28.184275385 +0000 UTC m=+0.449100231 container init d9b59ea58aaf572c0e8ab697f9ce4d5a15a2b28f14378d0c7d17fe3f38acd552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:08:28.603 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:28 smithi071 podman[67658]: 2024-03-21 09:08:28.187090705 +0000 UTC m=+0.451915552 container start d9b59ea58aaf572c0e8ab697f9ce4d5a15a2b28f14378d0c7d17fe3f38acd552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T09:08:28.603 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:28 smithi071 ceph-mds[67677]: -- 172.21.15.71:0/1892593752 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e067c71a40 con 0x55e067c44c00 2024-03-21T09:08:28.603 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i[67673]: starting mds.i at 2024-03-21T09:08:28.603 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:28 smithi071 bash[67658]: d9b59ea58aaf572c0e8ab697f9ce4d5a15a2b28f14378d0c7d17fe3f38acd552 2024-03-21T09:08:28.603 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:08:28 smithi071 systemd[1]: Started Ceph mds.i for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 -- 172.21.15.37:0/2706725787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 msgr2=0x7f5c98104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 --2- 172.21.15.37:0/2706725787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c98104e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009900 tx=0x7f5c8002bc50 comp rx=0 tx=0).stop 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 -- 172.21.15.37:0/2706725787 shutdown_connections 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 --2- 172.21.15.37:0/2706725787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c98105370 0x7f5c9810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 --2- 172.21.15.37:0/2706725787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c98104e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 --2- 172.21.15.37:0/2706725787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c981037c0 0x7f5c98103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 -- 172.21.15.37:0/2706725787 >> 172.21.15.37:0/2706725787 conn(0x7f5c980fed50 msgr2=0x7f5c98101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 -- 172.21.15.37:0/2706725787 shutdown_connections 2024-03-21T09:08:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.644+0000 7f5c97e00640 1 -- 172.21.15.37:0/2706725787 wait complete. 2024-03-21T09:08:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 Processor -- start 2024-03-21T09:08:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 -- start start 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981037c0 0x7f5c98199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c9819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c98105370 0x7f5c981a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9810f440 con 0x7f5c981049b0 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9810f2c0 con 0x7f5c981037c0 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c97e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9810f5c0 con 0x7f5c98105370 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.646+0000 7f5c8e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c9819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c8e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c9819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60848/0 (socket says 172.21.15.37:60848) 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981037c0 0x7f5c98199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c8e000640 1 -- 172.21.15.37:0/242502261 learned_addr learned my addr 172.21.15.37:0/242502261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:28.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c97400640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c98105370 0x7f5c981a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 -- 172.21.15.37:0/242502261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c98105370 msgr2=0x7f5c981a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c98105370 0x7f5c981a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 -- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 msgr2=0x7f5c9819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c9819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 -- 172.21.15.37:0/242502261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c981a1790 con 0x7f5c981037c0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c8e000640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c9819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.647+0000 7f5c96a00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981037c0 0x7f5c98199b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c88005fd0 tx=0x7f5c8800eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.648+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800cdb0 con 0x7f5c981037c0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.648+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c88003e50 con 0x7f5c981037c0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.648+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88010690 con 0x7f5c981037c0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.648+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c981a1a80 con 0x7f5c981037c0 2024-03-21T09:08:28.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.648+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c981a1f50 con 0x7f5c981037c0 2024-03-21T09:08:28.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.649+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9804f470 con 0x7f5c981037c0 2024-03-21T09:08:28.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.651+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5c880042d0 con 0x7f5c981037c0 2024-03-21T09:08:28.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.653+0000 7f5c8fe00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5c7c0a2f10 0x7f5c7c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:28.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.653+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c880cf820 con 0x7f5c981037c0 2024-03-21T09:08:28.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.653+0000 7f5c8e000640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5c7c0a2f10 0x7f5c7c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:28.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.654+0000 7f5c8e000640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5c7c0a2f10 0x7f5c7c0a53d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000c00 tx=0x7f5c8002c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.657+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c8809ca50 con 0x7f5c981037c0 2024-03-21T09:08:28.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.905+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c9810ab80 con 0x7f5c981037c0 2024-03-21T09:08:28.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.906+0000 7f5c8fe00640 1 -- 172.21.15.37:0/242502261 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7649 (secure 0 0 0) 0x7f5c8809c350 con 0x7f5c981037c0 2024-03-21T09:08:28.900 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:28.900 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T09:08:28.900 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5c7c0a2f10 msgr2=0x7f5c7c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5c7c0a2f10 0x7f5c7c0a53d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000c00 tx=0x7f5c8002c8a0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981037c0 msgr2=0x7f5c98199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981037c0 0x7f5c98199b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c88005fd0 tx=0x7f5c8800eed0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 shutdown_connections 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5c7c0a2f10 0x7f5c7c0a53d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c98105370 0x7f5c981a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c981049b0 0x7f5c9819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 --2- 172.21.15.37:0/242502261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c981037c0 0x7f5c98199b70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:28.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 >> 172.21.15.37:0/242502261 conn(0x7f5c980fed50 msgr2=0x7f5c98107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:28.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 shutdown_connections 2024-03-21T09:08:28.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:28.909+0000 7f5c97e00640 1 -- 172.21.15.37:0/242502261 wait complete. 2024-03-21T09:08:29.643 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:29.643 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.643 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.643 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.643 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:29.643 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:29.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:29.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: Deploying daemon mds.g on smithi037 2024-03-21T09:08:29.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] up:boot 2024-03-21T09:08:29.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: fsmap 9 up:standby 2024-03-21T09:08:29.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:08:29.644 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:29 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/242502261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: Deploying daemon mds.g on smithi037 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] up:boot 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: fsmap 9 up:standby 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:08:29.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:29 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/242502261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: Deploying daemon mds.g on smithi037 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] up:boot 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: fsmap 9 up:standby 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:08:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:29 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/242502261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:30.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:31.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.434+0000 7f4a33200640 1 -- 172.21.15.37:0/458873672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 msgr2=0x7f4a34100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:31.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.434+0000 7f4a33200640 1 --2- 172.21.15.37:0/458873672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a34100c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009900 tx=0x7f4a2002bc40 comp rx=0 tx=0).stop 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.435+0000 7f4a33200640 1 -- 172.21.15.37:0/458873672 shutdown_connections 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.436+0000 7f4a33200640 1 --2- 172.21.15.37:0/458873672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a34101180 0x7f4a34101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.436+0000 7f4a33200640 1 --2- 172.21.15.37:0/458873672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a34100c40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.436+0000 7f4a33200640 1 --2- 172.21.15.37:0/458873672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 0x7f4a341002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.436+0000 7f4a33200640 1 -- 172.21.15.37:0/458873672 >> 172.21.15.37:0/458873672 conn(0x7f4a340fba20 msgr2=0x7f4a340fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.439+0000 7f4a33200640 1 -- 172.21.15.37:0/458873672 shutdown_connections 2024-03-21T09:08:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.439+0000 7f4a33200640 1 -- 172.21.15.37:0/458873672 wait complete. 2024-03-21T09:08:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 Processor -- start 2024-03-21T09:08:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 -- start start 2024-03-21T09:08:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 0x7f4a34199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a3419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a34101180 0x7f4a341a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3410f3f0 con 0x7f4a341007e0 2024-03-21T09:08:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3410f270 con 0x7f4a340ffe80 2024-03-21T09:08:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.445+0000 7f4a33200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3410f570 con 0x7f4a34101180 2024-03-21T09:08:31.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.446+0000 7f4a31e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 0x7f4a34199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:31.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.446+0000 7f4a31400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a3419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:31.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.446+0000 7f4a31e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 0x7f4a34199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:52544/0 (socket says 172.21.15.37:52544) 2024-03-21T09:08:31.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.446+0000 7f4a31e00640 1 -- 172.21.15.37:0/2721221174 learned_addr learned my addr 172.21.15.37:0/2721221174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:31.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.446+0000 7f4a32800640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a34101180 0x7f4a341a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.447+0000 7f4a31400640 1 -- 172.21.15.37:0/2721221174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a34101180 msgr2=0x7f4a341a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.447+0000 7f4a31400640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a34101180 0x7f4a341a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.447+0000 7f4a31400640 1 -- 172.21.15.37:0/2721221174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 msgr2=0x7f4a34199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.447+0000 7f4a31400640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 0x7f4a34199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.447+0000 7f4a31400640 1 -- 172.21.15.37:0/2721221174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a341a1740 con 0x7f4a341007e0 2024-03-21T09:08:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.447+0000 7f4a31400640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a3419a060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a20036040 tx=0x7f4a2002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.448+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20043020 con 0x7f4a341007e0 2024-03-21T09:08:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.448+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a200035f0 con 0x7f4a341007e0 2024-03-21T09:08:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.448+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2003e6b0 con 0x7f4a341007e0 2024-03-21T09:08:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.448+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a341a19d0 con 0x7f4a341007e0 2024-03-21T09:08:31.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.448+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a341a1ed0 con 0x7f4a341007e0 2024-03-21T09:08:31.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.451+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3410abd0 con 0x7f4a341007e0 2024-03-21T09:08:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.457+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4a20046020 con 0x7f4a341007e0 2024-03-21T09:08:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.460+0000 7f4a2aa00640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4a180a3010 0x7f4a180a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:31.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.460+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a200f1940 con 0x7f4a341007e0 2024-03-21T09:08:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.464+0000 7f4a31e00640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4a180a3010 0x7f4a180a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.465+0000 7f4a31e00640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4a180a3010 0x7f4a180a54d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a24005fd0 tx=0x7f4a2400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:31.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.467+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a2004d020 con 0x7f4a341007e0 2024-03-21T09:08:31.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:31 smithi037 ceph-mon[31830]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:31 smithi053 ceph-mon[36169]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.719+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a3402d070 con 0x7f4a341007e0 2024-03-21T09:08:31.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.720+0000 7f4a2aa00640 1 -- 172.21.15.37:0/2721221174 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7649 (secure 0 0 0) 0x7f4a200bed20 con 0x7f4a341007e0 2024-03-21T09:08:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:31.714 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T09:08:31.714 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-03-21T09:08:31.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4a180a3010 msgr2=0x7f4a180a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4a180a3010 0x7f4a180a54d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a24005fd0 tx=0x7f4a2400b2c0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 msgr2=0x7f4a3419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a3419a060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a20036040 tx=0x7f4a2002c650 comp rx=0 tx=0).stop 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 shutdown_connections 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4a180a3010 0x7f4a180a54d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a34101180 0x7f4a341a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a341007e0 0x7f4a3419a060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 --2- 172.21.15.37:0/2721221174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a340ffe80 0x7f4a34199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.722+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 >> 172.21.15.37:0/2721221174 conn(0x7f4a340fba20 msgr2=0x7f4a340fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.723+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 shutdown_connections 2024-03-21T09:08:31.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:31.723+0000 7f4a33200640 1 -- 172.21.15.37:0/2721221174 wait complete. 2024-03-21T09:08:31.754 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:31 smithi071 ceph-mon[36597]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:32.271 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 podman[70256]: 2024-03-21 09:08:32.072301292 +0000 UTC m=+0.160291401 container create 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:08:32.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:32 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2721221174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:32.673 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 podman[70256]: 2024-03-21 09:08:32.400222556 +0000 UTC m=+0.488212645 container init 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-21T09:08:32.673 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 podman[70256]: 2024-03-21 09:08:32.410861139 +0000 UTC m=+0.498851219 container start 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:08:32.673 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 ceph-mds[70283]: -- 172.21.15.37:0/3154758439 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e3e418fa40 con 0x55e3e4162800 2024-03-21T09:08:32.673 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g[70277]: starting mds.g at 2024-03-21T09:08:32.673 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 bash[70256]: 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 2024-03-21T09:08:32.673 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:08:32 smithi037 systemd[1]: Started Ceph mds.g for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:32.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:32 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2721221174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:32 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2721221174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:33.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:33.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: Deploying daemon mds.a on smithi037 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.37:6838/508705459,v1:172.21.15.37:6839/508705459] up:boot 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: fsmap 10 up:standby 2024-03-21T09:08:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:33 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:08:33.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:33.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:33.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:33.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:33.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: Deploying daemon mds.a on smithi037 2024-03-21T09:08:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.37:6838/508705459,v1:172.21.15.37:6839/508705459] up:boot 2024-03-21T09:08:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: fsmap 10 up:standby 2024-03-21T09:08:33.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:33 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 -- 172.21.15.37:0/1012388213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29700756e0 msgr2=0x7f2970073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 --2- 172.21.15.37:0/1012388213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29700756e0 0x7f2970073b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2960009930 tx=0x7f296002bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 -- 172.21.15.37:0/1012388213 shutdown_connections 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 --2- 172.21.15.37:0/1012388213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f297010db70 0x7f297010ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 --2- 172.21.15.37:0/1012388213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f2970074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 --2- 172.21.15.37:0/1012388213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29700756e0 0x7f2970073b30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 -- 172.21.15.37:0/1012388213 >> 172.21.15.37:0/1012388213 conn(0x7f29700fcb30 msgr2=0x7f29700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 -- 172.21.15.37:0/1012388213 shutdown_connections 2024-03-21T09:08:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.958+0000 7f296fe00640 1 -- 172.21.15.37:0/1012388213 wait complete. 2024-03-21T09:08:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.959+0000 7f296fe00640 1 Processor -- start 2024-03-21T09:08:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.959+0000 7f296fe00640 1 -- start start 2024-03-21T09:08:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f297019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29700756e0 0x7f297019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f297010db70 0x7f29701a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970113810 con 0x7f2970074100 2024-03-21T09:08:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970113690 con 0x7f297010db70 2024-03-21T09:08:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970113990 con 0x7f29700756e0 2024-03-21T09:08:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f297019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f297019dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46238/0 (socket says 172.21.15.37:46238) 2024-03-21T09:08:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296ea00640 1 -- 172.21.15.37:0/2822364110 learned_addr learned my addr 172.21.15.37:0/2822364110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.960+0000 7f296f400640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f297010db70 0x7f29701a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f2966000640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29700756e0 0x7f297019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f296ea00640 1 -- 172.21.15.37:0/2822364110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29700756e0 msgr2=0x7f297019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f296ea00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29700756e0 0x7f297019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f296ea00640 1 -- 172.21.15.37:0/2822364110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f297010db70 msgr2=0x7f29701a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f296ea00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f297010db70 0x7f29701a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f296ea00640 1 -- 172.21.15.37:0/2822364110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29701a5b00 con 0x7f2970074100 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f2966000640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29700756e0 0x7f297019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.961+0000 7f296ea00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f297019dee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2960000c00 tx=0x7f296002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.962+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960041070 con 0x7f2970074100 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.962+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29701a5d90 con 0x7f2970074100 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.962+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29701a6210 con 0x7f2970074100 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.962+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2960004450 con 0x7f2970074100 2024-03-21T09:08:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.962+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960034440 con 0x7f2970074100 2024-03-21T09:08:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.965+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2970103d40 con 0x7f2970074100 2024-03-21T09:08:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.966+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2960044070 con 0x7f2970074100 2024-03-21T09:08:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.967+0000 7f2967e00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f29540a2f90 0x7f29540a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.967+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f29600f0bc0 con 0x7f2970074100 2024-03-21T09:08:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.968+0000 7f2966000640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f29540a2f90 0x7f29540a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.968+0000 7f2966000640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f29540a2f90 0x7f29540a5450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f297019f3f0 tx=0x7f2958005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:33.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:33.972+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f29600bde80 con 0x7f2970074100 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: Deploying daemon mds.a on smithi037 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.37:6838/508705459,v1:172.21.15.37:6839/508705459] up:boot 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: fsmap 10 up:standby 2024-03-21T09:08:34.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:33 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:08:34.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.222+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f297002d070 con 0x7f2970074100 2024-03-21T09:08:34.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.223+0000 7f2967e00640 1 -- 172.21.15.37:0/2822364110 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8438 (secure 0 0 0) 0x7f29600bd780 con 0x7f2970074100 2024-03-21T09:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T09:08:34.217 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 11 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.225+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f29540a2f90 msgr2=0x7f29540a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.225+0000 7f296fe00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f29540a2f90 0x7f29540a5450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f297019f3f0 tx=0x7f2958005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 msgr2=0x7f297019dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f297019dee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2960000c00 tx=0x7f296002c640 comp rx=0 tx=0).stop 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 shutdown_connections 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f29540a2f90 0x7f29540a5450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f297010db70 0x7f29701a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29700756e0 0x7f297019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 --2- 172.21.15.37:0/2822364110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2970074100 0x7f297019dee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 >> 172.21.15.37:0/2822364110 conn(0x7f29700fcb30 msgr2=0x7f2970100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 shutdown_connections 2024-03-21T09:08:34.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:34.226+0000 7f296fe00640 1 -- 172.21.15.37:0/2822364110 wait complete. 2024-03-21T09:08:34.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:34 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2822364110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:34.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:34 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2822364110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:35.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:34 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2822364110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:35.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:35 smithi037 ceph-mon[31830]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:35.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:35 smithi053 ceph-mon[36169]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:36.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:35 smithi071 ceph-mon[36597]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:36.107 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:35 smithi037 systemd[1]: Starting Ceph mds.a for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:36.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:36.410 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 podman[70611]: 2024-03-21 09:08:36.019883597 +0000 UTC m=+0.041143339 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:36.410 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 podman[70611]: 2024-03-21 09:08:36.144749032 +0000 UTC m=+0.166008834 container create 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 -- 172.21.15.37:0/1980907699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c074080 msgr2=0x7f607c0744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 --2- 172.21.15.37:0/1980907699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c074080 0x7f607c0744e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6068009900 tx=0x7f606802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 -- 172.21.15.37:0/1980907699 shutdown_connections 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 --2- 172.21.15.37:0/1980907699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c109470 0x7f607c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 --2- 172.21.15.37:0/1980907699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c074080 0x7f607c0744e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 --2- 172.21.15.37:0/1980907699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c075660 0x7f607c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 -- 172.21.15.37:0/1980907699 >> 172.21.15.37:0/1980907699 conn(0x7f607c0fcb30 msgr2=0x7f607c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 -- 172.21.15.37:0/1980907699 shutdown_connections 2024-03-21T09:08:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.471+0000 7f607b200640 1 -- 172.21.15.37:0/1980907699 wait complete. 2024-03-21T09:08:36.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.472+0000 7f607b200640 1 Processor -- start 2024-03-21T09:08:36.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.472+0000 7f607b200640 1 -- start start 2024-03-21T09:08:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 0x7f607c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c075660 0x7f607c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 0x7f607c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c10f470 con 0x7f607c109470 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c10f2f0 con 0x7f607c074080 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c10f5f0 con 0x7f607c075660 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 0x7f607c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 0x7f607c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 0x7f607c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:38502/0 (socket says 172.21.15.37:38502) 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 -- 172.21.15.37:0/801480164 learned_addr learned my addr 172.21.15.37:0/801480164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 0x7f607c1a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46254/0 (socket says 172.21.15.37:46254) 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079400640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c075660 0x7f607c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 -- 172.21.15.37:0/801480164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c075660 msgr2=0x7f607c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c075660 0x7f607c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 -- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 msgr2=0x7f607c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 0x7f607c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f6079e00640 1 -- 172.21.15.37:0/801480164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c1a16b0 con 0x7f607c074080 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.473+0000 7f607a800640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 0x7f607c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.474+0000 7f6079e00640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 0x7f607c199a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f606c00ea60 tx=0x7f606c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.474+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00cde0 con 0x7f607c074080 2024-03-21T09:08:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.474+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f606c0035f0 con 0x7f607c074080 2024-03-21T09:08:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.474+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c010670 con 0x7f607c074080 2024-03-21T09:08:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.474+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c1a19a0 con 0x7f607c074080 2024-03-21T09:08:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.474+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a1e70 con 0x7f607c074080 2024-03-21T09:08:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.476+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c108230 con 0x7f607c074080 2024-03-21T09:08:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.478+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f606c0037a0 con 0x7f607c074080 2024-03-21T09:08:36.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.480+0000 7f6072a00640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60600a2f10 0x7f60600a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:36.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.480+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f606c0cf2a0 con 0x7f607c074080 2024-03-21T09:08:36.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.480+0000 7f6079400640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60600a2f10 0x7f60600a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:36.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.481+0000 7f6079400640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60600a2f10 0x7f60600a53d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f606802c2e0 tx=0x7f606802c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:36.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.484+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f606c09c4d0 con 0x7f607c074080 2024-03-21T09:08:36.674 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 podman[70611]: 2024-03-21 09:08:36.415657371 +0000 UTC m=+0.436917173 container init 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-21T09:08:36.674 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 podman[70611]: 2024-03-21 09:08:36.425976602 +0000 UTC m=+0.447236353 container start 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:08:36.674 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 ceph-mds[70649]: -- 172.21.15.37:0/830470029 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56084c45ba40 con 0x56084c42f000 2024-03-21T09:08:36.674 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: starting mds.a at 2024-03-21T09:08:36.674 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 bash[70611]: 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 2024-03-21T09:08:36.674 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:36 smithi037 systemd[1]: Started Ceph mds.a for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.758+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f607c02d070 con 0x7f607c074080 2024-03-21T09:08:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.759+0000 7f6072a00640 1 -- 172.21.15.37:0/801480164 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9227 (secure 0 0 0) 0x7f606c09bdd0 con 0x7f607c074080 2024-03-21T09:08:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:36.753 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-03-21T09:08:36.753 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60600a2f10 msgr2=0x7f60600a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60600a2f10 0x7f60600a53d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f606802c2e0 tx=0x7f606802c920 comp rx=0 tx=0).stop 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 msgr2=0x7f607c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 0x7f607c199a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f606c00ea60 tx=0x7f606c00ef20 comp rx=0 tx=0).stop 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 shutdown_connections 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60600a2f10 0x7f60600a53d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f607c109470 0x7f607c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c075660 0x7f607c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 --2- 172.21.15.37:0/801480164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f607c074080 0x7f607c199a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:36.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 >> 172.21.15.37:0/801480164 conn(0x7f607c0fcb30 msgr2=0x7f607c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:36.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 shutdown_connections 2024-03-21T09:08:36.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:36.762+0000 7f607b200640 1 -- 172.21.15.37:0/801480164 wait complete. 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] up:boot 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: fsmap 11 up:standby 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:37.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:37.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:37.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:37.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: Deploying daemon mds.b on smithi053 2024-03-21T09:08:37.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/801480164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] up:boot 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: fsmap 11 up:standby 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:37.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: Deploying daemon mds.b on smithi053 2024-03-21T09:08:37.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/801480164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] up:boot 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: fsmap 11 up:standby 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:38.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T09:08:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T09:08:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: Deploying daemon mds.b on smithi053 2024-03-21T09:08:38.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/801480164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:38.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:38.941 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:38 smithi053 systemd[1]: Starting Ceph mds.b for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.019+0000 7f8896400640 1 -- 172.21.15.37:0/1369603040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890102790 msgr2=0x7f8890104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.019+0000 7f8896400640 1 --2- 172.21.15.37:0/1369603040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890102790 0x7f8890104c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8878009900 tx=0x7f887802bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 -- 172.21.15.37:0/1369603040 shutdown_connections 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 --2- 172.21.15.37:0/1369603040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890105140 0x7f88901055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 --2- 172.21.15.37:0/1369603040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890102790 0x7f8890104c00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 --2- 172.21.15.37:0/1369603040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f8890102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 -- 172.21.15.37:0/1369603040 >> 172.21.15.37:0/1369603040 conn(0x7f88900fba20 msgr2=0x7f88900fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 -- 172.21.15.37:0/1369603040 shutdown_connections 2024-03-21T09:08:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.020+0000 7f8896400640 1 -- 172.21.15.37:0/1369603040 wait complete. 2024-03-21T09:08:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.021+0000 7f8896400640 1 Processor -- start 2024-03-21T09:08:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.021+0000 7f8896400640 1 -- start start 2024-03-21T09:08:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8896400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f889019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8896400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102790 0x7f889019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8896400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890105140 0x7f88901a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8896400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890113870 con 0x7f8890102790 2024-03-21T09:08:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8896400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901136f0 con 0x7f8890105140 2024-03-21T09:08:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8896400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901139f0 con 0x7f88900ffe20 2024-03-21T09:08:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f889019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8887e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102790 0x7f889019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f889019de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:60672/0 (socket says 172.21.15.37:60672) 2024-03-21T09:08:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 -- 172.21.15.37:0/1637335511 learned_addr learned my addr 172.21.15.37:0/1637335511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895a00640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890105140 0x7f88901a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 -- 172.21.15.37:0/1637335511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890105140 msgr2=0x7f88901a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890105140 0x7f88901a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 -- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102790 msgr2=0x7f889019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102790 0x7f889019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.022+0000 7f8895000640 1 -- 172.21.15.37:0/1637335511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88901a5aa0 con 0x7f88900ffe20 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.023+0000 7f8895000640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f889019de80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f888000eb90 tx=0x7f888000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.023+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880004d20 con 0x7f88900ffe20 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.023+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88901a5d90 con 0x7f88900ffe20 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.023+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f888000ee90 con 0x7f88900ffe20 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.023+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880010830 con 0x7f88900ffe20 2024-03-21T09:08:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.023+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88901a61a0 con 0x7f88900ffe20 2024-03-21T09:08:39.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.024+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88900fd510 con 0x7f88900ffe20 2024-03-21T09:08:39.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.026+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8880044b40 con 0x7f88900ffe20 2024-03-21T09:08:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.028+0000 7f888e000640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f88740a2f10 0x7f88740a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.028+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88800cf6d0 con 0x7f88900ffe20 2024-03-21T09:08:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.028+0000 7f8887e00640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f88740a2f10 0x7f88740a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:39.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.029+0000 7f8887e00640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f88740a2f10 0x7f88740a53d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f887802c240 tx=0x7f887802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:39.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.030+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f888009c900 con 0x7f88900ffe20 2024-03-21T09:08:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.274+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f889002d070 con 0x7f88900ffe20 2024-03-21T09:08:39.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.275+0000 7f888e000640 1 -- 172.21.15.37:0/1637335511 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9227 (secure 0 0 0) 0x7f888009c200 con 0x7f88900ffe20 2024-03-21T09:08:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:39.269 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-03-21T09:08:39.269 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f88740a2f10 msgr2=0x7f88740a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f88740a2f10 0x7f88740a53d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f887802c240 tx=0x7f887802df90 comp rx=0 tx=0).stop 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 msgr2=0x7f889019de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f889019de80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f888000eb90 tx=0x7f888000c6a0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 shutdown_connections 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f88740a2f10 0x7f88740a53d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8890105140 0x7f88901a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102790 0x7f889019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 --2- 172.21.15.37:0/1637335511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88900ffe20 0x7f889019de80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 >> 172.21.15.37:0/1637335511 conn(0x7f88900fba20 msgr2=0x7f8890100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 shutdown_connections 2024-03-21T09:08:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:39.278+0000 7f8896400640 1 -- 172.21.15.37:0/1637335511 wait complete. 2024-03-21T09:08:39.341 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 podman[66825]: 2024-03-21 09:08:38.954745611 +0000 UTC m=+0.018468860 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:08:39.341 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 podman[66825]: 2024-03-21 09:08:39.069727071 +0000 UTC m=+0.133450306 container create 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:08:39.341 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 podman[66825]: 2024-03-21 09:08:39.347305834 +0000 UTC m=+0.411029062 container init 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T09:08:39.624 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 podman[66825]: 2024-03-21 09:08:39.35138222 +0000 UTC m=+0.415105451 container start 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T09:08:39.794 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mds[66845]: -- 172.21.15.53:0/2241512710 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566d7e8fa40 con 0x5566d7e62c00 2024-03-21T09:08:39.795 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b[66840]: starting mds.b at 2024-03-21T09:08:39.795 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 bash[66825]: 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c 2024-03-21T09:08:39.795 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:08:39 smithi053 systemd[1]: Started Ceph mds.b for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1637335511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:39 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1637335511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:39.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:39 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:40.067 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:40.068 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1637335511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:40.068 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:40.068 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:40.068 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:40.068 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:40.068 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:39 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:40 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] up:boot 2024-03-21T09:08:40.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:40 smithi053 ceph-mon[36169]: fsmap 12 up:standby 2024-03-21T09:08:40.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:40 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:08:40.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:40 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:40.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:40 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] up:boot 2024-03-21T09:08:40.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:40 smithi037 ceph-mon[31830]: fsmap 12 up:standby 2024-03-21T09:08:40.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:40 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:08:40.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:40 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:40 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] up:boot 2024-03-21T09:08:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:40 smithi071 ceph-mon[36597]: fsmap 12 up:standby 2024-03-21T09:08:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:08:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:40 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:41.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 -- 172.21.15.37:0/844453679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e8103170 msgr2=0x7f17e8109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 --2- 172.21.15.37:0/844453679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e8103170 0x7f17e8109a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e002bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 -- 172.21.15.37:0/844453679 shutdown_connections 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 --2- 172.21.15.37:0/844453679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e8103170 0x7f17e8109a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 --2- 172.21.15.37:0/844453679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e81027b0 0x7f17e8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 --2- 172.21.15.37:0/844453679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e81015c0 0x7f17e81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 -- 172.21.15.37:0/844453679 >> 172.21.15.37:0/844453679 conn(0x7f17e80fcb30 msgr2=0x7f17e80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 -- 172.21.15.37:0/844453679 shutdown_connections 2024-03-21T09:08:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.516+0000 7f17ed000640 1 -- 172.21.15.37:0/844453679 wait complete. 2024-03-21T09:08:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.517+0000 7f17ed000640 1 Processor -- start 2024-03-21T09:08:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.517+0000 7f17ed000640 1 -- start start 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17ed000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e81015c0 0x7f17e8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17ed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 0x7f17e8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8103170 0x7f17e81a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17ed000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e810d270 con 0x7f17e81015c0 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17ed000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e810d0f0 con 0x7f17e8103170 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17ed000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e810d3f0 con 0x7f17e81027b0 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17dea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 0x7f17e8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17e7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e81015c0 0x7f17e8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17dea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 0x7f17e8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:60686/0 (socket says 172.21.15.37:60686) 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17dea00640 1 -- 172.21.15.37:0/1333805659 learned_addr learned my addr 172.21.15.37:0/1333805659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.518+0000 7f17e7e00640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8103170 0x7f17e81a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17dea00640 1 -- 172.21.15.37:0/1333805659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8103170 msgr2=0x7f17e81a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17dea00640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8103170 0x7f17e81a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17dea00640 1 -- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e81015c0 msgr2=0x7f17e8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17dea00640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e81015c0 0x7f17e8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17dea00640 1 -- 172.21.15.37:0/1333805659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e81a1630 con 0x7f17e81027b0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17dea00640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 0x7f17e8199f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9f0 tx=0x7f17d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d000cd80 con 0x7f17e81027b0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17d0010910 con 0x7f17e81027b0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e81a1920 con 0x7f17e81027b0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0018990 con 0x7f17e81027b0 2024-03-21T09:08:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.519+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e81a1ec0 con 0x7f17e81027b0 2024-03-21T09:08:41.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.520+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e81a2120 con 0x7f17e81027b0 2024-03-21T09:08:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.522+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f17d0010ac0 con 0x7f17e81027b0 2024-03-21T09:08:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.523+0000 7f17e4c00640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f17cc0a2f90 0x7f17cc0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.524+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17d00cf250 con 0x7f17e81027b0 2024-03-21T09:08:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.524+0000 7f17e7400640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f17cc0a2f90 0x7f17cc0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:41.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.524+0000 7f17e7400640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f17cc0a2f90 0x7f17cc0a5450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f17d80099a0 tx=0x7f17d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:41.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.528+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17d009be30 con 0x7f17e81027b0 2024-03-21T09:08:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.791+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17e81a22f0 con 0x7f17e81027b0 2024-03-21T09:08:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.793+0000 7f17e4c00640 1 -- 172.21.15.37:0/1333805659 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10016 (secure 0 0 0) 0x7f17d009fed0 con 0x7f17e81027b0 2024-03-21T09:08:41.786 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:41.787 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-03-21T09:08:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f17cc0a2f90 msgr2=0x7f17cc0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f17cc0a2f90 0x7f17cc0a5450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f17d80099a0 tx=0x7f17d8009500 comp rx=0 tx=0).stop 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 msgr2=0x7f17e8199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 0x7f17e8199f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9f0 tx=0x7f17d000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 shutdown_connections 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f17cc0a2f90 0x7f17cc0a5450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17e8103170 0x7f17e81a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e81027b0 0x7f17e8199f50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 --2- 172.21.15.37:0/1333805659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e81015c0 0x7f17e8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 >> 172.21.15.37:0/1333805659 conn(0x7f17e80fcb30 msgr2=0x7f17e80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 shutdown_connections 2024-03-21T09:08:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:41.795+0000 7f17ed000640 1 -- 172.21.15.37:0/1333805659 wait complete. 2024-03-21T09:08:41.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:08:41.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:41 smithi037 ceph-mon[31830]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:41.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:41 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:41 smithi053 ceph-mon[36169]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:41.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:41 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 -- 172.21.15.37:0/595994811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff940077c40 msgr2=0x7ff9400784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 --2- 172.21.15.37:0/595994811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff940077c40 0x7ff9400784e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff938009be0 tx=0x7ff93802bef0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 -- 172.21.15.37:0/595994811 shutdown_connections 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 --2- 172.21.15.37:0/595994811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff940077c40 0x7ff9400784e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 --2- 172.21.15.37:0/595994811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9400772a0 0x7ff940077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 --2- 172.21.15.37:0/595994811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940076060 0x7ff940076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 -- 172.21.15.37:0/595994811 >> 172.21.15.37:0/595994811 conn(0x7ff9400fef00 msgr2=0x7ff940101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.016+0000 7ff945200640 1 -- 172.21.15.37:0/595994811 shutdown_connections 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.017+0000 7ff945200640 1 -- 172.21.15.37:0/595994811 wait complete. 2024-03-21T09:08:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.018+0000 7ff945200640 1 Processor -- start 2024-03-21T09:08:42.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.018+0000 7ff945200640 1 -- start start 2024-03-21T09:08:42.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.018+0000 7ff945200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 0x7ff94019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.018+0000 7ff945200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9400772a0 0x7ff94019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff945200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940077c40 0x7ff9401a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff945200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940113970 con 0x7ff940077c40 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff945200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401137f0 con 0x7ff9400772a0 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff945200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940113af0 con 0x7ff940076060 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9400772a0 0x7ff94019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940077c40 0x7ff9401a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 0x7ff94019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 0x7ff94019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:60700/0 (socket says 172.21.15.37:60700) 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9400772a0 0x7ff94019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:38574/0 (socket says 172.21.15.37:38574) 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 -- 172.21.15.37:0/1318014710 learned_addr learned my addr 172.21.15.37:0/1318014710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93e800640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940077c40 0x7ff9401a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46308/0 (socket says 172.21.15.37:46308) 2024-03-21T09:08:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 -- 172.21.15.37:0/1318014710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9400772a0 msgr2=0x7ff94019e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9400772a0 0x7ff94019e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 -- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940077c40 msgr2=0x7ff9401a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940077c40 0x7ff9401a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.019+0000 7ff93de00640 1 -- 172.21.15.37:0/1318014710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9401a5b90 con 0x7ff940076060 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.020+0000 7ff93de00640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 0x7ff94019df70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e9c0 tx=0x7ff93000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.020+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000cde0 con 0x7ff940076060 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.020+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9300035f0 con 0x7ff940076060 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.020+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401a5e80 con 0x7ff940076060 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.020+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930010690 con 0x7ff940076060 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.020+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a62f0 con 0x7ff940076060 2024-03-21T09:08:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.021+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940110410 con 0x7ff940076060 2024-03-21T09:08:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.023+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff9300037a0 con 0x7ff940076060 2024-03-21T09:08:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.024+0000 7ff936a00640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff9240a2fa0 0x7ff9240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.024+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9300cf280 con 0x7ff940076060 2024-03-21T09:08:42.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.025+0000 7ff93d400640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff9240a2fa0 0x7ff9240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.025+0000 7ff93d400640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff9240a2fa0 0x7ff9240a5460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff94019f480 tx=0x7ff92c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:42.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.028+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff93009c4b0 con 0x7ff940076060 2024-03-21T09:08:42.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:41 smithi071 ceph-mon[36597]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:42.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:42.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:42.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:42.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:42.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:41 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:42.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.241+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff940062340 con 0x7ff940076060 2024-03-21T09:08:42.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.242+0000 7ff936a00640 1 -- 172.21.15.37:0/1318014710 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18370 (secure 0 0 0) 0x7ff93009bdb0 con 0x7ff940076060 2024-03-21T09:08:42.236 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:42.236 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:07:49.100221+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff9240a2fa0 msgr2=0x7ff9240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff9240a2fa0 0x7ff9240a5460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff94019f480 tx=0x7ff92c005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 msgr2=0x7ff94019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 0x7ff94019df70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e9c0 tx=0x7ff93000ee80 comp rx=0 tx=0).stop 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 shutdown_connections 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff9240a2fa0 0x7ff9240a5460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff940077c40 0x7ff9401a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9400772a0 0x7ff94019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 --2- 172.21.15.37:0/1318014710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff940076060 0x7ff94019df70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 >> 172.21.15.37:0/1318014710 conn(0x7ff9400fef00 msgr2=0x7ff940100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 shutdown_connections 2024-03-21T09:08:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.245+0000 7ff945200640 1 -- 172.21.15.37:0/1318014710 wait complete. 2024-03-21T09:08:42.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T09:08:42.256 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.480+0000 7fbf49400640 1 -- 172.21.15.37:0/1944735498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440fff30 msgr2=0x7fbf44102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.480+0000 7fbf49400640 1 --2- 172.21.15.37:0/1944735498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440fff30 0x7fbf44102360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009900 tx=0x7fbf3402bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 -- 172.21.15.37:0/1944735498 shutdown_connections 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 --2- 172.21.15.37:0/1944735498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 0x7fbf441056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 --2- 172.21.15.37:0/1944735498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf441028a0 0x7fbf44104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 --2- 172.21.15.37:0/1944735498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440fff30 0x7fbf44102360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 -- 172.21.15.37:0/1944735498 >> 172.21.15.37:0/1944735498 conn(0x7fbf440fbb30 msgr2=0x7fbf440fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 -- 172.21.15.37:0/1944735498 shutdown_connections 2024-03-21T09:08:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.481+0000 7fbf49400640 1 -- 172.21.15.37:0/1944735498 wait complete. 2024-03-21T09:08:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.482+0000 7fbf49400640 1 Processor -- start 2024-03-21T09:08:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.482+0000 7fbf49400640 1 -- start start 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf49400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 0x7fbf4419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf49400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf441028a0 0x7fbf4419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf49400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 0x7fbf441a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf49400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf44113910 con 0x7fbf440fff30 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf49400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf44113790 con 0x7fbf441028a0 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf49400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf44113a90 con 0x7fbf44105250 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf42800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 0x7fbf4419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf42800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 0x7fbf4419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46336/0 (socket says 172.21.15.37:46336) 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf43200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 0x7fbf441a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf42800640 1 -- 172.21.15.37:0/560860767 learned_addr learned my addr 172.21.15.37:0/560860767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf41e00640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf441028a0 0x7fbf4419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf43200640 1 -- 172.21.15.37:0/560860767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf441028a0 msgr2=0x7fbf4419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.483+0000 7fbf43200640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf441028a0 0x7fbf4419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf43200640 1 -- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 msgr2=0x7fbf4419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf43200640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 0x7fbf4419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf43200640 1 -- 172.21.15.37:0/560860767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf441a5bc0 con 0x7fbf44105250 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf42800640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 0x7fbf4419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf43200640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 0x7fbf441a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e970 tx=0x7fbf3c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00ccc0 con 0x7fbf44105250 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c005020 con 0x7fbf44105250 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004a00 con 0x7fbf44105250 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf441a5eb0 con 0x7fbf44105250 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.484+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf441a6370 con 0x7fbf44105250 2024-03-21T09:08:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.485+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf440fd620 con 0x7fbf44105250 2024-03-21T09:08:42.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.488+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbf3c0051d0 con 0x7fbf44105250 2024-03-21T09:08:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.490+0000 7fbf3b400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf280a2fa0 0x7fbf280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.490+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbf3c0ceee0 con 0x7fbf44105250 2024-03-21T09:08:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.490+0000 7fbf42800640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf280a2fa0 0x7fbf280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.491+0000 7fbf42800640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf280a2fa0 0x7fbf280a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf34000c00 tx=0x7fbf3402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.493+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf3c09c110 con 0x7fbf44105250 2024-03-21T09:08:42.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.751+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf44062340 con 0x7fbf44105250 2024-03-21T09:08:42.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.752+0000 7fbf3b400640 1 -- 172.21.15.37:0/560860767 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf3c09ba10 con 0x7fbf44105250 2024-03-21T09:08:42.746 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf280a2fa0 msgr2=0x7fbf280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf280a2fa0 0x7fbf280a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf34000c00 tx=0x7fbf3402c880 comp rx=0 tx=0).stop 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 msgr2=0x7fbf441a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 0x7fbf441a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e970 tx=0x7fbf3c00ee30 comp rx=0 tx=0).stop 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 shutdown_connections 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbf280a2fa0 0x7fbf280a5460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf44105250 0x7fbf441a5550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf441028a0 0x7fbf4419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 --2- 172.21.15.37:0/560860767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf440fff30 0x7fbf4419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 >> 172.21.15.37:0/560860767 conn(0x7fbf440fbb30 msgr2=0x7fbf44103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:42.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 shutdown_connections 2024-03-21T09:08:42.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.755+0000 7fbf49400640 1 -- 172.21.15.37:0/560860767 wait complete. 2024-03-21T09:08:42.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-21T09:08:42.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:42 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1333805659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:42.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:42 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1318014710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 -- 172.21.15.37:0/1005560112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4108ed0 msgr2=0x7f6ff4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/1005560112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4108ed0 0x7f6ff4109350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009900 tx=0x7f6fe002bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 -- 172.21.15.37:0/1005560112 shutdown_connections 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/1005560112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ff4109890 0x7f6ff4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/1005560112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4108ed0 0x7f6ff4109350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/1005560112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4107ce0 0x7f6ff4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 -- 172.21.15.37:0/1005560112 >> 172.21.15.37:0/1005560112 conn(0x7f6ff4076ca0 msgr2=0x7f6ff40790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 -- 172.21.15.37:0/1005560112 shutdown_connections 2024-03-21T09:08:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.979+0000 7f6ff9e00640 1 -- 172.21.15.37:0/1005560112 wait complete. 2024-03-21T09:08:42.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 Processor -- start 2024-03-21T09:08:42.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 -- start start 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4107ce0 0x7f6ff419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 0x7f6ff419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ff4109890 0x7f6ff41a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4113990 con 0x7f6ff4107ce0 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4113810 con 0x7f6ff4108ed0 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.980+0000 7f6ff9e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4113b10 con 0x7f6ff4109890 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4107ce0 0x7f6ff419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 0x7f6ff419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 0x7f6ff419e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:38620/0 (socket says 172.21.15.37:38620) 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 -- 172.21.15.37:0/4086710894 learned_addr learned my addr 172.21.15.37:0/4086710894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4107ce0 0x7f6ff419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46348/0 (socket says 172.21.15.37:46348) 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff3400640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ff4109890 0x7f6ff41a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 -- 172.21.15.37:0/4086710894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ff4109890 msgr2=0x7f6ff41a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ff4109890 0x7f6ff41a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 -- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4107ce0 msgr2=0x7f6ff419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4107ce0 0x7f6ff419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.981+0000 7f6ff2000640 1 -- 172.21.15.37:0/4086710894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff41a5c00 con 0x7f6ff4108ed0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.982+0000 7f6ff2000640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 0x7f6ff419e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.982+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0043020 con 0x7f6ff4108ed0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.982+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fe00054e0 con 0x7f6ff4108ed0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.982+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0003c40 con 0x7f6ff4108ed0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.983+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff41a5e90 con 0x7f6ff4108ed0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.983+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff41a6300 con 0x7f6ff4108ed0 2024-03-21T09:08:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.984+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff410f000 con 0x7f6ff4108ed0 2024-03-21T09:08:42.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.987+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6fe0046020 con 0x7f6ff4108ed0 2024-03-21T09:08:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.988+0000 7f6ff8a00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6fd80a2f20 0x7f6fd80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.988+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6fe00f1340 con 0x7f6ff4108ed0 2024-03-21T09:08:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.989+0000 7f6ff2a00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6fd80a2f20 0x7f6fd80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:42.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.989+0000 7f6ff2a00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6fd80a2f20 0x7f6fd80a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4108d30 tx=0x7f6fe4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:42.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:42.990+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fe004d020 con 0x7f6ff4108ed0 2024-03-21T09:08:43.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:42 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1333805659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:43.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:42 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1318014710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:43.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:42 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1333805659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:43.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:42 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1318014710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:43.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.232+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ff402d2e0 con 0x7f6ff4108ed0 2024-03-21T09:08:43.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.232+0000 7f6ff8a00640 1 -- 172.21.15.37:0/4086710894 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6fe00be610 con 0x7f6ff4108ed0 2024-03-21T09:08:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:43.226 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-03-21T09:08:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:08:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-03-21T09:08:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-03-21T09:08:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:08:43.227 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6fd80a2f20 msgr2=0x7f6fd80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6fd80a2f20 0x7f6fd80a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4108d30 tx=0x7f6fe4009500 comp rx=0 tx=0).stop 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 msgr2=0x7f6ff419e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 0x7f6ff419e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe002c650 comp rx=0 tx=0).stop 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 shutdown_connections 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6fd80a2f20 0x7f6fd80a53e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ff4109890 0x7f6ff41a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff4108ed0 0x7f6ff419e520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 --2- 172.21.15.37:0/4086710894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ff4107ce0 0x7f6ff419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 >> 172.21.15.37:0/4086710894 conn(0x7f6ff4076ca0 msgr2=0x7f6ff410c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:43.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 shutdown_connections 2024-03-21T09:08:43.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.235+0000 7f6ff9e00640 1 -- 172.21.15.37:0/4086710894 wait complete. 2024-03-21T09:08:43.247 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-21T09:08:43.248 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 -- 172.21.15.37:0/2394224539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 msgr2=0x7f6980106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 --2- 172.21.15.37:0/2394224539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980106cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6968009900 tx=0x7f696802bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 -- 172.21.15.37:0/2394224539 shutdown_connections 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 --2- 172.21.15.37:0/2394224539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6980107210 0x7f6980109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 --2- 172.21.15.37:0/2394224539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980106cd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 --2- 172.21.15.37:0/2394224539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6980101fb0 0x7f69801043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 -- 172.21.15.37:0/2394224539 >> 172.21.15.37:0/2394224539 conn(0x7f69800fb980 msgr2=0x7f69800fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 -- 172.21.15.37:0/2394224539 shutdown_connections 2024-03-21T09:08:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.477+0000 7f697f200640 1 -- 172.21.15.37:0/2394224539 wait complete. 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.478+0000 7f697f200640 1 Processor -- start 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.478+0000 7f697f200640 1 -- start start 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6980101fb0 0x7f6980197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6980107210 0x7f698019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698010d1d0 con 0x7f6980101fb0 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698010d050 con 0x7f69801048e0 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698010d350 con 0x7f6980107210 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f6975400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f6975400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:38642/0 (socket says 172.21.15.37:38642) 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f6975400640 1 -- 172.21.15.37:0/2000453887 learned_addr learned my addr 172.21.15.37:0/2000453887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697de00640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6980101fb0 0x7f6980197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:43.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.479+0000 7f697e800640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6980107210 0x7f698019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:43.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6975400640 1 -- 172.21.15.37:0/2000453887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6980107210 msgr2=0x7f698019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:43.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6975400640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6980107210 0x7f698019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6975400640 1 -- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6980101fb0 msgr2=0x7f6980197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6975400640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6980101fb0 0x7f6980197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6975400640 1 -- 172.21.15.37:0/2000453887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698019f420 con 0x7f69801048e0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6975400640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980197d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f696802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968043020 con 0x7f69801048e0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69680054e0 con 0x7f69801048e0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698019f6b0 con 0x7f69801048e0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.480+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968003c40 con 0x7f69801048e0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.481+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698019fc50 con 0x7f69801048e0 2024-03-21T09:08:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.482+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69800fd470 con 0x7f69801048e0 2024-03-21T09:08:43.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.483+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6968046020 con 0x7f69801048e0 2024-03-21T09:08:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.485+0000 7f6977400640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f69640a2fa0 0x7f69640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.485+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69680f0f60 con 0x7f69801048e0 2024-03-21T09:08:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.485+0000 7f697de00640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f69640a2fa0 0x7f69640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.486+0000 7f697de00640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f69640a2fa0 0x7f69640a5460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6970005fd0 tx=0x7f6970009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.489+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f696804d020 con 0x7f69801048e0 2024-03-21T09:08:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:43.699+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f69800622a0 con 0x7f69801048e0 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/560860767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:44.002 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:43 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4086710894' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T09:08:44.010 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/560860767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:44.011 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:43 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4086710894' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T09:08:44.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:44.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:44.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:44.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:44.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/560860767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:08:44.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:44.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:43 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4086710894' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T09:08:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.721+0000 7f6977400640 1 -- 172.21.15.37:0/2000453887 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f69680be190 con 0x7f69801048e0 2024-03-21T09:08:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f69640a2fa0 msgr2=0x7f69640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f69640a2fa0 0x7f69640a5460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6970005fd0 tx=0x7f6970009500 comp rx=0 tx=0).stop 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 msgr2=0x7f6980197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980197d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f696802c650 comp rx=0 tx=0).stop 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 shutdown_connections 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f69640a2fa0 0x7f69640a5460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6980107210 0x7f698019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69801048e0 0x7f6980197d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 --2- 172.21.15.37:0/2000453887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6980101fb0 0x7f6980197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.729+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 >> 172.21.15.37:0/2000453887 conn(0x7f69800fb980 msgr2=0x7f6980102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:44.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.730+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 shutdown_connections 2024-03-21T09:08:44.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.730+0000 7f697f200640 1 -- 172.21.15.37:0/2000453887 wait complete. 2024-03-21T09:08:44.749 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-21T09:08:44.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:44 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2000453887' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T09:08:44.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:44 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 -- 172.21.15.37:0/827079158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 msgr2=0x7fd428106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 --2- 172.21.15.37:0/827079158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428106e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd414009900 tx=0x7fd41402bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 -- 172.21.15.37:0/827079158 shutdown_connections 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 --2- 172.21.15.37:0/827079158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4281073a0 0x7fd4281097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 --2- 172.21.15.37:0/827079158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428106e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 --2- 172.21.15.37:0/827079158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd428102140 0x7fd428104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 -- 172.21.15.37:0/827079158 >> 172.21.15.37:0/827079158 conn(0x7fd4280fbb50 msgr2=0x7fd4280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 -- 172.21.15.37:0/827079158 shutdown_connections 2024-03-21T09:08:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.970+0000 7fd430400640 1 -- 172.21.15.37:0/827079158 wait complete. 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.971+0000 7fd430400640 1 Processor -- start 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.971+0000 7fd430400640 1 -- start start 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.971+0000 7fd430400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd428102140 0x7fd428197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.971+0000 7fd430400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd430400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4281073a0 0x7fd42819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd430400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42810d390 con 0x7fd4281073a0 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd430400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42810d210 con 0x7fd428104a70 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd430400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42810d510 con 0x7fd428102140 2024-03-21T09:08:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4281073a0 0x7fd42819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd428102140 0x7fd428197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54368/0 (socket says 172.21.15.37:54368) 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42d000640 1 -- 172.21.15.37:0/2469984482 learned_addr learned my addr 172.21.15.37:0/2469984482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42da00640 1 -- 172.21.15.37:0/2469984482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 msgr2=0x7fd428197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42da00640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42da00640 1 -- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4281073a0 msgr2=0x7fd42819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42da00640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4281073a0 0x7fd42819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.972+0000 7fd42da00640 1 -- 172.21.15.37:0/2469984482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42819f620 con 0x7fd428102140 2024-03-21T09:08:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.973+0000 7fd42da00640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd428102140 0x7fd428197aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00e9f0 tx=0x7fd41c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.973+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00ccc0 con 0x7fd428102140 2024-03-21T09:08:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.973+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd41c010910 con 0x7fd428102140 2024-03-21T09:08:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.973+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c018960 con 0x7fd428102140 2024-03-21T09:08:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.973+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4280fff90 con 0x7fd428102140 2024-03-21T09:08:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.973+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4281004b0 con 0x7fd428102140 2024-03-21T09:08:44.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.975+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4280fd640 con 0x7fd428102140 2024-03-21T09:08:44.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.976+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd41c010ac0 con 0x7fd428102140 2024-03-21T09:08:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.977+0000 7fd426000640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4100a2f20 0x7fd4100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.978+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fd41c0cf4e0 con 0x7fd428102140 2024-03-21T09:08:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.978+0000 7fd42d000640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4100a2f20 0x7fd4100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.978+0000 7fd42d000640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4100a2f20 0x7fd4100a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd414006010 tx=0x7fd41402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:44.981+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd41c09bf50 con 0x7fd428102140 2024-03-21T09:08:45.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:44 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2000453887' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T09:08:45.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:44 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T09:08:45.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:44 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2000453887' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T09:08:45.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:44 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T09:08:45.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.206+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd428101890 con 0x7fd428102140 2024-03-21T09:08:45.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.801+0000 7fd426000640 1 -- 172.21.15.37:0/2469984482 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd41c09fff0 con 0x7fd428102140 2024-03-21T09:08:45.796 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.806+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4100a2f20 msgr2=0x7fd4100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.806+0000 7fd430400640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4100a2f20 0x7fd4100a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd414006010 tx=0x7fd41402df90 comp rx=0 tx=0).stop 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.806+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd428102140 msgr2=0x7fd428197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.806+0000 7fd430400640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd428102140 0x7fd428197aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00e9f0 tx=0x7fd41c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.807+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 shutdown_connections 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.807+0000 7fd430400640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4100a2f20 0x7fd4100a53e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.807+0000 7fd430400640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4281073a0 0x7fd42819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.807+0000 7fd430400640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd428104a70 0x7fd428197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:45.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.807+0000 7fd430400640 1 --2- 172.21.15.37:0/2469984482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd428102140 0x7fd428197aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:45.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.807+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 >> 172.21.15.37:0/2469984482 conn(0x7fd4280fbb50 msgr2=0x7fd428102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:45.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.808+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 shutdown_connections 2024-03-21T09:08:45.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:45.808+0000 7fd430400640 1 -- 172.21.15.37:0/2469984482 wait complete. 2024-03-21T09:08:45.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-21T09:08:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 -- 172.21.15.37:0/3804936195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec105490 msgr2=0x7f5cec10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 --2- 172.21.15.37:0/3804936195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec105490 0x7f5cec10bd20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009ca0 tx=0x7f5ce402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 -- 172.21.15.37:0/3804936195 shutdown_connections 2024-03-21T09:08:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 --2- 172.21.15.37:0/3804936195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec105490 0x7f5cec10bd20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 --2- 172.21.15.37:0/3804936195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec104ad0 0x7f5cec104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 --2- 172.21.15.37:0/3804936195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec1038e0 0x7f5cec103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 -- 172.21.15.37:0/3804936195 >> 172.21.15.37:0/3804936195 conn(0x7f5cec0fee50 msgr2=0x7f5cec101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 -- 172.21.15.37:0/3804936195 shutdown_connections 2024-03-21T09:08:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.031+0000 7f5cf3000640 1 -- 172.21.15.37:0/3804936195 wait complete. 2024-03-21T09:08:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.032+0000 7f5cf3000640 1 Processor -- start 2024-03-21T09:08:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 -- start start 2024-03-21T09:08:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec1038e0 0x7f5cec199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec104ad0 0x7f5cec19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 0x7f5cec1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec10f560 con 0x7f5cec1038e0 2024-03-21T09:08:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec10f3e0 con 0x7f5cec104ad0 2024-03-21T09:08:46.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec10f6e0 con 0x7f5cec105490 2024-03-21T09:08:46.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5ceb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec104ad0 0x7f5cec19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:46.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 0x7f5cec1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:46.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 0x7f5cec1a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54386/0 (socket says 172.21.15.37:54386) 2024-03-21T09:08:46.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5cf1000640 1 -- 172.21.15.37:0/1094521301 learned_addr learned my addr 172.21.15.37:0/1094521301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:46.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.033+0000 7f5ceb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec104ad0 0x7f5cec19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54392/0 (socket says 172.21.15.37:54392) 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cebe00640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec1038e0 0x7f5cec199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf1000640 1 -- 172.21.15.37:0/1094521301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec104ad0 msgr2=0x7f5cec19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf1000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec104ad0 0x7f5cec19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf1000640 1 -- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec1038e0 msgr2=0x7f5cec199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf1000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec1038e0 0x7f5cec199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf1000640 1 -- 172.21.15.37:0/1094521301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec1a17c0 con 0x7f5cec105490 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf1000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 0x7f5cec1a1150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4037040 tx=0x7f5ce4032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4041020 con 0x7f5cec105490 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ce4034cd0 con 0x7f5cec105490 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4039040 con 0x7f5cec105490 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.034+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec1a1a50 con 0x7f5cec105490 2024-03-21T09:08:46.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.035+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1a1ef0 con 0x7f5cec105490 2024-03-21T09:08:46.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.036+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec10ac00 con 0x7f5cec105490 2024-03-21T09:08:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.037+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5ce4044020 con 0x7f5cec105490 2024-03-21T09:08:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.039+0000 7f5ce8c00640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5cd00a2f20 0x7f5cd00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.039+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f5ce40f1250 con 0x7f5cec105490 2024-03-21T09:08:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.039+0000 7f5cebe00640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5cd00a2f20 0x7f5cd00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.040+0000 7f5cebe00640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5cd00a2f20 0x7f5cd00a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5cec104930 tx=0x7f5cdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:46.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.043+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ce404b020 con 0x7f5cec105490 2024-03-21T09:08:46.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:46.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T09:08:46.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T09:08:46.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:45 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:46.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T09:08:46.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T09:08:46.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:45 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:45 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.302+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5cec062310 con 0x7f5cec105490 2024-03-21T09:08:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.891+0000 7f5ce8c00640 1 -- 172.21.15.37:0/1094521301 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5ce40be220 con 0x7f5cec105490 2024-03-21T09:08:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5cd00a2f20 msgr2=0x7f5cd00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5cd00a2f20 0x7f5cd00a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5cec104930 tx=0x7f5cdc005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 msgr2=0x7f5cec1a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 0x7f5cec1a1150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4037040 tx=0x7f5ce4032f40 comp rx=0 tx=0).stop 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 shutdown_connections 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5cd00a2f20 0x7f5cd00a53e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec105490 0x7f5cec1a1150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cec104ad0 0x7f5cec19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 --2- 172.21.15.37:0/1094521301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec1038e0 0x7f5cec199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.896+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 >> 172.21.15.37:0/1094521301 conn(0x7f5cec0fee50 msgr2=0x7f5cec107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.897+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 shutdown_connections 2024-03-21T09:08:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:46.897+0000 7f5cf3000640 1 -- 172.21.15.37:0/1094521301 wait complete. 2024-03-21T09:08:46.914 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-21T09:08:46.914 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-21T09:08:47.075 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2469984482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: pgmap v313: 33 pgs: 12 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1094521301' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T09:08:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:46 smithi037 ceph-mon[31830]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-03-21T09:08:47.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2469984482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: pgmap v313: 33 pgs: 12 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1094521301' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T09:08:47.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T09:08:47.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:46 smithi071 ceph-mon[36597]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 -- 172.21.15.37:0/3687672595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e0103260 msgr2=0x7f42e0109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 --2- 172.21.15.37:0/3687672595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e0103260 0x7f42e0109af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009ca0 tx=0x7f42d802c0f0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 -- 172.21.15.37:0/3687672595 shutdown_connections 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 --2- 172.21.15.37:0/3687672595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e0103260 0x7f42e0109af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 --2- 172.21.15.37:0/3687672595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e01028a0 0x7f42e0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 --2- 172.21.15.37:0/3687672595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e01016b0 0x7f42e0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 -- 172.21.15.37:0/3687672595 >> 172.21.15.37:0/3687672595 conn(0x7f42e00fcc20 msgr2=0x7f42e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 -- 172.21.15.37:0/3687672595 shutdown_connections 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.109+0000 7f42e7200640 1 -- 172.21.15.37:0/3687672595 wait complete. 2024-03-21T09:08:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.110+0000 7f42e7200640 1 Processor -- start 2024-03-21T09:08:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.110+0000 7f42e7200640 1 -- start start 2024-03-21T09:08:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.110+0000 7f42e7200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e01016b0 0x7f42e01979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42e7200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 0x7f42e0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42e7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0103260 0x7f42e019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42e7200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e010d330 con 0x7f42e01028a0 2024-03-21T09:08:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42e7200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e010d1b0 con 0x7f42e01016b0 2024-03-21T09:08:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42e7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e010d4b0 con 0x7f42e0103260 2024-03-21T09:08:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 0x7f42e0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e01016b0 0x7f42e01979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 0x7f42e0197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37254/0 (socket says 172.21.15.37:37254) 2024-03-21T09:08:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 -- 172.21.15.37:0/536713505 learned_addr learned my addr 172.21.15.37:0/536713505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42e5200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0103260 0x7f42e019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 -- 172.21.15.37:0/536713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0103260 msgr2=0x7f42e019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0103260 0x7f42e019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 -- 172.21.15.37:0/536713505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e01016b0 msgr2=0x7f42e01979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e01016b0 0x7f42e01979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.111+0000 7f42df400640 1 -- 172.21.15.37:0/536713505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e019f600 con 0x7f42e01028a0 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.112+0000 7f42df400640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 0x7f42e0197f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00e9f0 tx=0x7f42cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.112+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00cd80 con 0x7f42e01028a0 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.112+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42cc005500 con 0x7f42e01028a0 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.112+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc010690 con 0x7f42e01028a0 2024-03-21T09:08:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.112+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e019f8f0 con 0x7f42e01028a0 2024-03-21T09:08:47.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.112+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e019fe40 con 0x7f42e01028a0 2024-03-21T09:08:47.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.114+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0108970 con 0x7f42e01028a0 2024-03-21T09:08:47.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.116+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f42cc0046c0 con 0x7f42e01028a0 2024-03-21T09:08:47.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.118+0000 7f42dcc00640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f42c40a2fa0 0x7f42c40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:47.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.118+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f42cc0cf6a0 con 0x7f42e01028a0 2024-03-21T09:08:47.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.118+0000 7f42dfe00640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f42c40a2fa0 0x7f42c40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:47.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.119+0000 7f42dfe00640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f42c40a2fa0 0x7f42c40a5460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42e0102700 tx=0x7f42d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:47.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.123+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42cc09c580 con 0x7f42e01028a0 2024-03-21T09:08:47.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2469984482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T09:08:47.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: pgmap v313: 33 pgs: 12 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1094521301' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T09:08:47.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:46 smithi053 ceph-mon[36169]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-03-21T09:08:47.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.358+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f42e002d070 con 0x7f42e01028a0 2024-03-21T09:08:47.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.796+0000 7f42dcc00640 1 -- 172.21.15.37:0/536713505 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) v1 ==== 157+0+0 (secure 0 0 0) 0x7f42cc09be80 con 0x7f42e01028a0 2024-03-21T09:08:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f42c40a2fa0 msgr2=0x7f42c40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f42c40a2fa0 0x7f42c40a5460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42e0102700 tx=0x7f42d0005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 msgr2=0x7f42e0197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:47.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 0x7f42e0197f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00e9f0 tx=0x7f42cc00edf0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 shutdown_connections 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f42c40a2fa0 0x7f42c40a5460 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42e0102700 tx=0x7f42d0005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0103260 0x7f42e019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e01028a0 0x7f42e0197f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 --2- 172.21.15.37:0/536713505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e01016b0 0x7f42e01979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.801+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 >> 172.21.15.37:0/536713505 conn(0x7f42e00fcc20 msgr2=0x7f42e0105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.802+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 shutdown_connections 2024-03-21T09:08:47.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:47.802+0000 7f42e7200640 1 -- 172.21.15.37:0/536713505 wait complete. 2024-03-21T09:08:47.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-21T09:08:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 -- 172.21.15.37:0/4056696958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec1071a0 msgr2=0x7fcdec1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/4056696958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec1071a0 0x7fcdec1095f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009be0 tx=0x7fcde402bef0 comp rx=0 tx=0).stop 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 -- 172.21.15.37:0/4056696958 shutdown_connections 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/4056696958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec1071a0 0x7fcdec1095f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/4056696958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec104870 0x7fcdec106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/4056696958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec101f40 0x7fcdec104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 -- 172.21.15.37:0/4056696958 >> 172.21.15.37:0/4056696958 conn(0x7fcdec0fb930 msgr2=0x7fcdec0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 -- 172.21.15.37:0/4056696958 shutdown_connections 2024-03-21T09:08:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.036+0000 7fcdf2e00640 1 -- 172.21.15.37:0/4056696958 wait complete. 2024-03-21T09:08:48.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.037+0000 7fcdf2e00640 1 Processor -- start 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.037+0000 7fcdf2e00640 1 -- start start 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec101f40 0x7fcdec199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec104870 0x7fcdec199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec1071a0 0x7fcdec1a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec10d150 con 0x7fcdec101f40 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec10cfd0 con 0x7fcdec1071a0 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec10d2d0 con 0x7fcdec104870 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec101f40 0x7fcdec199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec1071a0 0x7fcdec1a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec101f40 0x7fcdec199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37276/0 (socket says 172.21.15.37:37276) 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf1a00640 1 -- 172.21.15.37:0/1997025167 learned_addr learned my addr 172.21.15.37:0/1997025167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf1000640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec104870 0x7fcdec199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2400640 1 -- 172.21.15.37:0/1997025167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec104870 msgr2=0x7fcdec199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2400640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec104870 0x7fcdec199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2400640 1 -- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec101f40 msgr2=0x7fcdec199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2400640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec101f40 0x7fcdec199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.038+0000 7fcdf2400640 1 -- 172.21.15.37:0/1997025167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec1a1530 con 0x7fcdec1071a0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.039+0000 7fcdf2400640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec1071a0 0x7fcdec1a0ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcde402c430 tx=0x7fcde402cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.039+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4043020 con 0x7fcdec1071a0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.039+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec1a17c0 con 0x7fcdec1071a0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.039+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcde4003e50 con 0x7fcdec1071a0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.039+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4034430 con 0x7fcdec1071a0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.039+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1a1d10 con 0x7fcdec1071a0 2024-03-21T09:08:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.040+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec108c90 con 0x7fcdec1071a0 2024-03-21T09:08:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.044+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcde4046020 con 0x7fcdec1071a0 2024-03-21T09:08:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.046+0000 7fcdea000640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdd40a2f10 0x7fcdd40a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.046+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fcde40f2160 con 0x7fcdec1071a0 2024-03-21T09:08:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.046+0000 7fcdf1a00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdd40a2f10 0x7fcdd40a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.046+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcde404d020 con 0x7fcdec1071a0 2024-03-21T09:08:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.047+0000 7fcdf1a00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdd40a2f10 0x7fcdd40a53d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009cd0 tx=0x7fcde0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:48.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:47 smithi071 ceph-mon[36597]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T09:08:48.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:47 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T09:08:48.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:47 smithi071 ceph-mon[36597]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-03-21T09:08:48.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:47 smithi071 ceph-mon[36597]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-03-21T09:08:48.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:47 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/536713505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T09:08:48.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:47 smithi037 ceph-mon[31830]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T09:08:48.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:47 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T09:08:48.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:47 smithi037 ceph-mon[31830]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-03-21T09:08:48.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:47 smithi037 ceph-mon[31830]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-03-21T09:08:48.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:47 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/536713505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T09:08:48.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:47 smithi053 ceph-mon[36169]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T09:08:48.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:47 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T09:08:48.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:47 smithi053 ceph-mon[36169]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-03-21T09:08:48.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:47 smithi053 ceph-mon[36169]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-03-21T09:08:48.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:47 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/536713505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T09:08:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:48.295+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcdec0622a0 con 0x7fcdec1071a0 2024-03-21T09:08:49.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: pgmap v316: 97 pgs: 18 creating+peering, 60 unknown, 19 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/536713505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: Cluster is now healthy 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] up:active 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1997025167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T09:08:49.103 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:48 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T09:08:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: pgmap v316: 97 pgs: 18 creating+peering, 60 unknown, 19 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-21T09:08:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/536713505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T09:08:49.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: Cluster is now healthy 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] up:active 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1997025167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T09:08:49.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:48 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T09:08:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: pgmap v316: 97 pgs: 18 creating+peering, 60 unknown, 19 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-21T09:08:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/536713505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T09:08:49.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: Cluster is now healthy 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] up:active 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1997025167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T09:08:49.192 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:48 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T09:08:49.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.934+0000 7fcdea000640 1 -- 172.21.15.37:0/1997025167 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) v1 ==== 228+0+0 (secure 0 0 0) 0x7fcde40bef00 con 0x7fcdec1071a0 2024-03-21T09:08:49.930 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' created 2024-03-21T09:08:49.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.941+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdd40a2f10 msgr2=0x7fcdd40a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:49.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.941+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdd40a2f10 0x7fcdd40a53d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009cd0 tx=0x7fcde0005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:49.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.941+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec1071a0 msgr2=0x7fcdec1a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:49.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.941+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec1071a0 0x7fcdec1a0ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcde402c430 tx=0x7fcde402cb90 comp rx=0 tx=0).stop 2024-03-21T09:08:49.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 shutdown_connections 2024-03-21T09:08:49.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdd40a2f10 0x7fcdd40a53d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:49.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdec1071a0 0x7fcdec1a0ec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:49.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec104870 0x7fcdec199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:49.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 --2- 172.21.15.37:0/1997025167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdec101f40 0x7fcdec199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:49.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 >> 172.21.15.37:0/1997025167 conn(0x7fcdec0fb930 msgr2=0x7fcdec101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:49.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.942+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 shutdown_connections 2024-03-21T09:08:49.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:49.943+0000 7fcdf2e00640 1 -- 172.21.15.37:0/1997025167 wait complete. 2024-03-21T09:08:49.963 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-21T09:08:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 -- 172.21.15.37:0/315892311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f08105080 msgr2=0x7f3f08105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 --2- 172.21.15.37:0/315892311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f08105080 0x7f3f08105500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009ca0 tx=0x7f3f0402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 -- 172.21.15.37:0/315892311 shutdown_connections 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 --2- 172.21.15.37:0/315892311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f08105080 0x7f3f08105500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 --2- 172.21.15.37:0/315892311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f081026d0 0x7f3f08104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 --2- 172.21.15.37:0/315892311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 0x7f3f08102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 -- 172.21.15.37:0/315892311 >> 172.21.15.37:0/315892311 conn(0x7f3f080fb980 msgr2=0x7f3f080fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 -- 172.21.15.37:0/315892311 shutdown_connections 2024-03-21T09:08:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.172+0000 7f3f0f200640 1 -- 172.21.15.37:0/315892311 wait complete. 2024-03-21T09:08:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.173+0000 7f3f0f200640 1 Processor -- start 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.173+0000 7f3f0f200640 1 -- start start 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 0x7f3f0819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f081026d0 0x7f3f0819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 0x7f3f081a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08113740 con 0x7f3f08105080 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f081135c0 con 0x7f3f080ffd60 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f081138c0 con 0x7f3f081026d0 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 0x7f3f081a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 0x7f3f0819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f081026d0 0x7f3f0819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 0x7f3f081a5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37296/0 (socket says 172.21.15.37:37296) 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.174+0000 7f3f0e800640 1 -- 172.21.15.37:0/1452736866 learned_addr learned my addr 172.21.15.37:0/1452736866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0de00640 1 -- 172.21.15.37:0/1452736866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f081026d0 msgr2=0x7f3f0819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0de00640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f081026d0 0x7f3f0819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0de00640 1 -- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 msgr2=0x7f3f081a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0de00640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 0x7f3f081a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0de00640 1 -- 172.21.15.37:0/1452736866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f081a5970 con 0x7f3f080ffd60 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0d400640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f081026d0 0x7f3f0819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0e800640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 0x7f3f081a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0de00640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 0x7f3f0819dd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00e9f0 tx=0x7f3efc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00ccc0 con 0x7f3f080ffd60 2024-03-21T09:08:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f081a5c60 con 0x7f3f080ffd60 2024-03-21T09:08:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3efc010910 con 0x7f3f080ffd60 2024-03-21T09:08:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc018960 con 0x7f3f080ffd60 2024-03-21T09:08:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.175+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f081a61b0 con 0x7f3f080ffd60 2024-03-21T09:08:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.176+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f080fd470 con 0x7f3f080ffd60 2024-03-21T09:08:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.179+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3efc010ac0 con 0x7f3f080ffd60 2024-03-21T09:08:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.180+0000 7f3f02a00640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ef00a2f20 0x7f3ef00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.180+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f3efc0cf9f0 con 0x7f3f080ffd60 2024-03-21T09:08:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.181+0000 7f3f0d400640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ef00a2f20 0x7f3ef00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.181+0000 7f3f0d400640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ef00a2f20 0x7f3ef00a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3f0819f260 tx=0x7f3ef8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:50.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.185+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3efc09bf50 con 0x7f3f080ffd60 2024-03-21T09:08:50.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:49 smithi053 ceph-mon[36169]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T09:08:50.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:49 smithi071 ceph-mon[36597]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T09:08:50.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.385+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f3f08062340 con 0x7f3f080ffd60 2024-03-21T09:08:50.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:49 smithi037 ceph-mon[31830]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T09:08:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.939+0000 7f3f02a00640 1 -- 172.21.15.37:0/1452736866 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3efc09fff0 con 0x7f3f080ffd60 2024-03-21T09:08:50.933 INFO:teuthology.orchestra.run.smithi037.stderr:set pool 4 allow_ec_overwrites to true 2024-03-21T09:08:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.945+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ef00a2f20 msgr2=0x7f3ef00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.945+0000 7f3f0f200640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ef00a2f20 0x7f3ef00a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3f0819f260 tx=0x7f3ef8005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.945+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 msgr2=0x7f3f0819dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.945+0000 7f3f0f200640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 0x7f3f0819dd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00e9f0 tx=0x7f3efc00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.947+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 shutdown_connections 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.947+0000 7f3f0f200640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ef00a2f20 0x7f3ef00a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.947+0000 7f3f0f200640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f08105080 0x7f3f081a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.947+0000 7f3f0f200640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f081026d0 0x7f3f0819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.947+0000 7f3f0f200640 1 --2- 172.21.15.37:0/1452736866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080ffd60 0x7f3f0819dd50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.947+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 >> 172.21.15.37:0/1452736866 conn(0x7f3f080fb980 msgr2=0x7f3f08100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.948+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 shutdown_connections 2024-03-21T09:08:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:50.948+0000 7f3f0f200640 1 -- 172.21.15.37:0/1452736866 wait complete. 2024-03-21T09:08:50.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 -- 172.21.15.37:0/629367910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9390103260 msgr2=0x7f9390109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 --2- 172.21.15.37:0/629367910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9390103260 0x7f9390109af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9388009ca0 tx=0x7f938802c100 comp rx=0 tx=0).stop 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 -- 172.21.15.37:0/629367910 shutdown_connections 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 --2- 172.21.15.37:0/629367910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9390103260 0x7f9390109af0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 --2- 172.21.15.37:0/629367910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93901028a0 0x7f9390102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 --2- 172.21.15.37:0/629367910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901016b0 0x7f9390101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.180+0000 7f9397e00640 1 -- 172.21.15.37:0/629367910 >> 172.21.15.37:0/629367910 conn(0x7f93900fcc20 msgr2=0x7f93900ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.181+0000 7f9397e00640 1 -- 172.21.15.37:0/629367910 shutdown_connections 2024-03-21T09:08:51.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.181+0000 7f9397e00640 1 -- 172.21.15.37:0/629367910 wait complete. 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 Processor -- start 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 -- start start 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 0x7f9390197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901028a0 0x7f9390197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9390103260 0x7f939019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939010d330 con 0x7f93901016b0 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939010d1b0 con 0x7f9390103260 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9397e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939010d4b0 con 0x7f93901028a0 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9395400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 0x7f9390197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9395400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 0x7f9390197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37316/0 (socket says 172.21.15.37:37316) 2024-03-21T09:08:51.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9395400640 1 -- 172.21.15.37:0/1419564599 learned_addr learned my addr 172.21.15.37:0/1419564599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9394a00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901028a0 0x7f9390197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.182+0000 7f9395e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9390103260 0x7f939019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9395400640 1 -- 172.21.15.37:0/1419564599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901028a0 msgr2=0x7f9390197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9395400640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901028a0 0x7f9390197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9395400640 1 -- 172.21.15.37:0/1419564599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9390103260 msgr2=0x7f939019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9395400640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9390103260 0x7f939019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9395400640 1 -- 172.21.15.37:0/1419564599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939019f590 con 0x7f93901016b0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9394a00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901028a0 0x7f9390197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f9395400640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 0x7f9390197970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f938400e9c0 tx=0x7f938400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.183+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938400cd80 con 0x7f93901016b0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.184+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93840035f0 con 0x7f93901016b0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.184+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939019f880 con 0x7f93901016b0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.184+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384010690 con 0x7f93901016b0 2024-03-21T09:08:51.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.184+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939019fd80 con 0x7f93901016b0 2024-03-21T09:08:51.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.185+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9390108970 con 0x7f93901016b0 2024-03-21T09:08:51.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.187+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f93840037a0 con 0x7f93901016b0 2024-03-21T09:08:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.188+0000 7f938e000640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f93780a2f20 0x7f93780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.189+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f93840cfa60 con 0x7f93901016b0 2024-03-21T09:08:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.189+0000 7f9394a00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f93780a2f20 0x7f93780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:51.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.189+0000 7f9394a00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f93780a2f20 0x7f93780a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f937c0095f0 tx=0x7f937c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:51.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.192+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f938409c610 con 0x7f93901016b0 2024-03-21T09:08:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:50 smithi071 ceph-mon[36597]: pgmap v319: 97 pgs: 18 creating+peering, 14 unknown, 65 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-03-21T09:08:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:50 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T09:08:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:50 smithi071 ceph-mon[36597]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T09:08:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:50 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1452736866' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T09:08:51.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:50 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T09:08:51.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:50 smithi037 ceph-mon[31830]: pgmap v319: 97 pgs: 18 creating+peering, 14 unknown, 65 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-03-21T09:08:51.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:50 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T09:08:51.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:50 smithi037 ceph-mon[31830]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T09:08:51.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:50 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1452736866' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T09:08:51.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:50 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T09:08:51.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.440+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f939002d290 con 0x7f93901016b0 2024-03-21T09:08:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:50 smithi053 ceph-mon[36169]: pgmap v319: 97 pgs: 18 creating+peering, 14 unknown, 65 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-03-21T09:08:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:50 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T09:08:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:50 smithi053 ceph-mon[36169]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T09:08:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:50 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1452736866' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T09:08:51.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:50 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T09:08:51.923 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:08:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: 2024-03-21T09:08:51.660+0000 7f4901000700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-21T09:08:51.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.956+0000 7f938e000640 1 -- 172.21.15.37:0/1419564599 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f938409bf10 con 0x7f93901016b0 2024-03-21T09:08:51.950 INFO:teuthology.orchestra.run.smithi037.stderr:added data pool 4 to fsmap 2024-03-21T09:08:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.961+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f93780a2f20 msgr2=0x7f93780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.961+0000 7f9397e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f93780a2f20 0x7f93780a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f937c0095f0 tx=0x7f937c005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.961+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 msgr2=0x7f9390197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.961+0000 7f9397e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 0x7f9390197970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f938400e9c0 tx=0x7f938400ee80 comp rx=0 tx=0).stop 2024-03-21T09:08:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 shutdown_connections 2024-03-21T09:08:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f93780a2f20 0x7f93780a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9390103260 0x7f939019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93901028a0 0x7f9390197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 --2- 172.21.15.37:0/1419564599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93901016b0 0x7f9390197970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 >> 172.21.15.37:0/1419564599 conn(0x7f93900fcc20 msgr2=0x7f9390105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:51.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.962+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 shutdown_connections 2024-03-21T09:08:51.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:51.963+0000 7f9397e00640 1 -- 172.21.15.37:0/1419564599 wait complete. 2024-03-21T09:08:51.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:52.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 -- 172.21.15.37:0/4082092220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 msgr2=0x7f1ca0100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 --2- 172.21.15.37:0/4082092220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0100240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009900 tx=0x7f1c9002bc40 comp rx=0 tx=0).stop 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 -- 172.21.15.37:0/4082092220 shutdown_connections 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 --2- 172.21.15.37:0/4082092220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0101120 0x7f1ca01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 --2- 172.21.15.37:0/4082092220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0100780 0x7f1ca0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 --2- 172.21.15.37:0/4082092220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0100240 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 -- 172.21.15.37:0/4082092220 >> 172.21.15.37:0/4082092220 conn(0x7f1ca00fba20 msgr2=0x7f1ca00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.204+0000 7f1ca5600640 1 -- 172.21.15.37:0/4082092220 shutdown_connections 2024-03-21T09:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.205+0000 7f1ca5600640 1 -- 172.21.15.37:0/4082092220 wait complete. 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 Processor -- start 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 -- start start 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0100780 0x7f1ca0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 0x7f1ca01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca010f3f0 con 0x7f1ca0100780 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca010f270 con 0x7f1ca0101120 2024-03-21T09:08:52.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.206+0000 7f1ca5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca010f570 con 0x7f1ca00ffe20 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1c9f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0100780 0x7f1ca0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 0x7f1ca01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 0x7f1ca01a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54482/0 (socket says 172.21.15.37:54482) 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 -- 172.21.15.37:0/3889694001 learned_addr learned my addr 172.21.15.37:0/3889694001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1c9fe00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 -- 172.21.15.37:0/3889694001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 msgr2=0x7f1ca0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 -- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0100780 msgr2=0x7f1ca0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0100780 0x7f1ca0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 -- 172.21.15.37:0/3889694001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca01a16b0 con 0x7f1ca0101120 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1c9fe00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1c9f400640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0100780 0x7f1ca0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.207+0000 7f1ca4c00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 0x7f1ca01a1040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800eb10 tx=0x7f1c9800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:52.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.208+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98004b20 con 0x7f1ca0101120 2024-03-21T09:08:52.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.208+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c98004cc0 con 0x7f1ca0101120 2024-03-21T09:08:52.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.208+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98010760 con 0x7f1ca0101120 2024-03-21T09:08:52.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.208+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca01a19a0 con 0x7f1ca0101120 2024-03-21T09:08:52.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.208+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca01a1ea0 con 0x7f1ca0101120 2024-03-21T09:08:52.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.209+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca010ab70 con 0x7f1ca0101120 2024-03-21T09:08:52.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.212+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1c98005020 con 0x7f1ca0101120 2024-03-21T09:08:52.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.213+0000 7f1c9cc00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1c840a2f10 0x7f1c840a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.213+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1c980cfbe0 con 0x7f1ca0101120 2024-03-21T09:08:52.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.213+0000 7f1c9fe00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1c840a2f10 0x7f1c840a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.214+0000 7f1c9fe00640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1c840a2f10 0x7f1c840a53d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002c7c0 tx=0x7f1c90005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:52.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.217+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c9809c770 con 0x7f1ca0101120 2024-03-21T09:08:52.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:51 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T09:08:52.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:51 smithi071 ceph-mon[36597]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T09:08:52.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:51 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1419564599' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T09:08:52.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:51 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T09:08:52.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:51 smithi037 ceph-mon[31830]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T09:08:52.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:51 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1419564599' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T09:08:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:51 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T09:08:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:51 smithi053 ceph-mon[36169]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T09:08:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:51 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1419564599' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T09:08:52.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.462+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ca002d070 con 0x7f1ca0101120 2024-03-21T09:08:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.463+0000 7f1c9cc00640 1 -- 172.21.15.37:0/3889694001 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11300 (secure 0 0 0) 0x7f1c9809c070 con 0x7f1ca0101120 2024-03-21T09:08:52.458 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:52.458 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:08:51.441981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24589,"qdb_cluster":[24589]},"id":1}]} 2024-03-21T09:08:52.458 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1c840a2f10 msgr2=0x7f1c840a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1c840a2f10 0x7f1c840a53d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002c7c0 tx=0x7f1c90005fd0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 msgr2=0x7f1ca01a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 0x7f1ca01a1040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800eb10 tx=0x7f1c9800c2d0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 shutdown_connections 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1c840a2f10 0x7f1c840a53d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ca0101120 0x7f1ca01a1040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca0100780 0x7f1ca0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 --2- 172.21.15.37:0/3889694001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ca00ffe20 0x7f1ca0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 >> 172.21.15.37:0/3889694001 conn(0x7f1ca00fba20 msgr2=0x7f1ca00fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:52.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 shutdown_connections 2024-03-21T09:08:52.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.467+0000 7f1ca5600640 1 -- 172.21.15.37:0/3889694001 wait complete. 2024-03-21T09:08:52.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:08:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 -- 172.21.15.37:0/2614664315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 msgr2=0x7f1308104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 --2- 172.21.15.37:0/2614664315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 0x7f1308104e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009900 tx=0x7f12f002bc30 comp rx=0 tx=0).stop 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 -- 172.21.15.37:0/2614664315 shutdown_connections 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 --2- 172.21.15.37:0/2614664315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1308105370 0x7f130810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 --2- 172.21.15.37:0/2614664315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 0x7f1308104e30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 --2- 172.21.15.37:0/2614664315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13081037c0 0x7f1308103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.706+0000 7f1307200640 1 -- 172.21.15.37:0/2614664315 >> 172.21.15.37:0/2614664315 conn(0x7f13080fed50 msgr2=0x7f1308101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.707+0000 7f1307200640 1 -- 172.21.15.37:0/2614664315 shutdown_connections 2024-03-21T09:08:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.707+0000 7f1307200640 1 -- 172.21.15.37:0/2614664315 wait complete. 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 Processor -- start 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 -- start start 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 0x7f1308199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 0x7f1308199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1308105370 0x7f13081a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130810f490 con 0x7f13081049b0 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130810f310 con 0x7f13081037c0 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.708+0000 7f1307200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130810f610 con 0x7f1308105370 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 0x7f1308199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f12ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 0x7f1308199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 0x7f1308199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54512/0 (socket says 172.21.15.37:54512) 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 -- 172.21.15.37:0/4922467 learned_addr learned my addr 172.21.15.37:0/4922467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1306800640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1308105370 0x7f13081a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 -- 172.21.15.37:0/4922467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1308105370 msgr2=0x7f13081a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1308105370 0x7f13081a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 -- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 msgr2=0x7f1308199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 0x7f1308199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 -- 172.21.15.37:0/4922467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13081a16c0 con 0x7f13081037c0 2024-03-21T09:08:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.709+0000 7f1305e00640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 0x7f1308199aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9c0 tx=0x7f12f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.710+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800cd80 con 0x7f13081037c0 2024-03-21T09:08:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.710+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13081a19b0 con 0x7f13081037c0 2024-03-21T09:08:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.710+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12f80035f0 con 0x7f13081037c0 2024-03-21T09:08:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.710+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8010690 con 0x7f13081037c0 2024-03-21T09:08:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.710+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13081a1ed0 con 0x7f13081037c0 2024-03-21T09:08:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.711+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130804f470 con 0x7f13081037c0 2024-03-21T09:08:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.713+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f12f80037a0 con 0x7f13081037c0 2024-03-21T09:08:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.714+0000 7f12fea00640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12ec0a2f10 0x7f12ec0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.714+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f12f80cfa30 con 0x7f13081037c0 2024-03-21T09:08:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.715+0000 7f12ffe00640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12ec0a2f10 0x7f12ec0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:52.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.715+0000 7f12ffe00640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12ec0a2f10 0x7f12ec0a53d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f002c240 tx=0x7f12f002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.718+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12f809c5c0 con 0x7f13081037c0 2024-03-21T09:08:52.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.925+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13080622a0 con 0x7f13081037c0 2024-03-21T09:08:52.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.927+0000 7f12fea00640 1 -- 172.21.15.37:0/4922467 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f12f809bec0 con 0x7f13081037c0 2024-03-21T09:08:52.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:52.921 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.929+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12ec0a2f10 msgr2=0x7f12ec0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.929+0000 7f1307200640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12ec0a2f10 0x7f12ec0a53d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f002c240 tx=0x7f12f002df90 comp rx=0 tx=0).stop 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 msgr2=0x7f1308199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 0x7f1308199aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9c0 tx=0x7f12f800ee80 comp rx=0 tx=0).stop 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 shutdown_connections 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12ec0a2f10 0x7f12ec0a53d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1308105370 0x7f13081a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081049b0 0x7f1308199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 --2- 172.21.15.37:0/4922467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13081037c0 0x7f1308199aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 >> 172.21.15.37:0/4922467 conn(0x7f13080fed50 msgr2=0x7f1308107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 shutdown_connections 2024-03-21T09:08:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:52.930+0000 7f1307200640 1 -- 172.21.15.37:0/4922467 wait complete. 2024-03-21T09:08:52.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:08:53.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 -- 172.21.15.37:0/683824082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 msgr2=0x7fcdc8110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 --2- 172.21.15.37:0/683824082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc8110000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009ca0 tx=0x7fcdc002c100 comp rx=0 tx=0).stop 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 -- 172.21.15.37:0/683824082 shutdown_connections 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 --2- 172.21.15.37:0/683824082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc8110000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 --2- 172.21.15.37:0/683824082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 0x7fcdc8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 --2- 172.21.15.37:0/683824082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 -- 172.21.15.37:0/683824082 >> 172.21.15.37:0/683824082 conn(0x7fcdc8076b60 msgr2=0x7fcdc8078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 -- 172.21.15.37:0/683824082 shutdown_connections 2024-03-21T09:08:53.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.170+0000 7fcdcea00640 1 -- 172.21.15.37:0/683824082 wait complete. 2024-03-21T09:08:53.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.172+0000 7fcdcea00640 1 Processor -- start 2024-03-21T09:08:53.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.172+0000 7fcdcea00640 1 -- start start 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.172+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 0x7fcdc819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc81a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8113820 con 0x7fcdc8109770 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81136a0 con 0x7fcdc8107bc0 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81139a0 con 0x7fcdc8108db0 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54530/0 (socket says 172.21.15.37:54530) 2024-03-21T09:08:53.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcd600640 1 -- 172.21.15.37:0/629475083 learned_addr learned my addr 172.21.15.37:0/629475083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdce000640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc81a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdccc00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 0x7fcdc819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdce000640 1 -- 172.21.15.37:0/629475083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 msgr2=0x7fcdc819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdce000640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 0x7fcdc819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdce000640 1 -- 172.21.15.37:0/629475083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 msgr2=0x7fcdc819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdce000640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdce000640 1 -- 172.21.15.37:0/629475083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc81a5a60 con 0x7fcdc8109770 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdcd600640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.173+0000 7fcdccc00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 0x7fcdc819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.174+0000 7fcdce000640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc81a53f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002c710 tx=0x7fcdc0035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.174+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0039040 con 0x7fcdc8109770 2024-03-21T09:08:53.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.174+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc81a5cf0 con 0x7fcdc8109770 2024-03-21T09:08:53.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.174+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdc0044070 con 0x7fcdc8109770 2024-03-21T09:08:53.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.174+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0004470 con 0x7fcdc8109770 2024-03-21T09:08:53.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.174+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc81a6180 con 0x7fcdc8109770 2024-03-21T09:08:53.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.175+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc810ee80 con 0x7fcdc8109770 2024-03-21T09:08:53.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.177+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcdc0004610 con 0x7fcdc8109770 2024-03-21T09:08:53.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.179+0000 7fcdc6000640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdb00a2f10 0x7fcdb00a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.179+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcdc00f1b40 con 0x7fcdc8109770 2024-03-21T09:08:53.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.179+0000 7fcdcd600640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdb00a2f10 0x7fcdb00a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.180+0000 7fcdcd600640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdb00a2f10 0x7fcdb00a53d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0058c0 tx=0x7fcdbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:53.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.185+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdc004e020 con 0x7fcdc8109770 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: pgmap v322: 129 pgs: 1 creating+peering, 30 unknown, 98 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1419564599' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3889694001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:52 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4922467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.410+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdc80622a0 con 0x7fcdc8109770 2024-03-21T09:08:53.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.414+0000 7fcdc6000640 1 -- 172.21.15.37:0/629475083 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7fcdc00be6d0 con 0x7fcdc8109770 2024-03-21T09:08:53.408 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:53.408 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdb00a2f10 msgr2=0x7fcdb00a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdb00a2f10 0x7fcdb00a53d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0058c0 tx=0x7fcdbc005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 msgr2=0x7fcdc81a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc81a53f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002c710 tx=0x7fcdc0035d90 comp rx=0 tx=0).stop 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 shutdown_connections 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcdb00a2f10 0x7fcdb00a53d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdc8109770 0x7fcdc81a53f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc8108db0 0x7fcdc819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 --2- 172.21.15.37:0/629475083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc8107bc0 0x7fcdc819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 >> 172.21.15.37:0/629475083 conn(0x7fcdc8076b60 msgr2=0x7fcdc810c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:53.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 shutdown_connections 2024-03-21T09:08:53.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.417+0000 7fcdcea00640 1 -- 172.21.15.37:0/629475083 wait complete. 2024-03-21T09:08:53.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: pgmap v322: 129 pgs: 1 creating+peering, 30 unknown, 98 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-21T09:08:53.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T09:08:53.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1419564599' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T09:08:53.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T09:08:53.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:53.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3889694001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:53.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:52 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4922467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:53.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:08:53.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: pgmap v322: 129 pgs: 1 creating+peering, 30 unknown, 98 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-21T09:08:53.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T09:08:53.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1419564599' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T09:08:53.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T09:08:53.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:08:53.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3889694001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:53.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:52 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4922467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:53.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 -- 172.21.15.37:0/1447715133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d01021f0 msgr2=0x7ff6d0108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 --2- 172.21.15.37:0/1447715133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d01021f0 0x7ff6d0108a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009be0 tx=0x7ff6c802bef0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 -- 172.21.15.37:0/1447715133 shutdown_connections 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 --2- 172.21.15.37:0/1447715133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d01021f0 0x7ff6d0108a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 --2- 172.21.15.37:0/1447715133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0101830 0x7ff6d0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 --2- 172.21.15.37:0/1447715133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d0100640 0x7ff6d0100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 -- 172.21.15.37:0/1447715133 >> 172.21.15.37:0/1447715133 conn(0x7ff6d00fbbb0 msgr2=0x7ff6d00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 -- 172.21.15.37:0/1447715133 shutdown_connections 2024-03-21T09:08:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.671+0000 7ff6d5600640 1 -- 172.21.15.37:0/1447715133 wait complete. 2024-03-21T09:08:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.673+0000 7ff6d5600640 1 Processor -- start 2024-03-21T09:08:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.673+0000 7ff6d5600640 1 -- start start 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.673+0000 7ff6d5600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 0x7ff6d006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.673+0000 7ff6d5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d0101830 0x7ff6d019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6d5600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d01021f0 0x7ff6d019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6d5600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d010c280 con 0x7ff6d01021f0 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6d5600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d010c100 con 0x7ff6d0100640 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6d5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d010c400 con 0x7ff6d0101830 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 0x7ff6d006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 0x7ff6d006a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54548/0 (socket says 172.21.15.37:54548) 2024-03-21T09:08:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 -- 172.21.15.37:0/2588645170 learned_addr learned my addr 172.21.15.37:0/2588645170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cf400640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d01021f0 0x7ff6d019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6ce000640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d0101830 0x7ff6d019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 -- 172.21.15.37:0/2588645170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d0101830 msgr2=0x7ff6d019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d0101830 0x7ff6d019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 -- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d01021f0 msgr2=0x7ff6d019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d01021f0 0x7ff6d019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.674+0000 7ff6cea00640 1 -- 172.21.15.37:0/2588645170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d019f550 con 0x7ff6d0100640 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.675+0000 7ff6cea00640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 0x7ff6d006a8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000e9f0 tx=0x7ff6c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.675+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000ccc0 con 0x7ff6d0100640 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.675+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d0196450 con 0x7ff6d0100640 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.675+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6c00035f0 con 0x7ff6d0100640 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.675+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000ce60 con 0x7ff6d0100640 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.675+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d01969f0 con 0x7ff6d0100640 2024-03-21T09:08:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.676+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d0107900 con 0x7ff6d0100640 2024-03-21T09:08:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.678+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff6c00037a0 con 0x7ff6d0100640 2024-03-21T09:08:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.680+0000 7ff6c7400640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff6b40a2fa0 0x7ff6b40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.680+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff6c00cf860 con 0x7ff6d0100640 2024-03-21T09:08:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.680+0000 7ff6ce000640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff6b40a2fa0 0x7ff6b40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.681+0000 7ff6ce000640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff6b40a2fa0 0x7ff6b40a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff6d006a660 tx=0x7ff6bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.682+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6c009c3f0 con 0x7ff6d0100640 2024-03-21T09:08:53.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.932+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6d002d070 con 0x7ff6d0100640 2024-03-21T09:08:53.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.933+0000 7ff6c7400640 1 -- 172.21.15.37:0/2588645170 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11300 (secure 0 0 0) 0x7ff6c009bcf0 con 0x7ff6d0100640 2024-03-21T09:08:53.927 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:08:51.441981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24589,"qdb_cluster":[24589]},"id":1}]} 2024-03-21T09:08:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-03-21T09:08:53.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff6b40a2fa0 msgr2=0x7ff6b40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff6b40a2fa0 0x7ff6b40a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff6d006a660 tx=0x7ff6bc005f90 comp rx=0 tx=0).stop 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 msgr2=0x7ff6d006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 0x7ff6d006a8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000e9f0 tx=0x7ff6c000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 shutdown_connections 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff6b40a2fa0 0x7ff6b40a5460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d01021f0 0x7ff6d019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d0101830 0x7ff6d019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 --2- 172.21.15.37:0/2588645170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6d0100640 0x7ff6d006a8f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 >> 172.21.15.37:0/2588645170 conn(0x7ff6d00fbbb0 msgr2=0x7ff6d0104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 shutdown_connections 2024-03-21T09:08:53.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:53.937+0000 7ff6d5600640 1 -- 172.21.15.37:0/2588645170 wait complete. 2024-03-21T09:08:53.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:08:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 -- 172.21.15.37:0/918908777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 msgr2=0x7fb614104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 --2- 172.21.15.37:0/918908777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614104480 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb608009900 tx=0x7fb60802bc40 comp rx=0 tx=0).stop 2024-03-21T09:08:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 -- 172.21.15.37:0/918908777 shutdown_connections 2024-03-21T09:08:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 --2- 172.21.15.37:0/918908777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141072f0 0x7fb6141096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 --2- 172.21.15.37:0/918908777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6141049c0 0x7fb614106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 --2- 172.21.15.37:0/918908777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614104480 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 -- 172.21.15.37:0/918908777 >> 172.21.15.37:0/918908777 conn(0x7fb6140fba20 msgr2=0x7fb6140fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 -- 172.21.15.37:0/918908777 shutdown_connections 2024-03-21T09:08:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.181+0000 7fb61a600640 1 -- 172.21.15.37:0/918908777 wait complete. 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 Processor -- start 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 -- start start 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141049c0 0x7fb614199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6141072f0 0x7fb6141a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61410d270 con 0x7fb614102090 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61410d0f0 con 0x7fb6141072f0 2024-03-21T09:08:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.182+0000 7fb61a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61410d3f0 con 0x7fb6141049c0 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb619200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb619200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51202/0 (socket says 172.21.15.37:51202) 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb619200640 1 -- 172.21.15.37:0/432224885 learned_addr learned my addr 172.21.15.37:0/432224885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb613e00640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141049c0 0x7fb614199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb619c00640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6141072f0 0x7fb6141a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb613e00640 1 -- 172.21.15.37:0/432224885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6141072f0 msgr2=0x7fb6141a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb613e00640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6141072f0 0x7fb6141a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb613e00640 1 -- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 msgr2=0x7fb614199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb613e00640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb613e00640 1 -- 172.21.15.37:0/432224885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6141a1620 con 0x7fb6141049c0 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.183+0000 7fb619200640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:54.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.184+0000 7fb613e00640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141049c0 0x7fb614199f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c940 tx=0x7fb60000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.184+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600007bb0 con 0x7fb6141049c0 2024-03-21T09:08:54.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.184+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6141a1910 con 0x7fb6141049c0 2024-03-21T09:08:54.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.184+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb600007d50 con 0x7fb6141049c0 2024-03-21T09:08:54.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.184+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600018760 con 0x7fb6141049c0 2024-03-21T09:08:54.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.184+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6141a1eb0 con 0x7fb6141049c0 2024-03-21T09:08:54.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.186+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb614108de0 con 0x7fb6141049c0 2024-03-21T09:08:54.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.188+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb60000f450 con 0x7fb6141049c0 2024-03-21T09:08:54.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.190+0000 7fb611600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5fc0a2f10 0x7fb5fc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.190+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb6000cf670 con 0x7fb6141049c0 2024-03-21T09:08:54.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.190+0000 7fb619200640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5fc0a2f10 0x7fb5fc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.191+0000 7fb619200640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5fc0a2f10 0x7fb5fc0a53d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb608006010 tx=0x7fb60802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.194+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb600014030 con 0x7fb6141049c0 2024-03-21T09:08:54.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:53 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/629475083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:54.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:53 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2588645170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.400+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6140622a0 con 0x7fb6141049c0 2024-03-21T09:08:54.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.402+0000 7fb611600640 1 -- 172.21.15.37:0/432224885 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7fb60009c200 con 0x7fb6141049c0 2024-03-21T09:08:54.396 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:08:54.396 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5fc0a2f10 msgr2=0x7fb5fc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5fc0a2f10 0x7fb5fc0a53d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb608006010 tx=0x7fb60802cf90 comp rx=0 tx=0).stop 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141049c0 msgr2=0x7fb614199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141049c0 0x7fb614199f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c940 tx=0x7fb60000ce00 comp rx=0 tx=0).stop 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 shutdown_connections 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5fc0a2f10 0x7fb5fc0a53d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6141072f0 0x7fb6141a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6141049c0 0x7fb614199f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 --2- 172.21.15.37:0/432224885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb614102090 0x7fb614199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 >> 172.21.15.37:0/432224885 conn(0x7fb6140fba20 msgr2=0x7fb614101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 shutdown_connections 2024-03-21T09:08:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.406+0000 7fb61a600640 1 -- 172.21.15.37:0/432224885 wait complete. 2024-03-21T09:08:54.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:53 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/629475083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:54.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:53 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2588645170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:54.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-21T09:08:46.306108+0000', 'modified': '2024-03-21T09:08:51.441981+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24589': {'gid': 24589, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6841/600888190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 600888190}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 600888190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24589, 'qdb_cluster': [24589]} 2024-03-21T09:08:54.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-21T09:08:54.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-21T09:08:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:53 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/629475083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:53 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2588645170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:08:54.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.640+0000 7fb2ca600640 1 -- 172.21.15.37:0/1339508463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 msgr2=0x7fb2c4076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.640+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1339508463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c4076480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009900 tx=0x7fb2b402bc40 comp rx=0 tx=0).stop 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 -- 172.21.15.37:0/1339508463 shutdown_connections 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1339508463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4077c40 0x7fb2c40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1339508463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c40772a0 0x7fb2c4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1339508463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c4076480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 -- 172.21.15.37:0/1339508463 >> 172.21.15.37:0/1339508463 conn(0x7fb2c40fee50 msgr2=0x7fb2c4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 -- 172.21.15.37:0/1339508463 shutdown_connections 2024-03-21T09:08:54.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.641+0000 7fb2ca600640 1 -- 172.21.15.37:0/1339508463 wait complete. 2024-03-21T09:08:54.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 Processor -- start 2024-03-21T09:08:54.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 -- start start 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 0x7fb2c419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c4077c40 0x7fb2c41a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c4113990 con 0x7fb2c40772a0 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c4113810 con 0x7fb2c4076060 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2ca600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c4113b10 con 0x7fb2c4077c40 2024-03-21T09:08:54.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 0x7fb2c419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.642+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 0x7fb2c419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51216/0 (socket says 172.21.15.37:51216) 2024-03-21T09:08:54.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 -- 172.21.15.37:0/1028325752 learned_addr learned my addr 172.21.15.37:0/1028325752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:54.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c4077c40 0x7fb2c41a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 -- 172.21.15.37:0/1028325752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c4077c40 msgr2=0x7fb2c41a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c4077c40 0x7fb2c41a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 -- 172.21.15.37:0/1028325752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 msgr2=0x7fb2c419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 -- 172.21.15.37:0/1028325752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c41a5bf0 con 0x7fb2c40772a0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c3400640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c4077c40 0x7fb2c41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.643+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 0x7fb2c419e580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000e9f0 tx=0x7fb2b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.644+0000 7fb2bbe00640 1 -- 172.21.15.37:0/1028325752 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000cd80 con 0x7fb2c40772a0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.644+0000 7fb2bbe00640 1 -- 172.21.15.37:0/1028325752 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2b0005500 con 0x7fb2c40772a0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.644+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c41a5ee0 con 0x7fb2c40772a0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.644+0000 7fb2bbe00640 1 -- 172.21.15.37:0/1028325752 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0010690 con 0x7fb2c40772a0 2024-03-21T09:08:54.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.644+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c41a6350 con 0x7fb2c40772a0 2024-03-21T09:08:54.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.648+0000 7fb2cd3d6740 1 Processor -- start 2024-03-21T09:08:54.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.648+0000 7fb2bbe00640 1 -- 172.21.15.37:0/1028325752 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb2b00046c0 con 0x7fb2c40772a0 2024-03-21T09:08:54.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 -- start start 2024-03-21T09:08:54.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edaba0 0x55c2c6edafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 0x55c2c6ee0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6ee2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6e3fc90 con 0x55c2c6edaba0 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6de8980 con 0x55c2c6edb590 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6e3bd30 con 0x55c2c6ee0700 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 0x55c2c6ee0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edaba0 0x55c2c6edafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 0x55c2c6ee0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35950/0 (socket says 172.21.15.37:35950) 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2c3400640 1 -- 172.21.15.37:0/3282924938 learned_addr learned my addr 172.21.15.37:0/3282924938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.649+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6ee2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2c3400640 1 -- 172.21.15.37:0/3282924938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 msgr2=0x55c2c6ee2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2c3400640 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6ee2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2c3400640 1 -- 172.21.15.37:0/3282924938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edaba0 msgr2=0x55c2c6edafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2c3400640 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edaba0 0x55c2c6edafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2c3400640 1 -- 172.21.15.37:0/3282924938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c6e00510 con 0x55c2c6edb590 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2c3400640 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 0x55c2c6ee0090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005ab0 tx=0x7fb2b4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2adfaf4adee888 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2bbe00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb2a80a2f20 0x7fb2a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2b8c00640 1 -- 172.21.15.37:0/3282924938 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4042070 con 0x55c2c6edb590 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2b8c00640 1 -- 172.21.15.37:0/3282924938 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2b402cd40 con 0x55c2c6edb590 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2b8c00640 1 -- 172.21.15.37:0/3282924938 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4005210 con 0x55c2c6edb590 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.650+0000 7fb2bbe00640 1 -- 172.21.15.37:0/1028325752 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb2b00cf9f0 con 0x7fb2c40772a0 2024-03-21T09:08:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb2a80a2f20 0x7fb2a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3282924938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 msgr2=0x55c2c6ee0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 0x55c2c6ee0090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005ab0 tx=0x7fb2b4000bc0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3282924938 shutdown_connections 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6ee2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edb590 0x55c2c6ee0090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3282924938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edaba0 0x55c2c6edafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3282924938 >> 172.21.15.37:0/3282924938 conn(0x55c2c6ed6110 msgr2=0x55c2c6ed6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb2a80a2f20 0x7fb2a80a53e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55c2c6edbdf0 tx=0x7fb2bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3282924938 shutdown_connections 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.651+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3282924938 wait complete. 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.652+0000 7fb2cd3d6740 1 -- start start 2024-03-21T09:08:54.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.652+0000 7fb2cd3d6740 1 Processor -- start 2024-03-21T09:08:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6eec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ee3650 con 0x55c2c6edb590 2024-03-21T09:08:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6dfbdd0 con 0x55c2c6edaba0 2024-03-21T09:08:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ee37d0 con 0x55c2c6ee0700 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2cd3d6740 10 client.0 _ll_register_callbacks cb 0x55c2c6b53ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35964/0 (socket says 172.21.15.37:35964) 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.653+0000 7fb2c3400640 1 -- 172.21.15.37:0/3215165696 learned_addr learned my addr 172.21.15.37:0/3215165696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6eec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3400640 1 -- 172.21.15.37:0/3215165696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 msgr2=0x55c2c6eec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3400640 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6eec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3400640 1 -- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 msgr2=0x55c2c6ee55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3400640 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3400640 1 -- 172.21.15.37:0/3215165696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c6b542d0 con 0x55c2c6edaba0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2c3400640 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee50b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402c250 tx=0x7fb2b4005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e836978a8f576ddd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.654+0000 7fb2ad600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:08:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2ad600640 1 -- 172.21.15.37:0/3215165696 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b404f060 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2ad600640 1 -- 172.21.15.37:0/3215165696 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb2b4004ad0 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2b402cd40 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2ad600640 1 -- 172.21.15.37:0/3215165696 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2b4042070 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2cd3d6740 10 client.24680 fetch_fsmap learned FSMap version 17 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55c2c6c1e6b0 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.655+0000 7fb2ad600640 1 -- 172.21.15.37:0/3215165696 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2b402cd40 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2ad600640 1 -- 172.21.15.37:0/3215165696 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb2b4034440 con 0x55c2c6edaba0 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2cd3d6740 10 client.24680 fetch_fsmap finished waiting for FSMap version 17 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2cd3d6740 10 client.24680 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2cd3d6740 20 client.24680 populate_metadata read hostname 'smithi037' 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6ef0790 0x55c2c6ef5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2cd3d6740 4 client.24680 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2c6de1520 con 0x55c2c6ef0790 2024-03-21T09:08:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.656+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6ef0790 0x55c2c6ef5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.657+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6ef0790 0x55c2c6ef5860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3182663589ffa20c server_cookie=8ae17da9887ff37 in_seq=0 out_seq=0 2024-03-21T09:08:54.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.657+0000 7fb2ad600640 10 client.24680 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.657+0000 7fb2ad600640 1 -- 172.21.15.37:0/3215165696 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2c6de1520 con 0x55c2c6ef0790 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.657+0000 7fb2ad600640 10 client.24680 handle_command_reply: tid=0 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 client.24680 shutdown 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 msgr2=0x55c2c6ee50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee50b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402c250 tx=0x7fb2b4005660 comp rx=0 tx=0).stop 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 shutdown_connections 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6ee0700 0x55c2c6eec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6ef0790 0x55c2c6ef5860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/3215165696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee50b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 >> 172.21.15.37:0/3215165696 conn(0x55c2c6ed6110 msgr2=0x55c2c6ed5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2ad600640 0 client.24680 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:08:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 shutdown_connections 2024-03-21T09:08:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/3215165696 wait complete. 2024-03-21T09:08:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.658+0000 7fb2cd3d6740 20 client.24680 trim_cache size 0 max 16384 2024-03-21T09:08:54.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 Processor -- start 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 -- start start 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ed64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6eec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6ed4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ef0c00 con 0x55c2c6edb590 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ef1260 con 0x55c2c6ee0700 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.677+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ef18c0 con 0x55c2c6edaba0 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6eec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6eec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51242/0 (socket says 172.21.15.37:51242) 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c2a00640 1 -- 172.21.15.37:0/2845071614 learned_addr learned my addr 172.21.15.37:0/2845071614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3400640 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ed64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6ed4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3400640 1 -- 172.21.15.37:0/2845071614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 msgr2=0x55c2c6ed4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3400640 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6ed4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3400640 1 -- 172.21.15.37:0/2845071614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 msgr2=0x55c2c6eec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3400640 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6eec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c3400640 1 -- 172.21.15.37:0/2845071614 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c6edb970 con 0x55c2c6edaba0 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.678+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6eec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.679+0000 7fb2c3400640 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ed64f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4034860 tx=0x7fb2b4009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ef69178dff2756 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.679+0000 7fb2afe00640 1 -- 172.21.15.37:0/2845071614 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b404f070 con 0x55c2c6edaba0 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.679+0000 7fb2afe00640 1 -- 172.21.15.37:0/2845071614 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2b402cd40 con 0x55c2c6edaba0 2024-03-21T09:08:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.679+0000 7fb2afe00640 1 -- 172.21.15.37:0/2845071614 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40476c0 con 0x55c2c6edaba0 2024-03-21T09:08:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/2845071614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 msgr2=0x55c2c6ed64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ed64f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4034860 tx=0x7fb2b4009660 comp rx=0 tx=0).stop 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/2845071614 shutdown_connections 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6ed4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6eec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/2845071614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ed64f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/2845071614 >> 172.21.15.37:0/2845071614 conn(0x55c2c6ed6110 msgr2=0x55c2c6edc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/2845071614 shutdown_connections 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/2845071614 wait complete. 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 -- start start 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.680+0000 7fb2cd3d6740 1 Processor -- start 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6eeb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ef2580 con 0x55c2c6edb590 2024-03-21T09:08:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ef1f20 con 0x55c2c6ee0700 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c6ef2be0 con 0x55c2c6edaba0 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2cd3d6740 10 client.0 _ll_register_callbacks cb 0x7fb2bc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51244/0 (socket says 172.21.15.37:51244) 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c2a00640 1 -- 172.21.15.37:0/639085216 learned_addr learned my addr 172.21.15.37:0/639085216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c3e00640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6eeb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c3400640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c2a00640 1 -- 172.21.15.37:0/639085216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 msgr2=0x55c2c6ee7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.681+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2c2a00640 1 -- 172.21.15.37:0/639085216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 msgr2=0x55c2c6eeb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6eeb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2c2a00640 1 -- 172.21.15.37:0/639085216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c6eeb860 con 0x55c2c6edb590 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2c2a00640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee82a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0098340 tx=0x7fb2b00d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c41ee4367fccf3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2ad600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2ad600640 1 -- 172.21.15.37:0/639085216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b009bcb0 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2ad600640 1 -- 172.21.15.37:0/639085216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb2b009a1e0 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2b402cd40 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2ad600640 1 -- 172.21.15.37:0/639085216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2b009b840 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2cd3d6740 10 client.14586 fetch_fsmap learned FSMap version 17 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.682+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55c2c6f4cab0 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2ad600640 1 -- 172.21.15.37:0/639085216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2b402cd40 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2ad600640 1 -- 172.21.15.37:0/639085216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb2b0098550 con 0x55c2c6edb590 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2cd3d6740 10 client.14586 fetch_fsmap finished waiting for FSMap version 17 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2cd3d6740 10 client.14586 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2cd3d6740 20 client.14586 populate_metadata read hostname 'smithi037' 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6f51170 0x55c2c6f4cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2cd3d6740 4 client.14586 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:08:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c2c6e4d1d0 con 0x55c2c6f51170 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.683+0000 7fb2c3400640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6f51170 0x55c2c6f4cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2c3400640 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6f51170 0x55c2c6f4cda0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca12ed2618df16d server_cookie=5d0085f626b542c0 in_seq=0 out_seq=0 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2ad600640 10 client.14586 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2ad600640 1 -- 172.21.15.37:0/639085216 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c2c6e4d1d0 con 0x55c2c6f51170 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2ad600640 10 client.14586 handle_command_reply: tid=0 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 client.14586 shutdown 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 msgr2=0x55c2c6ee82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee82a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0098340 tx=0x7fb2b00d5530 comp rx=0 tx=0).stop 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 shutdown_connections 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c2c6f51170 0x55c2c6f4cda0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c2c6ee0700 0x55c2c6eeb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2c6edb590 0x55c2c6ee82a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 --2- 172.21.15.37:0/639085216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2c6edaba0 0x55c2c6ee7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 >> 172.21.15.37:0/639085216 conn(0x55c2c6ed6110 msgr2=0x55c2c6ed64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.684+0000 7fb2ad600640 0 client.14586 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 shutdown_connections 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2cd3d6740 1 -- 172.21.15.37:0/639085216 wait complete. 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2cd3d6740 20 client.14586 trim_cache size 0 max 16384 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-03-21T09:08:54.683 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24589, 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 7.7943602829999996, 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 17, 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 89, 2024-03-21T09:08:54.684 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 18.207048254 2024-03-21T09:08:54.685 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:08:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb2a80a2f20 msgr2=0x7fb2a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb2a80a2f20 0x7fb2a80a53e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55c2c6edbdf0 tx=0x7fb2bc009500 comp rx=0 tx=0).stop 2024-03-21T09:08:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 msgr2=0x7fb2c419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.685+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 0x7fb2c419e580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000e9f0 tx=0x7fb2b000edf0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 shutdown_connections 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb2a80a2f20 0x7fb2a80a53e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c4077c40 0x7fb2c41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c40772a0 0x7fb2c419e580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 --2- 172.21.15.37:0/1028325752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2c4076060 0x7fb2c419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 >> 172.21.15.37:0/1028325752 conn(0x7fb2c40fee50 msgr2=0x7fb2c4100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 shutdown_connections 2024-03-21T09:08:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:54.686+0000 7fb2ca600640 1 -- 172.21.15.37:0/1028325752 wait complete. 2024-03-21T09:08:54.697 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T09:08:54.698 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:08:54.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T09:08:55.178 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:08:55.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:54 smithi037 ceph-mon[31830]: pgmap v324: 129 pgs: 1 creating+peering, 18 unknown, 110 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-21T09:08:55.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:54 smithi037 ceph-mon[31830]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T09:08:55.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:54 smithi037 ceph-mon[31830]: Cluster is now healthy 2024-03-21T09:08:55.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:54 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/432224885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:54 smithi071 ceph-mon[36597]: pgmap v324: 129 pgs: 1 creating+peering, 18 unknown, 110 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-21T09:08:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:54 smithi071 ceph-mon[36597]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T09:08:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:54 smithi071 ceph-mon[36597]: Cluster is now healthy 2024-03-21T09:08:55.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:54 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/432224885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:55.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:54 smithi053 ceph-mon[36169]: pgmap v324: 129 pgs: 1 creating+peering, 18 unknown, 110 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-21T09:08:55.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:54 smithi053 ceph-mon[36169]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T09:08:55.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:54 smithi053 ceph-mon[36169]: Cluster is now healthy 2024-03-21T09:08:55.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:54 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/432224885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:08:57.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:56 smithi037 ceph-mon[31830]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-03-21T09:08:57.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:56 smithi071 ceph-mon[36597]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-03-21T09:08:57.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:56 smithi053 ceph-mon[36169]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-03-21T09:08:57.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 -- 172.21.15.37:0/2455648945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 msgr2=0x7ff150102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:57.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 --2- 172.21.15.37:0/2455648945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff150102660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0099c0 tx=0x7ff13c009cd0 comp rx=0 tx=0).stop 2024-03-21T09:08:57.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 -- 172.21.15.37:0/2455648945 shutdown_connections 2024-03-21T09:08:57.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 --2- 172.21.15.37:0/2455648945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff150102ba0 0x7ff15010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:57.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 --2- 172.21.15.37:0/2455648945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 0x7ff1501085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 --2- 172.21.15.37:0/2455648945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff150102660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 -- 172.21.15.37:0/2455648945 >> 172.21.15.37:0/2455648945 conn(0x7ff1500792c0 msgr2=0x7ff1500796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 -- 172.21.15.37:0/2455648945 shutdown_connections 2024-03-21T09:08:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.634+0000 7ff154e00700 1 -- 172.21.15.37:0/2455648945 wait complete. 2024-03-21T09:08:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.636+0000 7ff154e00700 1 Processor -- start 2024-03-21T09:08:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.636+0000 7ff154e00700 1 -- start start 2024-03-21T09:08:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.636+0000 7ff154e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff15019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff154e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff150102ba0 0x7ff15019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff154e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 0x7ff150198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff154e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1501124c0 con 0x7ff1501021e0 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff154e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff150112340 con 0x7ff1501081c0 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff154e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff150112640 con 0x7ff150102ba0 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff15019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff15019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51254/0 (socket says 172.21.15.37:51254) 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 -- 172.21.15.37:0/3663916165 learned_addr learned my addr 172.21.15.37:0/3663916165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:08:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14e800700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 0x7ff150198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14d400700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff150102ba0 0x7ff15019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 -- 172.21.15.37:0/3663916165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff150102ba0 msgr2=0x7ff15019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff150102ba0 0x7ff15019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 -- 172.21.15.37:0/3663916165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 msgr2=0x7ff150198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 0x7ff150198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14de00700 1 -- 172.21.15.37:0/3663916165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150198fe0 con 0x7ff1501021e0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.637+0000 7ff14e800700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 0x7ff150198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.638+0000 7ff14de00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff15019e630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff14000ef10 tx=0x7ff14000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.638+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000c9c0 con 0x7ff1501021e0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.638+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff140019070 con 0x7ff1501021e0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.638+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140007e60 con 0x7ff1501021e0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.638+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff150199270 con 0x7ff1501021e0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.638+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1501a5490 con 0x7ff1501021e0 2024-03-21T09:08:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.640+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150103870 con 0x7ff1501021e0 2024-03-21T09:08:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.642+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff140012050 con 0x7ff1501021e0 2024-03-21T09:08:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.643+0000 7ff146a00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff1340a3030 0x7ff1340a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:08:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.643+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff1400cf540 con 0x7ff1501021e0 2024-03-21T09:08:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.643+0000 7ff14d400700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff1340a3030 0x7ff1340a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:08:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.644+0000 7ff14d400700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff1340a3030 0x7ff1340a54f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1501070d0 tx=0x7ff13c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:08:57.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:57.649+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff140010040 con 0x7ff1501021e0 2024-03-21T09:08:58.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.012+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff15002da30 con 0x7ff1501021e0 2024-03-21T09:08:58.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.013+0000 7ff146a00700 1 -- 172.21.15.37:0/3663916165 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff14009c0d0 con 0x7ff1501021e0 2024-03-21T09:08:58.007 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T09:08:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff1340a3030 msgr2=0x7ff1340a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff1340a3030 0x7ff1340a54f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1501070d0 tx=0x7ff13c005fb0 comp rx=0 tx=0).stop 2024-03-21T09:08:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 msgr2=0x7ff15019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:08:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff15019e630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff14000ef10 tx=0x7ff14000c3a0 comp rx=0 tx=0).stop 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 shutdown_connections 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff1340a3030 0x7ff1340a54f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1501081c0 0x7ff150198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1501021e0 0x7ff15019e630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 --2- 172.21.15.37:0/3663916165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff150102ba0 0x7ff15019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 >> 172.21.15.37:0/3663916165 conn(0x7ff1500792c0 msgr2=0x7ff150100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.016+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 shutdown_connections 2024-03-21T09:08:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:08:58.017+0000 7ff154e00700 1 -- 172.21.15.37:0/3663916165 wait complete. 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T09:08:58.974 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T09:08:58.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:08:58.975 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-03-21T09:08:59.028 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T09:08:59.028 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:08:59.029 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42022sec preferred_lft 42022sec 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:08:59.030 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-03-21T09:08:59.031 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T09:08:59.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T09:08:59.032 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:08:59.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:58 smithi037 ceph-mon[31830]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:59.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:08:58 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3663916165' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T09:08:59.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:58 smithi071 ceph-mon[36597]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:59.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:08:58 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3663916165' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T09:08:59.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:58 smithi053 ceph-mon[36169]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:08:59.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:08:58 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3663916165' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T09:08:59.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:08:59.490 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T09:08:59.578 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-03-21T09:08:59.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:08:59.582 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-03-21T09:08:59.639 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-03-21T09:08:59.639 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T09:08:59.639 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T09:08:59.639 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T09:08:59.639 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T09:08:59.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:08:59.640 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:08:59.640 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:08:59.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T09:08:59.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T09:08:59.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T09:08:59.640 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:09:00.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:00.369 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-21T09:09:00.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:00.424 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-03-21T09:09:00.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:00.479 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:09:00.479 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:09:00.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:00.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-21T09:09:00.480 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:09:00.631 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-21T09:09:00.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-21T09:09:00.631 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:09:00.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:00.920 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:09:00.920 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:09:00.920 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-03-21T09:09:00.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T09:09:00.921 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:09:01.086 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-21T09:09:01.087 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T09:09:01.087 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:01.131 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T09:09:01.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T09:09:01.131 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:01.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:00 smithi037 ceph-mon[31830]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:09:01.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-03-21T09:09:01.298 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-03-21T09:09:01.345 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T09:09:01.346 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/08c1551ecad77086678a7edce9e97653ee9930cd3e1ddfda85a6d8d6c086160a/merged 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/f62261160b485a806d345e67410f4438dee2ecbe7c76e30fba7b276b0fd195b2/merged 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/0 2024-03-21T09:09:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/c6f64aa65d8f64e5bcf05d5ae2667d31c5912067745819b2bb8776a734d52ab4/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/66a7ce2a8ef6bfb31f3b17fa4e269b64651dada15acb5055bef52a12cc825bfe/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/c5924d65b8887d373932b174bbe4c9331171b06d1e3e1f0f0481bbcb22f2fbda/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/7cdb189d3eaf74cf5f1fa19379cb10b69c1e4f54a9ea1f9cce336c90e9287432/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/fc14d6b2bd27321fe2e69a01bb0cf1c51e5a6bb2c1864e93d50bce20831b54ea/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/04faf4c509cd7e5f102fc886109ef045ba710157f3fd7edc969bd07981c4879d/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/ce17853b8d90ebc5341d3a9121f547813212e6f4dec44cbaf7e6260606abac81/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/bf8dc586ca1f46b35097bf8cdf2f62c932ae80f128ab6069c8ca67300cf1b989/merged 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:01.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:01.349 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-03-21T09:09:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:00 smithi071 ceph-mon[36597]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:09:01.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T09:09:01.439 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-21T09:09:01.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:00 smithi053 ceph-mon[36169]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:09:01.485 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-03-21T09:09:01.565 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T09:09:01.611 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-03-21T09:09:01.611 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-03-21T09:09:01.611 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-03-21T09:09:01.611 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-03-21T09:09:01.611 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-03-21T09:09:01.612 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/08c1551ecad77086678a7edce9e97653ee9930cd3e1ddfda85a6d8d6c086160a/merged 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/f62261160b485a806d345e67410f4438dee2ecbe7c76e30fba7b276b0fd195b2/merged 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/0 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/c6f64aa65d8f64e5bcf05d5ae2667d31c5912067745819b2bb8776a734d52ab4/merged 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/66a7ce2a8ef6bfb31f3b17fa4e269b64651dada15acb5055bef52a12cc825bfe/merged 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/c5924d65b8887d373932b174bbe4c9331171b06d1e3e1f0f0481bbcb22f2fbda/merged 2024-03-21T09:09:01.613 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/7cdb189d3eaf74cf5f1fa19379cb10b69c1e4f54a9ea1f9cce336c90e9287432/merged 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/fc14d6b2bd27321fe2e69a01bb0cf1c51e5a6bb2c1864e93d50bce20831b54ea/merged 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/04faf4c509cd7e5f102fc886109ef045ba710157f3fd7edc969bd07981c4879d/merged 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/ce17853b8d90ebc5341d3a9121f547813212e6f4dec44cbaf7e6260606abac81/merged 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/bf8dc586ca1f46b35097bf8cdf2f62c932ae80f128ab6069c8ca67300cf1b989/merged 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:01.614 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:01.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:01.615 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-03-21T09:09:01.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:2024-03-21T09:09:01.693+0000 7fd9e3049480 -1 init, newargv = 0x55d195ef1240 newargc=15 2024-03-21T09:09:01.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[72815]: starting ceph client 2024-03-21T09:09:01.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[72815]: starting fuse 2024-03-21T09:09:02.758 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-03-21T09:09:02.853 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T09:09:02.895 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-03-21T09:09:02.895 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-03-21T09:09:02.895 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-03-21T09:09:02.896 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay 2024-03-21T09:09:02.897 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/08c1551ecad77086678a7edce9e97653ee9930cd3e1ddfda85a6d8d6c086160a/merged 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/f62261160b485a806d345e67410f4438dee2ecbe7c76e30fba7b276b0fd195b2/merged 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/0 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/c6f64aa65d8f64e5bcf05d5ae2667d31c5912067745819b2bb8776a734d52ab4/merged 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/66a7ce2a8ef6bfb31f3b17fa4e269b64651dada15acb5055bef52a12cc825bfe/merged 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/c5924d65b8887d373932b174bbe4c9331171b06d1e3e1f0f0481bbcb22f2fbda/merged 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/7cdb189d3eaf74cf5f1fa19379cb10b69c1e4f54a9ea1f9cce336c90e9287432/merged 2024-03-21T09:09:02.898 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/fc14d6b2bd27321fe2e69a01bb0cf1c51e5a6bb2c1864e93d50bce20831b54ea/merged 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/04faf4c509cd7e5f102fc886109ef045ba710157f3fd7edc969bd07981c4879d/merged 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/ce17853b8d90ebc5341d3a9121f547813212e6f4dec44cbaf7e6260606abac81/merged 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/bf8dc586ca1f46b35097bf8cdf2f62c932ae80f128ab6069c8ca67300cf1b989/merged 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:09:02.899 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:02.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:02.900 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-03-21T09:09:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:118 2024-03-21T09:09:02.988 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-21T09:09:02.988 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-03-21T09:09:02.988 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-03-21T09:09:02.988 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-03-21T09:09:02.989 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.admin")) 2024-03-21T09:09:02.990 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-03-21T09:09:03.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:03 smithi037 ceph-mon[31830]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:09:03.209 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.admin.72815.asok 2024-03-21T09:09:03.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T09:09:03.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72815.asok 2024-03-21T09:09:03.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:03.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72815.asok status 2024-03-21T09:09:03.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:03 smithi071 ceph-mon[36597]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:09:03.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:03 smithi053 ceph-mon[36169]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:09:03.453 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "2edfee68ad7243bed422a049b6f5f58619300f11", 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "admin", 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "72815", 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-03-21T09:09:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24721, 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 24721 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4153573887 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-03-21T09:09:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4153573887 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.24721 192.168.0.1:0/4153573887", 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/4153573887", 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 17, 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 90, 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-03-21T09:09:03.456 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:09:03.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:03.467 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-03-21T09:09:03.467 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:03.519 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-03-21T09:09:03.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:03.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:03.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:03.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:03.610 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-21T09:09:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:# file: . 2024-03-21T09:09:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-21T09:09:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:03.688 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:09:03.688 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:09:03.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:09:04.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 -- 172.21.15.37:0/4250100670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4581027b0 msgr2=0x7ff458102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 --2- 172.21.15.37:0/4250100670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4581027b0 0x7ff458102c30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff440009900 tx=0x7ff44002bc40 comp rx=0 tx=0).stop 2024-03-21T09:09:04.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 -- 172.21.15.37:0/4250100670 shutdown_connections 2024-03-21T09:09:04.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 --2- 172.21.15.37:0/4250100670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff458103170 0x7ff458109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 --2- 172.21.15.37:0/4250100670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4581027b0 0x7ff458102c30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 --2- 172.21.15.37:0/4250100670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4581015c0 0x7ff4581019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 -- 172.21.15.37:0/4250100670 >> 172.21.15.37:0/4250100670 conn(0x7ff4580fcb30 msgr2=0x7ff4580fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:04.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 -- 172.21.15.37:0/4250100670 shutdown_connections 2024-03-21T09:09:04.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.014+0000 7ff45da00640 1 -- 172.21.15.37:0/4250100670 wait complete. 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.015+0000 7ff45da00640 1 Processor -- start 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 -- start start 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 0x7ff458197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4581027b0 0x7ff458197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff458103170 0x7ff45819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45810d2a0 con 0x7ff4581015c0 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45810d120 con 0x7ff458103170 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff45da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45810d420 con 0x7ff4581027b0 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff457e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 0x7ff458197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff457e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 0x7ff458197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56298/0 (socket says 172.21.15.37:56298) 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.016+0000 7ff457e00640 1 -- 172.21.15.37:0/2301746028 learned_addr learned my addr 172.21.15.37:0/2301746028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457400640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4581027b0 0x7ff458197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff45d000640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff458103170 0x7ff45819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457e00640 1 -- 172.21.15.37:0/2301746028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4581027b0 msgr2=0x7ff458197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457e00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4581027b0 0x7ff458197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457e00640 1 -- 172.21.15.37:0/2301746028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff458103170 msgr2=0x7ff45819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457e00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff458103170 0x7ff45819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457e00640 1 -- 172.21.15.37:0/2301746028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45819f4a0 con 0x7ff4581015c0 2024-03-21T09:09:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.017+0000 7ff457e00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 0x7ff458197880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff44800eb70 tx=0x7ff44800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.018+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448003b70 con 0x7ff4581015c0 2024-03-21T09:09:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.018+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45819f790 con 0x7ff4581015c0 2024-03-21T09:09:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.018+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44800ee70 con 0x7ff4581015c0 2024-03-21T09:09:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.018+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448010830 con 0x7ff4581015c0 2024-03-21T09:09:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.018+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45819fc60 con 0x7ff4581015c0 2024-03-21T09:09:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.019+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4581088e0 con 0x7ff4581015c0 2024-03-21T09:09:04.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.021+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff448004020 con 0x7ff4581015c0 2024-03-21T09:09:04.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.022+0000 7ff454c00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff43c0a2f10 0x7ff43c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.023+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff4480cfd00 con 0x7ff4581015c0 2024-03-21T09:09:04.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.023+0000 7ff457400640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff43c0a2f10 0x7ff43c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.023+0000 7ff457400640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff43c0a2f10 0x7ff43c0a53d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff44002c250 tx=0x7ff44002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:04.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.031+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff44809c890 con 0x7ff4581015c0 2024-03-21T09:09:04.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.237+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4580622a0 con 0x7ff4581015c0 2024-03-21T09:09:04.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.239+0000 7ff454c00640 1 -- 172.21.15.37:0/2301746028 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7ff44809c190 con 0x7ff4581015c0 2024-03-21T09:09:04.234 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff43c0a2f10 msgr2=0x7ff43c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff43c0a2f10 0x7ff43c0a53d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff44002c250 tx=0x7ff44002cf90 comp rx=0 tx=0).stop 2024-03-21T09:09:04.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 msgr2=0x7ff458197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 0x7ff458197880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff44800eb70 tx=0x7ff44800c6a0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 shutdown_connections 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff43c0a2f10 0x7ff43c0a53d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff458103170 0x7ff45819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4581027b0 0x7ff458197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 --2- 172.21.15.37:0/2301746028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581015c0 0x7ff458197880 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.242+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 >> 172.21.15.37:0/2301746028 conn(0x7ff4580fcb30 msgr2=0x7ff458105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.243+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 shutdown_connections 2024-03-21T09:09:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.243+0000 7ff45da00640 1 -- 172.21.15.37:0/2301746028 wait complete. 2024-03-21T09:09:04.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:09:04.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.478+0000 7f256ce00640 1 -- 172.21.15.37:0/1198022286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25680ffd60 msgr2=0x7f2568100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.478+0000 7f256ce00640 1 --2- 172.21.15.37:0/1198022286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25680ffd60 0x7f2568100180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2558009900 tx=0x7f255802bc30 comp rx=0 tx=0).stop 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 -- 172.21.15.37:0/1198022286 shutdown_connections 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 --2- 172.21.15.37:0/1198022286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568101060 0x7f25681014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 --2- 172.21.15.37:0/1198022286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25681006c0 0x7f2568100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 --2- 172.21.15.37:0/1198022286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25680ffd60 0x7f2568100180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 -- 172.21.15.37:0/1198022286 >> 172.21.15.37:0/1198022286 conn(0x7f25680fb980 msgr2=0x7f25680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 -- 172.21.15.37:0/1198022286 shutdown_connections 2024-03-21T09:09:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.479+0000 7f256ce00640 1 -- 172.21.15.37:0/1198022286 wait complete. 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 Processor -- start 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 -- start start 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25680ffd60 0x7f25681999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25681006c0 0x7f2568199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2568101060 0x7f25681a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810f380 con 0x7f25681006c0 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810f200 con 0x7f25680ffd60 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.480+0000 7f256ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810f500 con 0x7f2568101060 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f255fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25681006c0 0x7f2568199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25680ffd60 0x7f25681999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f255fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25681006c0 0x7f2568199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56310/0 (socket says 172.21.15.37:56310) 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f255fe00640 1 -- 172.21.15.37:0/1880178970 learned_addr learned my addr 172.21.15.37:0/1880178970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567e00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2568101060 0x7f25681a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567e00640 1 -- 172.21.15.37:0/1880178970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25680ffd60 msgr2=0x7f25681999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567e00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25680ffd60 0x7f25681999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567e00640 1 -- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25681006c0 msgr2=0x7f2568199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567e00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25681006c0 0x7f2568199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.481+0000 7f2567e00640 1 -- 172.21.15.37:0/1880178970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25681a15f0 con 0x7f2568101060 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.482+0000 7f2567e00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2568101060 0x7f25681a0f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2560009fd0 tx=0x7f256000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.482+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000ce20 con 0x7f2568101060 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.482+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2560005020 con 0x7f2568101060 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.482+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560010720 con 0x7f2568101060 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.482+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25681a18e0 con 0x7f2568101060 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.482+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a1e30 con 0x7f2568101060 2024-03-21T09:09:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.483+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256810aab0 con 0x7f2568101060 2024-03-21T09:09:04.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.485+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2560005540 con 0x7f2568101060 2024-03-21T09:09:04.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.487+0000 7f2564c00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f254c0a2fa0 0x7f254c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:04.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.487+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f25600cfc70 con 0x7f2568101060 2024-03-21T09:09:04.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.487+0000 7f2567400640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f254c0a2fa0 0x7f254c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:04.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.488+0000 7f2567400640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f254c0a2fa0 0x7f254c0a5460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f255802c240 tx=0x7f255802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.489+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f256009c800 con 0x7f2568101060 2024-03-21T09:09:04.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.778+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25680622a0 con 0x7f2568101060 2024-03-21T09:09:04.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.779+0000 7f2564c00640 1 -- 172.21.15.37:0/1880178970 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f256009c100 con 0x7f2568101060 2024-03-21T09:09:04.773 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.781+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f254c0a2fa0 msgr2=0x7f254c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.781+0000 7f256ce00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f254c0a2fa0 0x7f254c0a5460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f255802c240 tx=0x7f255802df90 comp rx=0 tx=0).stop 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.781+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2568101060 msgr2=0x7f25681a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.781+0000 7f256ce00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2568101060 0x7f25681a0f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2560009fd0 tx=0x7f256000ef10 comp rx=0 tx=0).stop 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 shutdown_connections 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f254c0a2fa0 0x7f254c0a5460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2568101060 0x7f25681a0f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25681006c0 0x7f2568199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 --2- 172.21.15.37:0/1880178970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25680ffd60 0x7f25681999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 >> 172.21.15.37:0/1880178970 conn(0x7f25680fb980 msgr2=0x7f25680fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 shutdown_connections 2024-03-21T09:09:04.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:04.782+0000 7f256ce00640 1 -- 172.21.15.37:0/1880178970 wait complete. 2024-03-21T09:09:04.798 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.051+0000 7fb3f4a00640 1 -- 172.21.15.37:0/2799252849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f0101fa0 msgr2=0x7fb3f0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.051+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/2799252849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f0101fa0 0x7fb3f0108830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009ca0 tx=0x7fb3e802c0f0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 -- 172.21.15.37:0/2799252849 shutdown_connections 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/2799252849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f0101fa0 0x7fb3f0108830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/2799252849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3f01015e0 0x7fb3f0101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/2799252849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 -- 172.21.15.37:0/2799252849 >> 172.21.15.37:0/2799252849 conn(0x7fb3f00fb980 msgr2=0x7fb3f00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 -- 172.21.15.37:0/2799252849 shutdown_connections 2024-03-21T09:09:05.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.052+0000 7fb3f4a00640 1 -- 172.21.15.37:0/2799252849 wait complete. 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 Processor -- start 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 -- start start 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 0x7fb3f0199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3f0101fa0 0x7fb3f01a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f010af30 con 0x7fb3f01015e0 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f010adb0 con 0x7fb3f01003f0 2024-03-21T09:09:05.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3f4a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f010b0b0 con 0x7fb3f0101fa0 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.053+0000 7fb3eea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 0x7fb3f0199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3eea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 0x7fb3f0199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56326/0 (socket says 172.21.15.37:56326) 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57192/0 (socket says 172.21.15.37:57192) 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3eea00640 1 -- 172.21.15.37:0/682806335 learned_addr learned my addr 172.21.15.37:0/682806335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3efe00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3f0101fa0 0x7fb3f01a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 -- 172.21.15.37:0/682806335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3f0101fa0 msgr2=0x7fb3f01a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3f0101fa0 0x7fb3f01a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 -- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 msgr2=0x7fb3f0199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 0x7fb3f0199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 -- 172.21.15.37:0/682806335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f01a15b0 con 0x7fb3f01003f0 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3eea00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 0x7fb3f0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.054+0000 7fb3ef400640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0199990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000a840 tx=0x7fb3e000ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:05.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.055+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0004020 con 0x7fb3f01003f0 2024-03-21T09:09:05.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.055+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3e00041c0 con 0x7fb3f01003f0 2024-03-21T09:09:05.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.055+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0003d40 con 0x7fb3f01003f0 2024-03-21T09:09:05.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.055+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f01a18a0 con 0x7fb3f01003f0 2024-03-21T09:09:05.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.055+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f01a1da0 con 0x7fb3f01003f0 2024-03-21T09:09:05.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.056+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0107710 con 0x7fb3f01003f0 2024-03-21T09:09:05.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.058+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb3e00079b0 con 0x7fb3f01003f0 2024-03-21T09:09:05.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.060+0000 7fb3e7e00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb3d40a2fa0 0x7fb3d40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.061+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb3e00ce970 con 0x7fb3f01003f0 2024-03-21T09:09:05.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.061+0000 7fb3eea00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb3d40a2fa0 0x7fb3d40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.061+0000 7fb3eea00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb3d40a2fa0 0x7fb3d40a5460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3f019aea0 tx=0x7fb3dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:05.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.064+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb3e009aeb0 con 0x7fb3f01003f0 2024-03-21T09:09:05.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:05 smithi037 ceph-mon[31830]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-03-21T09:09:05.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:05 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2301746028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:05.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:05 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1880178970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:09:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.269+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3f0062340 con 0x7fb3f01003f0 2024-03-21T09:09:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.270+0000 7fb3e7e00640 1 -- 172.21.15.37:0/682806335 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3e009ef50 con 0x7fb3f01003f0 2024-03-21T09:09:05.264 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb3d40a2fa0 msgr2=0x7fb3d40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb3d40a2fa0 0x7fb3d40a5460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3f019aea0 tx=0x7fb3dc005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 msgr2=0x7fb3f0199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0199990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000a840 tx=0x7fb3e000ad00 comp rx=0 tx=0).stop 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 shutdown_connections 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb3d40a2fa0 0x7fb3d40a5460 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3f0101fa0 0x7fb3f01a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f01015e0 0x7fb3f0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 --2- 172.21.15.37:0/682806335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f01003f0 0x7fb3f0199990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 >> 172.21.15.37:0/682806335 conn(0x7fb3f00fb980 msgr2=0x7fb3f01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 shutdown_connections 2024-03-21T09:09:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.273+0000 7fb3f4a00640 1 -- 172.21.15.37:0/682806335 wait complete. 2024-03-21T09:09:05.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:09:05.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:05 smithi071 ceph-mon[36597]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-03-21T09:09:05.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:05 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2301746028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:05.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:05 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1880178970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:09:05.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:05 smithi053 ceph-mon[36169]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-03-21T09:09:05.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:05 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2301746028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:05.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:05 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1880178970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 -- 172.21.15.37:0/3993274763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 msgr2=0x7fb778109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 --2- 172.21.15.37:0/3993274763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778109620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb770009ca0 tx=0x7fb77002c100 comp rx=0 tx=0).stop 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 -- 172.21.15.37:0/3993274763 shutdown_connections 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 --2- 172.21.15.37:0/3993274763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778109620 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 --2- 172.21.15.37:0/3993274763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7781048a0 0x7fb778106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 --2- 172.21.15.37:0/3993274763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb778101f70 0x7fb778104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 -- 172.21.15.37:0/3993274763 >> 172.21.15.37:0/3993274763 conn(0x7fb7780fb980 msgr2=0x7fb7780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 -- 172.21.15.37:0/3993274763 shutdown_connections 2024-03-21T09:09:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.514+0000 7fb77ea00640 1 -- 172.21.15.37:0/3993274763 wait complete. 2024-03-21T09:09:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.515+0000 7fb77ea00640 1 Processor -- start 2024-03-21T09:09:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 -- start start 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb778101f70 0x7fb778101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7781048a0 0x7fb7780fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77810d180 con 0x7fb7781071d0 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77810d000 con 0x7fb778101f70 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77ea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77810d300 con 0x7fb7781048a0 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb778101f70 0x7fb778101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56344/0 (socket says 172.21.15.37:56344) 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77e000640 1 -- 172.21.15.37:0/1332884473 learned_addr learned my addr 172.21.15.37:0/1332884473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.516+0000 7fb77cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7781048a0 0x7fb7780fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77e000640 1 -- 172.21.15.37:0/1332884473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7781048a0 msgr2=0x7fb7780fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77e000640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7781048a0 0x7fb7780fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77e000640 1 -- 172.21.15.37:0/1332884473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb778101f70 msgr2=0x7fb778101870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77e000640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb778101f70 0x7fb778101870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77e000640 1 -- 172.21.15.37:0/1332884473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778100c80 con 0x7fb7781071d0 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77d600640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb778101f70 0x7fb778101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:05.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.517+0000 7fb77e000640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778100460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb77002c640 tx=0x7fb770035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:05.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.518+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770039040 con 0x7fb7781071d0 2024-03-21T09:09:05.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.518+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb770044070 con 0x7fb7781071d0 2024-03-21T09:09:05.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.518+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7781a1860 con 0x7fb7781071d0 2024-03-21T09:09:05.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.518+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700043c0 con 0x7fb7781071d0 2024-03-21T09:09:05.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.518+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7781a1d50 con 0x7fb7781071d0 2024-03-21T09:09:05.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.519+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77804f400 con 0x7fb7781071d0 2024-03-21T09:09:05.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.521+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb770004560 con 0x7fb7781071d0 2024-03-21T09:09:05.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.522+0000 7fb776000640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7600a2f20 0x7fb7600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.523+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb7700f2280 con 0x7fb7781071d0 2024-03-21T09:09:05.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.523+0000 7fb77d600640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7600a2f20 0x7fb7600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.524+0000 7fb77d600640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7600a2f20 0x7fb7600a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009fc0 tx=0x7fb76c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:05.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.528+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb7700bee10 con 0x7fb7781071d0 2024-03-21T09:09:05.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.730+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb77802d070 con 0x7fb7781071d0 2024-03-21T09:09:05.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.732+0000 7fb776000640 1 -- 172.21.15.37:0/1332884473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7700be710 con 0x7fb7781071d0 2024-03-21T09:09:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7600a2f20 msgr2=0x7fb7600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7600a2f20 0x7fb7600a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009fc0 tx=0x7fb76c005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 msgr2=0x7fb778100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778100460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb77002c640 tx=0x7fb770035d10 comp rx=0 tx=0).stop 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 shutdown_connections 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7600a2f20 0x7fb7600a53e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7781071d0 0x7fb778100460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7781048a0 0x7fb7780fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 --2- 172.21.15.37:0/1332884473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb778101f70 0x7fb778101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 >> 172.21.15.37:0/1332884473 conn(0x7fb7780fb980 msgr2=0x7fb778102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 shutdown_connections 2024-03-21T09:09:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.735+0000 7fb77ea00640 1 -- 172.21.15.37:0/1332884473 wait complete. 2024-03-21T09:09:05.749 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:09:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 -- 172.21.15.37:0/169728461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201014f0 msgr2=0x7f1e20101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 --2- 172.21.15.37:0/169728461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201014f0 0x7f1e20101910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009900 tx=0x7f1e1402bc50 comp rx=0 tx=0).stop 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 -- 172.21.15.37:0/169728461 shutdown_connections 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 --2- 172.21.15.37:0/169728461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 0x7f1e20109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 --2- 172.21.15.37:0/169728461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201026e0 0x7f1e20102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 --2- 172.21.15.37:0/169728461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201014f0 0x7f1e20101910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.975+0000 7f1e27200640 1 -- 172.21.15.37:0/169728461 >> 172.21.15.37:0/169728461 conn(0x7f1e200fca60 msgr2=0x7f1e200feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.976+0000 7f1e27200640 1 -- 172.21.15.37:0/169728461 shutdown_connections 2024-03-21T09:09:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.976+0000 7f1e27200640 1 -- 172.21.15.37:0/169728461 wait complete. 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 Processor -- start 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 -- start start 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201014f0 0x7f1e201999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 0x7f1e20199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 0x7f1e201a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010d1d0 con 0x7f1e201014f0 2024-03-21T09:09:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010d050 con 0x7f1e201030a0 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.977+0000 7f1e27200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010d350 con 0x7f1e201026e0 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e25e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201014f0 0x7f1e201999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e25400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 0x7f1e20199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e25400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 0x7f1e20199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:60400/0 (socket says 172.21.15.37:60400) 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e25400640 1 -- 172.21.15.37:0/4186555305 learned_addr learned my addr 172.21.15.37:0/4186555305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e26800640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 0x7f1e201a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e26800640 1 -- 172.21.15.37:0/4186555305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 msgr2=0x7f1e20199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e26800640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 0x7f1e20199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e26800640 1 -- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201014f0 msgr2=0x7f1e201999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e26800640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201014f0 0x7f1e201999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e26800640 1 -- 172.21.15.37:0/4186555305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e201a1610 con 0x7f1e201030a0 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e25400640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 0x7f1e20199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.978+0000 7f1e25e00640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201014f0 0x7f1e201999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.979+0000 7f1e26800640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 0x7f1e201a0fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009fd0 tx=0x7f1e1c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.979+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00ce20 con 0x7f1e201030a0 2024-03-21T09:09:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.979+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e1c005020 con 0x7f1e201030a0 2024-03-21T09:09:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.979+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e201a1900 con 0x7f1e201030a0 2024-03-21T09:09:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.979+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c010720 con 0x7f1e201030a0 2024-03-21T09:09:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.979+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201a1e50 con 0x7f1e201030a0 2024-03-21T09:09:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.980+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e201a20e0 con 0x7f1e201030a0 2024-03-21T09:09:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.982+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1e1c005540 con 0x7f1e201030a0 2024-03-21T09:09:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.985+0000 7f1e1aa00640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1e080a2f20 0x7f1e080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.985+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1e1c0cfcd0 con 0x7f1e201030a0 2024-03-21T09:09:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.985+0000 7f1e25e00640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1e080a2f20 0x7f1e080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.986+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1e1c09c860 con 0x7f1e201030a0 2024-03-21T09:09:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:05.986+0000 7f1e25e00640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1e080a2f20 0x7f1e080a53e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1e1402c190 tx=0x7f1e14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:06.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.192+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1e201a22b0 con 0x7f1e201030a0 2024-03-21T09:09:06.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.193+0000 7f1e1aa00640 1 -- 172.21.15.37:0/4186555305 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1e1c09c160 con 0x7f1e201030a0 2024-03-21T09:09:06.186 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:09:06.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1e080a2f20 msgr2=0x7f1e080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1e080a2f20 0x7f1e080a53e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1e1402c190 tx=0x7f1e14000bc0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 msgr2=0x7f1e201a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 0x7f1e201a0fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009fd0 tx=0x7f1e1c00ef10 comp rx=0 tx=0).stop 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 shutdown_connections 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1e080a2f20 0x7f1e080a53e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e201030a0 0x7f1e201a0fa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201026e0 0x7f1e20199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 --2- 172.21.15.37:0/4186555305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e201014f0 0x7f1e201999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.195+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 >> 172.21.15.37:0/4186555305 conn(0x7f1e200fca60 msgr2=0x7f1e200738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.196+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 shutdown_connections 2024-03-21T09:09:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.196+0000 7f1e27200640 1 -- 172.21.15.37:0/4186555305 wait complete. 2024-03-21T09:09:06.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T09:09:06.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:06 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/682806335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:09:06.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:06 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1332884473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:09:06.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:06 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/682806335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:09:06.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:06 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1332884473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 -- 172.21.15.37:0/3288586970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268101510 msgr2=0x7f0268101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 --2- 172.21.15.37:0/3288586970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268101510 0x7f0268101930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0258009900 tx=0x7f025802bc40 comp rx=0 tx=0).stop 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 -- 172.21.15.37:0/3288586970 shutdown_connections 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 --2- 172.21.15.37:0/3288586970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02681030c0 0x7f0268109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 --2- 172.21.15.37:0/3288586970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268102700 0x7f0268102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 --2- 172.21.15.37:0/3288586970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268101510 0x7f0268101930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 -- 172.21.15.37:0/3288586970 >> 172.21.15.37:0/3288586970 conn(0x7f02680fca80 msgr2=0x7f02680feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:06.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 -- 172.21.15.37:0/3288586970 shutdown_connections 2024-03-21T09:09:06.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.438+0000 7f026dc00640 1 -- 172.21.15.37:0/3288586970 wait complete. 2024-03-21T09:09:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.439+0000 7f026dc00640 1 Processor -- start 2024-03-21T09:09:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.439+0000 7f026dc00640 1 -- start start 2024-03-21T09:09:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 0x7f0268199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268102700 0x7f0268199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02681030c0 0x7f02681a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810d1a0 con 0x7f0268101510 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810d020 con 0x7f0268102700 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810d320 con 0x7f02681030c0 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f0267e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 0x7f0268199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f0267400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268102700 0x7f0268199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f0267e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 0x7f0268199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56374/0 (socket says 172.21.15.37:56374) 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f0267400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268102700 0x7f0268199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57242/0 (socket says 172.21.15.37:57242) 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f0267e00640 1 -- 172.21.15.37:0/1645878944 learned_addr learned my addr 172.21.15.37:0/1645878944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.440+0000 7f026d200640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02681030c0 0x7f02681a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0267e00640 1 -- 172.21.15.37:0/1645878944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02681030c0 msgr2=0x7f02681a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0267e00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02681030c0 0x7f02681a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0267e00640 1 -- 172.21.15.37:0/1645878944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268102700 msgr2=0x7f0268199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0267e00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268102700 0x7f0268199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0267e00640 1 -- 172.21.15.37:0/1645878944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02681a1650 con 0x7f0268101510 2024-03-21T09:09:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0267e00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 0x7f0268199a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0258000c00 tx=0x7f025802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258041070 con 0x7f0268101510 2024-03-21T09:09:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0258004450 con 0x7f0268101510 2024-03-21T09:09:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02681a18e0 con 0x7f0268101510 2024-03-21T09:09:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.441+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258034440 con 0x7f0268101510 2024-03-21T09:09:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.442+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02681a1dd0 con 0x7f0268101510 2024-03-21T09:09:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.443+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02681a1fd0 con 0x7f0268101510 2024-03-21T09:09:06.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.445+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0258044070 con 0x7f0268101510 2024-03-21T09:09:06.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.446+0000 7f0264c00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f024c0a2f20 0x7f024c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:06.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.446+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f02580f12f0 con 0x7f0268101510 2024-03-21T09:09:06.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.446+0000 7f0267400640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f024c0a2f20 0x7f024c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:06.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.447+0000 7f0267400640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f024c0a2f20 0x7f024c0a53e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f026819af40 tx=0x7f0254005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:06.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:06 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/682806335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:09:06.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:06 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1332884473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:09:06.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.449+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02580bd920 con 0x7f0268101510 2024-03-21T09:09:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.673+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f02681a21a0 con 0x7f0268101510 2024-03-21T09:09:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.673+0000 7f0264c00640 1 -- 172.21.15.37:0/1645878944 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+588 (secure 0 0 0) 0x7f02580c19c0 con 0x7f0268101510 2024-03-21T09:09:06.667 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2468741411 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:06.667 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/798132403 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2055653180 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1907385341 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2644652589 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/276911571 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/276911571 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3643193292 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2468741411 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/295866901 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:listed 10 entries 2024-03-21T09:09:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.676+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f024c0a2f20 msgr2=0x7f024c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.676+0000 7f026dc00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f024c0a2f20 0x7f024c0a53e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f026819af40 tx=0x7f0254005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.676+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 msgr2=0x7f0268199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.676+0000 7f026dc00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 0x7f0268199a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0258000c00 tx=0x7f025802c650 comp rx=0 tx=0).stop 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.676+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 shutdown_connections 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f024c0a2f20 0x7f024c0a53e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02681030c0 0x7f02681a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0268102700 0x7f0268199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 --2- 172.21.15.37:0/1645878944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0268101510 0x7f0268199a30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 >> 172.21.15.37:0/1645878944 conn(0x7f02680fca80 msgr2=0x7f02680738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 shutdown_connections 2024-03-21T09:09:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:06.677+0000 7f026dc00640 1 -- 172.21.15.37:0/1645878944 wait complete. 2024-03-21T09:09:06.691 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:09:06.691 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:09:06.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:09:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 -- 172.21.15.37:0/1769248872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008109890 msgr2=0x7f9008110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 --2- 172.21.15.37:0/1769248872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008109890 0x7f9008110120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9000009ca0 tx=0x7f900002c0f0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 -- 172.21.15.37:0/1769248872 shutdown_connections 2024-03-21T09:09:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 --2- 172.21.15.37:0/1769248872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008109890 0x7f9008110120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 --2- 172.21.15.37:0/1769248872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9008108ed0 0x7f9008109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 --2- 172.21.15.37:0/1769248872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008107ce0 0x7f9008108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 -- 172.21.15.37:0/1769248872 >> 172.21.15.37:0/1769248872 conn(0x7f9008076ca0 msgr2=0x7f90080790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:07.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 -- 172.21.15.37:0/1769248872 shutdown_connections 2024-03-21T09:09:07.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.028+0000 7f900ca00640 1 -- 172.21.15.37:0/1769248872 wait complete. 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 Processor -- start 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 -- start start 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 0x7f900819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9008108ed0 0x7f900819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008109890 0x7f90081a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9008113940 con 0x7f9008109890 2024-03-21T09:09:07.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90081137c0 con 0x7f9008107ce0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.029+0000 7f900ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9008113ac0 con 0x7f9008108ed0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 0x7f900819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9006800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008109890 0x7f90081a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 0x7f900819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57250/0 (socket says 172.21.15.37:57250) 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 -- 172.21.15.37:0/3514741972 learned_addr learned my addr 172.21.15.37:0/3514741972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f8fffe00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9008108ed0 0x7f900819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 -- 172.21.15.37:0/3514741972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9008108ed0 msgr2=0x7f900819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9008108ed0 0x7f900819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 -- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008109890 msgr2=0x7f90081a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008109890 0x7f90081a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 -- 172.21.15.37:0/3514741972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90081a5b80 con 0x7f9008107ce0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9006800640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008109890 0x7f90081a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.030+0000 7f9005e00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 0x7f900819df60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800e9c0 tx=0x7f8ff800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:07.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.031+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800cd80 con 0x7f9008107ce0 2024-03-21T09:09:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.031+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ff80035f0 con 0x7f9008107ce0 2024-03-21T09:09:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.031+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90081a5e70 con 0x7f9008107ce0 2024-03-21T09:09:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.031+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8010690 con 0x7f9008107ce0 2024-03-21T09:09:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.031+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90081a6330 con 0x7f9008107ce0 2024-03-21T09:09:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.032+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900810f000 con 0x7f9008107ce0 2024-03-21T09:09:07.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.034+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8ff80037a0 con 0x7f9008107ce0 2024-03-21T09:09:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.035+0000 7f8ffea00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8fec0a2fa0 0x7f8fec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.035+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8ff80cf9f0 con 0x7f9008107ce0 2024-03-21T09:09:07.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.036+0000 7f8fffe00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8fec0a2fa0 0x7f8fec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.036+0000 7f8fffe00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8fec0a2fa0 0x7f8fec0a5460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f900819f470 tx=0x7f8ff0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:07.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.040+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8ff809c580 con 0x7f9008107ce0 2024-03-21T09:09:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.246+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9008062340 con 0x7f9008107ce0 2024-03-21T09:09:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.248+0000 7f8ffea00640 1 -- 172.21.15.37:0/3514741972 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f8ff809be80 con 0x7f9008107ce0 2024-03-21T09:09:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:07.243 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:07.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8fec0a2fa0 msgr2=0x7f8fec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8fec0a2fa0 0x7f8fec0a5460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f900819f470 tx=0x7f8ff0005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 msgr2=0x7f900819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 0x7f900819df60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800e9c0 tx=0x7f8ff800ee80 comp rx=0 tx=0).stop 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 shutdown_connections 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8fec0a2fa0 0x7f8fec0a5460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9008109890 0x7f90081a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9008108ed0 0x7f900819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 --2- 172.21.15.37:0/3514741972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9008107ce0 0x7f900819df60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.251+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 >> 172.21.15.37:0/3514741972 conn(0x7f9008076ca0 msgr2=0x7f900810c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.252+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 shutdown_connections 2024-03-21T09:09:07.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.252+0000 7f900ca00640 1 -- 172.21.15.37:0/3514741972 wait complete. 2024-03-21T09:09:07.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:07 smithi037 ceph-mon[31830]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T09:09:07.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:07 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4186555305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:09:07.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:07 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1645878944' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:09:07.264 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:09:07.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:09:07.343 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:07 smithi053 ceph-mon[36169]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T09:09:07.343 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:07 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4186555305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:09:07.343 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:07 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1645878944' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:09:07.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:07 smithi071 ceph-mon[36597]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T09:09:07.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:07 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4186555305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:09:07.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:07 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1645878944' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 -- 172.21.15.37:0/1804337280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 msgr2=0x7fde64104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 --2- 172.21.15.37:0/1804337280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde64104f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde50009900 tx=0x7fde5002bc40 comp rx=0 tx=0).stop 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 -- 172.21.15.37:0/1804337280 shutdown_connections 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 --2- 172.21.15.37:0/1804337280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde64105490 0x7fde6410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 --2- 172.21.15.37:0/1804337280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde64104f50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 --2- 172.21.15.37:0/1804337280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde641038e0 0x7fde64103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 -- 172.21.15.37:0/1804337280 >> 172.21.15.37:0/1804337280 conn(0x7fde640fee50 msgr2=0x7fde64101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 -- 172.21.15.37:0/1804337280 shutdown_connections 2024-03-21T09:09:07.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.485+0000 7fde6ae00640 1 -- 172.21.15.37:0/1804337280 wait complete. 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.486+0000 7fde6ae00640 1 Processor -- start 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.486+0000 7fde6ae00640 1 -- start start 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.486+0000 7fde6ae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde641038e0 0x7fde64199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.486+0000 7fde6ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde6419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde6ae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64105490 0x7fde641a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde6ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6410f560 con 0x7fde64104ad0 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde6ae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6410f3e0 con 0x7fde641038e0 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde6ae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6410f6e0 con 0x7fde64105490 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde6419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde641038e0 0x7fde64199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde6419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56392/0 (socket says 172.21.15.37:56392) 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63400640 1 -- 172.21.15.37:0/2725753655 learned_addr learned my addr 172.21.15.37:0/2725753655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde68e00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64105490 0x7fde641a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63e00640 1 -- 172.21.15.37:0/2725753655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64105490 msgr2=0x7fde641a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63e00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64105490 0x7fde641a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63e00640 1 -- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 msgr2=0x7fde6419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63e00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde6419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.487+0000 7fde63e00640 1 -- 172.21.15.37:0/2725753655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde641a1830 con 0x7fde641038e0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.488+0000 7fde63e00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde641038e0 0x7fde64199c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9c0 tx=0x7fde5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.488+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5400ccc0 con 0x7fde641038e0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.488+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde641a1b20 con 0x7fde641038e0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.488+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde540035f0 con 0x7fde641038e0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.488+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5400ce60 con 0x7fde641038e0 2024-03-21T09:09:07.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.488+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde641a2070 con 0x7fde641038e0 2024-03-21T09:09:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.489+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6410ac00 con 0x7fde641038e0 2024-03-21T09:09:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.491+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fde540037a0 con 0x7fde641038e0 2024-03-21T09:09:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.492+0000 7fde60c00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fde480a2f20 0x7fde480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.493+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fde540cfd30 con 0x7fde641038e0 2024-03-21T09:09:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.493+0000 7fde63400640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fde480a2f20 0x7fde480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.494+0000 7fde63400640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fde480a2f20 0x7fde480a53e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde50000c00 tx=0x7fde5002c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.495+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde5409c8c0 con 0x7fde641038e0 2024-03-21T09:09:07.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.696+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde6402d070 con 0x7fde641038e0 2024-03-21T09:09:07.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.697+0000 7fde60c00640 1 -- 172.21.15.37:0/2725753655 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde5409c1c0 con 0x7fde641038e0 2024-03-21T09:09:07.691 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:09:07.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fde480a2f20 msgr2=0x7fde480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fde480a2f20 0x7fde480a53e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde50000c00 tx=0x7fde5002c890 comp rx=0 tx=0).stop 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde641038e0 msgr2=0x7fde64199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde641038e0 0x7fde64199c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9c0 tx=0x7fde5400ee80 comp rx=0 tx=0).stop 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 shutdown_connections 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fde480a2f20 0x7fde480a53e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64105490 0x7fde641a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde64104ad0 0x7fde6419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.699+0000 7fde6ae00640 1 --2- 172.21.15.37:0/2725753655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde641038e0 0x7fde64199c20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.700+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 >> 172.21.15.37:0/2725753655 conn(0x7fde640fee50 msgr2=0x7fde64107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.700+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 shutdown_connections 2024-03-21T09:09:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.700+0000 7fde6ae00640 1 -- 172.21.15.37:0/2725753655 wait complete. 2024-03-21T09:09:07.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:09:07.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.949+0000 7f86ffe00640 1 -- 172.21.15.37:0/2582149258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8700101f50 msgr2=0x7f8700104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.949+0000 7f86ffe00640 1 --2- 172.21.15.37:0/2582149258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8700101f50 0x7f8700104340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002bc40 comp rx=0 tx=0).stop 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 -- 172.21.15.37:0/2582149258 shutdown_connections 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 --2- 172.21.15.37:0/2582149258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87001071b0 0x7f8700109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 --2- 172.21.15.37:0/2582149258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700104880 0x7f8700106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 --2- 172.21.15.37:0/2582149258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8700101f50 0x7f8700104340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 -- 172.21.15.37:0/2582149258 >> 172.21.15.37:0/2582149258 conn(0x7f87000fb980 msgr2=0x7f87000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 -- 172.21.15.37:0/2582149258 shutdown_connections 2024-03-21T09:09:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.950+0000 7f86ffe00640 1 -- 172.21.15.37:0/2582149258 wait complete. 2024-03-21T09:09:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 Processor -- start 2024-03-21T09:09:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 -- start start 2024-03-21T09:09:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 0x7f8700199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8700104880 0x7f8700199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87001071b0 0x7f87001a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870010d1b0 con 0x7f8700104880 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870010d030 con 0x7f8700101f50 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.951+0000 7f86ffe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870010d330 con 0x7f87001071b0 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 0x7f8700199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 0x7f8700199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57296/0 (socket says 172.21.15.37:57296) 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86f6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8700104880 0x7f8700199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 -- 172.21.15.37:0/139360862 learned_addr learned my addr 172.21.15.37:0/139360862 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86ff400640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87001071b0 0x7f87001a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 -- 172.21.15.37:0/139360862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87001071b0 msgr2=0x7f87001a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87001071b0 0x7f87001a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 -- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8700104880 msgr2=0x7f8700199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8700104880 0x7f8700199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 -- 172.21.15.37:0/139360862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87001a1680 con 0x7f8700101f50 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.952+0000 7f86fea00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 0x7f8700199a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f86f002c8f0 tx=0x7f86f002c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.953+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0041070 con 0x7f8700101f50 2024-03-21T09:09:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.953+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f0004450 con 0x7f8700101f50 2024-03-21T09:09:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.953+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0003df0 con 0x7f8700101f50 2024-03-21T09:09:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.953+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87001a1910 con 0x7f8700101f50 2024-03-21T09:09:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.953+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a1e50 con 0x7f8700101f50 2024-03-21T09:09:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.954+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8700108d00 con 0x7f8700101f50 2024-03-21T09:09:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.956+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f86f0044070 con 0x7f8700101f50 2024-03-21T09:09:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.957+0000 7f86f7e00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f86e40a2fa0 0x7f86e40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.957+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f86f00f10f0 con 0x7f8700101f50 2024-03-21T09:09:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.958+0000 7f86f6000640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f86e40a2fa0 0x7f86e40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.958+0000 7f86f6000640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f86e40a2fa0 0x7f86e40a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f870019af70 tx=0x7f86e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:07.962+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86f00bdc80 con 0x7f8700101f50 2024-03-21T09:09:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.169+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87000622a0 con 0x7f8700101f50 2024-03-21T09:09:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.170+0000 7f86f7e00640 1 -- 172.21.15.37:0/139360862 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86f00bd580 con 0x7f8700101f50 2024-03-21T09:09:08.164 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:09:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.172+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f86e40a2fa0 msgr2=0x7f86e40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.172+0000 7f86ffe00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f86e40a2fa0 0x7f86e40a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f870019af70 tx=0x7f86e8005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.172+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 msgr2=0x7f8700199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.172+0000 7f86ffe00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 0x7f8700199a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f86f002c8f0 tx=0x7f86f002c660 comp rx=0 tx=0).stop 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 shutdown_connections 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f86e40a2fa0 0x7f86e40a5460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87001071b0 0x7f87001a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8700104880 0x7f8700199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 --2- 172.21.15.37:0/139360862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8700101f50 0x7f8700199a60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 >> 172.21.15.37:0/139360862 conn(0x7f87000fb980 msgr2=0x7f8700101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 shutdown_connections 2024-03-21T09:09:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.173+0000 7f86ffe00640 1 -- 172.21.15.37:0/139360862 wait complete. 2024-03-21T09:09:08.187 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:09:08.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:08 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3514741972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:08.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:08 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2725753655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:09:08.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:08 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3514741972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:08.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:08 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2725753655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:09:08.356 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:08 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3514741972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:08.356 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:08 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2725753655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:09:08.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 -- 172.21.15.37:0/1967060436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49241021e0 msgr2=0x7f49241045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 --2- 172.21.15.37:0/1967060436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49241021e0 0x7f49241045d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4918009900 tx=0x7f491802bc30 comp rx=0 tx=0).stop 2024-03-21T09:09:08.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 -- 172.21.15.37:0/1967060436 shutdown_connections 2024-03-21T09:09:08.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 --2- 172.21.15.37:0/1967060436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4924107440 0x7f4924109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 --2- 172.21.15.37:0/1967060436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924104b10 0x7f4924106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 --2- 172.21.15.37:0/1967060436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49241021e0 0x7f49241045d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 -- 172.21.15.37:0/1967060436 >> 172.21.15.37:0/1967060436 conn(0x7f49240fbb30 msgr2=0x7f49240fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:08.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 -- 172.21.15.37:0/1967060436 shutdown_connections 2024-03-21T09:09:08.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.419+0000 7f492c400640 1 -- 172.21.15.37:0/1967060436 wait complete. 2024-03-21T09:09:08.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.420+0000 7f492c400640 1 Processor -- start 2024-03-21T09:09:08.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.420+0000 7f492c400640 1 -- start start 2024-03-21T09:09:08.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492c400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 0x7f4924199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492c400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4924104b10 0x7f492419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492c400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4924107440 0x7f49241a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492c400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492410d330 con 0x7f4924107440 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492c400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492410d1b0 con 0x7f49241021e0 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492c400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492410d4b0 con 0x7f4924104b10 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f4929a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 0x7f4924199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f4929a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 0x7f4924199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57314/0 (socket says 172.21.15.37:57314) 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f4929a00640 1 -- 172.21.15.37:0/1107225322 learned_addr learned my addr 172.21.15.37:0/1107225322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f492a400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4924107440 0x7f49241a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.421+0000 7f4929000640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4924104b10 0x7f492419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4929a00640 1 -- 172.21.15.37:0/1107225322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4924104b10 msgr2=0x7f492419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4929a00640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4924104b10 0x7f492419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4929a00640 1 -- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4924107440 msgr2=0x7f49241a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4929a00640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4924107440 0x7f49241a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4929a00640 1 -- 172.21.15.37:0/1107225322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49241a1850 con 0x7f49241021e0 2024-03-21T09:09:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4929a00640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 0x7f4924199c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4918009230 tx=0x7f491802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918041070 con 0x7f49241021e0 2024-03-21T09:09:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49241a1ae0 con 0x7f49241021e0 2024-03-21T09:09:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4918004450 con 0x7f49241021e0 2024-03-21T09:09:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918003e20 con 0x7f49241021e0 2024-03-21T09:09:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.422+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49241a2020 con 0x7f49241021e0 2024-03-21T09:09:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.424+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4924108f30 con 0x7f49241021e0 2024-03-21T09:09:08.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.426+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4918044070 con 0x7f49241021e0 2024-03-21T09:09:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.428+0000 7f4922000640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f490c0a2f20 0x7f490c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.428+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f49180f12b0 con 0x7f49241021e0 2024-03-21T09:09:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.429+0000 7f4929000640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f490c0a2f20 0x7f490c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.429+0000 7f4929000640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f490c0a2f20 0x7f490c0a53e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4924069430 tx=0x7f4910005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:08.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.435+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49180bd7f0 con 0x7f49241021e0 2024-03-21T09:09:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.651+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49240622a0 con 0x7f49241021e0 2024-03-21T09:09:08.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.652+0000 7f4922000640 1 -- 172.21.15.37:0/1107225322 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49180c1890 con 0x7f49241021e0 2024-03-21T09:09:08.646 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f490c0a2f20 msgr2=0x7f490c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f490c0a2f20 0x7f490c0a53e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4924069430 tx=0x7f4910005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 msgr2=0x7f4924199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 0x7f4924199c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4918009230 tx=0x7f491802c650 comp rx=0 tx=0).stop 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 shutdown_connections 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f490c0a2f20 0x7f490c0a53e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4924107440 0x7f49241a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4924104b10 0x7f492419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 --2- 172.21.15.37:0/1107225322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49241021e0 0x7f4924199c40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 >> 172.21.15.37:0/1107225322 conn(0x7f49240fbb30 msgr2=0x7f49241019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 shutdown_connections 2024-03-21T09:09:08.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.655+0000 7f492c400640 1 -- 172.21.15.37:0/1107225322 wait complete. 2024-03-21T09:09:08.670 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:09:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 -- 172.21.15.37:0/1870739505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc102160 msgr2=0x7f7dcc104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/1870739505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc102160 0x7f7dcc104550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009ca0 tx=0x7f7dbc02c100 comp rx=0 tx=0).stop 2024-03-21T09:09:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 -- 172.21.15.37:0/1870739505 shutdown_connections 2024-03-21T09:09:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/1870739505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc1073c0 0x7f7dcc109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/1870739505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc104a90 0x7f7dcc106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/1870739505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc102160 0x7f7dcc104550 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 -- 172.21.15.37:0/1870739505 >> 172.21.15.37:0/1870739505 conn(0x7f7dcc0fbb70 msgr2=0x7f7dcc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 -- 172.21.15.37:0/1870739505 shutdown_connections 2024-03-21T09:09:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.895+0000 7f7dd0c00640 1 -- 172.21.15.37:0/1870739505 wait complete. 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.896+0000 7f7dd0c00640 1 Processor -- start 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.896+0000 7f7dd0c00640 1 -- start start 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dd0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 0x7f7dcc101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dd0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc104a90 0x7f7dcc100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dd0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc1073c0 0x7f7dcc100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dd0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc10d390 con 0x7f7dcc1073c0 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dd0c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc10d210 con 0x7f7dcc102160 2024-03-21T09:09:08.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dd0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc10d510 con 0x7f7dcc104a90 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 0x7f7dcc101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc104a90 0x7f7dcc100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 0x7f7dcc101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57318/0 (socket says 172.21.15.37:57318) 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dca800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc1073c0 0x7f7dcc100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9e00640 1 -- 172.21.15.37:0/4047727594 learned_addr learned my addr 172.21.15.37:0/4047727594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9400640 1 -- 172.21.15.37:0/4047727594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 msgr2=0x7f7dcc101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9400640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 0x7f7dcc101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9400640 1 -- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc1073c0 msgr2=0x7f7dcc100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9400640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc1073c0 0x7f7dcc100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9400640 1 -- 172.21.15.37:0/4047727594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc100ee0 con 0x7f7dcc104a90 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.897+0000 7f7dc9e00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 0x7f7dcc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dca800640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc1073c0 0x7f7dcc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dc9400640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc104a90 0x7f7dcc100110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7db400d6a0 tx=0x7f7db400db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4017070 con 0x7f7dcc104a90 2024-03-21T09:09:08.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7db4009820 con 0x7f7dcc104a90 2024-03-21T09:09:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4004d00 con 0x7f7dcc104a90 2024-03-21T09:09:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc1a1b20 con 0x7f7dcc104a90 2024-03-21T09:09:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.898+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc1a2020 con 0x7f7dcc104a90 2024-03-21T09:09:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.900+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc0fd660 con 0x7f7dcc104a90 2024-03-21T09:09:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.901+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7db40099d0 con 0x7f7dcc104a90 2024-03-21T09:09:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.903+0000 7f7dc2a00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7db00a2fa0 0x7f7db00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.903+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7db40ceeb0 con 0x7f7dcc104a90 2024-03-21T09:09:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.903+0000 7f7dc9e00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7db00a2fa0 0x7f7db00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.904+0000 7f7dc9e00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7db00a2fa0 0x7f7db00a5460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc004af0 tx=0x7f7dbc0325b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:08.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:08.907+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7db409b3f0 con 0x7f7dcc104a90 2024-03-21T09:09:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.124+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7dcc062340 con 0x7f7dcc104a90 2024-03-21T09:09:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.125+0000 7f7dc2a00640 1 -- 172.21.15.37:0/4047727594 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7db409f490 con 0x7f7dcc104a90 2024-03-21T09:09:09.119 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:09:09.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.127+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7db00a2fa0 msgr2=0x7f7db00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.127+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7db00a2fa0 0x7f7db00a5460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc004af0 tx=0x7f7dbc0325b0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc104a90 msgr2=0x7f7dcc100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc104a90 0x7f7dcc100110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7db400d6a0 tx=0x7f7db400db60 comp rx=0 tx=0).stop 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 shutdown_connections 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7db00a2fa0 0x7f7db00a5460 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dcc1073c0 0x7f7dcc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dcc104a90 0x7f7dcc100110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 --2- 172.21.15.37:0/4047727594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dcc102160 0x7f7dcc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 >> 172.21.15.37:0/4047727594 conn(0x7f7dcc0fbb70 msgr2=0x7f7dcc102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 shutdown_connections 2024-03-21T09:09:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.128+0000 7f7dd0c00640 1 -- 172.21.15.37:0/4047727594 wait complete. 2024-03-21T09:09:09.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T09:09:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:09 smithi071 ceph-mon[36597]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:09 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/139360862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:09:09.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:09 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1107225322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:09:09.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 -- 172.21.15.37:0/2326829281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04101fd0 msgr2=0x7fdb041043c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 --2- 172.21.15.37:0/2326829281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04101fd0 0x7fdb041043c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02bc50 comp rx=0 tx=0).stop 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 -- 172.21.15.37:0/2326829281 shutdown_connections 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 --2- 172.21.15.37:0/2326829281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04107230 0x7fdb04109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 --2- 172.21.15.37:0/2326829281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 0x7fdb04106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 --2- 172.21.15.37:0/2326829281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04101fd0 0x7fdb041043c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 -- 172.21.15.37:0/2326829281 >> 172.21.15.37:0/2326829281 conn(0x7fdb040fb980 msgr2=0x7fdb040fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 -- 172.21.15.37:0/2326829281 shutdown_connections 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.372+0000 7fdb0a600640 1 -- 172.21.15.37:0/2326829281 wait complete. 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.373+0000 7fdb0a600640 1 Processor -- start 2024-03-21T09:09:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.373+0000 7fdb0a600640 1 -- start start 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.373+0000 7fdb0a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04101fd0 0x7fdb041999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.373+0000 7fdb0a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 0x7fdb04199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb0a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04107230 0x7fdb041a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb0a600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0410d1d0 con 0x7fdb04101fd0 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb0a600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0410d050 con 0x7fdb04104900 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb0a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0410d350 con 0x7fdb04107230 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb09200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04101fd0 0x7fdb041999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb09200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04101fd0 0x7fdb041999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56466/0 (socket says 172.21.15.37:56466) 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb09200640 1 -- 172.21.15.37:0/4261397830 learned_addr learned my addr 172.21.15.37:0/4261397830 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb03e00640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 0x7fdb04199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb09c00640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04107230 0x7fdb041a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb03e00640 1 -- 172.21.15.37:0/4261397830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04107230 msgr2=0x7fdb041a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb03e00640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04107230 0x7fdb041a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb03e00640 1 -- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04101fd0 msgr2=0x7fdb041999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb03e00640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04101fd0 0x7fdb041999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.374+0000 7fdb03e00640 1 -- 172.21.15.37:0/4261397830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb041a1600 con 0x7fdb04104900 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.375+0000 7fdb03e00640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 0x7fdb04199f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005fd0 tx=0x7fdaf400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.375+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4010430 con 0x7fdb04104900 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.375+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdaf4010ab0 con 0x7fdb04104900 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.375+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4017b70 con 0x7fdb04104900 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.375+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb041a18f0 con 0x7fdb04104900 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.375+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb041a1e40 con 0x7fdb04104900 2024-03-21T09:09:09.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.376+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb04108d20 con 0x7fdb04104900 2024-03-21T09:09:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.378+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdaf4017d10 con 0x7fdb04104900 2024-03-21T09:09:09.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.380+0000 7fdb01600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdae80a2f20 0x7fdae80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.380+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdaf40cff90 con 0x7fdb04104900 2024-03-21T09:09:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.380+0000 7fdb09200640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdae80a2f20 0x7fdae80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:09.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.381+0000 7fdb09200640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdae80a2f20 0x7fdae80a53e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0096c0 tx=0x7fdaec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.388+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdaf409cb20 con 0x7fdb04104900 2024-03-21T09:09:09.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:09 smithi037 ceph-mon[31830]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:09.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:09 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/139360862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:09:09.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:09 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1107225322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:09:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:09 smithi053 ceph-mon[36169]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:09 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/139360862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:09:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:09 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1107225322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:09:09.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.661+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdb040622a0 con 0x7fdb04104900 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.662+0000 7fdb01600640 1 -- 172.21.15.37:0/4261397830 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+588 (secure 0 0 0) 0x7fdaf409c420 con 0x7fdb04104900 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2468741411 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/798132403 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2055653180 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1907385341 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2644652589 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/276911571 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:09.656 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/276911571 2024-03-22T09:00:11.515361+0000 2024-03-21T09:09:09.657 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3643193292 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:09.657 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2468741411 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:09.657 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/295866901 2024-03-22T09:00:45.623535+0000 2024-03-21T09:09:09.657 INFO:teuthology.orchestra.run.smithi037.stderr:listed 10 entries 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdae80a2f20 msgr2=0x7fdae80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdae80a2f20 0x7fdae80a53e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0096c0 tx=0x7fdaec000bc0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 msgr2=0x7fdb04199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 0x7fdb04199f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005fd0 tx=0x7fdaf400ef90 comp rx=0 tx=0).stop 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 shutdown_connections 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdae80a2f20 0x7fdae80a53e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04107230 0x7fdb041a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb04104900 0x7fdb04199f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 --2- 172.21.15.37:0/4261397830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb04101fd0 0x7fdb041999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 >> 172.21.15.37:0/4261397830 conn(0x7fdb040fb980 msgr2=0x7fdb04101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 shutdown_connections 2024-03-21T09:09:09.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:09.665+0000 7fdb0a600640 1 -- 172.21.15.37:0/4261397830 wait complete. 2024-03-21T09:09:09.686 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-03-21T09:09:09.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:09.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-21T09:09:09.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[72815]: fuse finished with error 0 and tester_r 0 2024-03-21T09:09:09.766 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T09:09:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:10 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4047727594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:09:10.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:10 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4261397830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:09:10.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:10 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4047727594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:09:10.369 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:10 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4261397830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:09:10.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:10 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4047727594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:09:10.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:10 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4261397830' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:09:11.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:11 smithi071 ceph-mon[36597]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:11 smithi037 ceph-mon[31830]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:11.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:11 smithi053 ceph-mon[36169]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:13.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:13 smithi071 ceph-mon[36597]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:13.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:13 smithi037 ceph-mon[31830]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:13.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:13 smithi053 ceph-mon[36169]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:15.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:15 smithi071 ceph-mon[36597]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:15 smithi037 ceph-mon[31830]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T09:09:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:15 smithi053 ceph-mon[36169]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T09:09:15.868 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:09:15.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:09:15.869 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-21T09:09:15.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:09:16.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 -- 172.21.15.37:0/4240900643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 msgr2=0x7faa141096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 --2- 172.21.15.37:0/4240900643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141096e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009be0 tx=0x7faa0c02bef0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 -- 172.21.15.37:0/4240900643 shutdown_connections 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 --2- 172.21.15.37:0/4240900643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141096e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 --2- 172.21.15.37:0/4240900643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa14104960 0x7faa14106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 --2- 172.21.15.37:0/4240900643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 0x7faa14104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.163+0000 7faa1a600640 1 -- 172.21.15.37:0/4240900643 >> 172.21.15.37:0/4240900643 conn(0x7faa140fba20 msgr2=0x7faa140fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.164+0000 7faa1a600640 1 -- 172.21.15.37:0/4240900643 shutdown_connections 2024-03-21T09:09:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.164+0000 7faa1a600640 1 -- 172.21.15.37:0/4240900643 wait complete. 2024-03-21T09:09:16.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.165+0000 7faa1a600640 1 Processor -- start 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.165+0000 7faa1a600640 1 -- start start 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.165+0000 7faa1a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 0x7faa14199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.165+0000 7faa1a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa14104960 0x7faa1419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.165+0000 7faa1a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa1a600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1410d240 con 0x7faa14102030 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa1a600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1410d0c0 con 0x7faa14107290 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa1a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1410d3c0 con 0x7faa14104960 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 0x7faa14199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 0x7faa14199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44670/0 (socket says 172.21.15.37:44670) 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19200640 1 -- 172.21.15.37:0/681601132 learned_addr learned my addr 172.21.15.37:0/681601132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19c00640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa0be00640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa14104960 0x7faa1419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19c00640 1 -- 172.21.15.37:0/681601132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa14104960 msgr2=0x7faa1419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19c00640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa14104960 0x7faa1419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19c00640 1 -- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 msgr2=0x7faa14199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19c00640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 0x7faa14199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.166+0000 7faa19c00640 1 -- 172.21.15.37:0/681601132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa141a1680 con 0x7faa14107290 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.167+0000 7faa19c00640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141a1010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa0c02c430 tx=0x7faa0c0337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.167+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c043020 con 0x7faa14107290 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.167+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa0c02ca60 con 0x7faa14107290 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.167+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c034690 con 0x7faa14107290 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.167+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa141a1910 con 0x7faa14107290 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.167+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa141a1e10 con 0x7faa14107290 2024-03-21T09:09:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.168+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa14108d80 con 0x7faa14107290 2024-03-21T09:09:16.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.170+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faa0c046020 con 0x7faa14107290 2024-03-21T09:09:16.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.171+0000 7faa12000640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa9f40a2f10 0x7fa9f40a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.171+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7faa0c0f1c80 con 0x7faa14107290 2024-03-21T09:09:16.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.172+0000 7faa19200640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa9f40a2f10 0x7fa9f40a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.172+0000 7faa19200640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa9f40a2f10 0x7fa9f40a53d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa0400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:16.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.176+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa0c04d020 con 0x7faa14107290 2024-03-21T09:09:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.379+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa140622a0 con 0x7faa14107290 2024-03-21T09:09:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.381+0000 7faa12000640 1 -- 172.21.15.37:0/681601132 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7faa0c0be810 con 0x7faa14107290 2024-03-21T09:09:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:16.375 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.383+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa9f40a2f10 msgr2=0x7fa9f40a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.383+0000 7faa1a600640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa9f40a2f10 0x7fa9f40a53d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa0400b4a0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.383+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 msgr2=0x7faa141a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.383+0000 7faa1a600640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141a1010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa0c02c430 tx=0x7faa0c0337b0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 shutdown_connections 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa9f40a2f10 0x7fa9f40a53d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa14107290 0x7faa141a1010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa14104960 0x7faa1419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 --2- 172.21.15.37:0/681601132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa14102030 0x7faa14199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 >> 172.21.15.37:0/681601132 conn(0x7faa140fba20 msgr2=0x7faa141017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 shutdown_connections 2024-03-21T09:09:16.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.384+0000 7faa1a600640 1 -- 172.21.15.37:0/681601132 wait complete. 2024-03-21T09:09:16.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:09:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 -- 172.21.15.37:0/3061869228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 msgr2=0x7f8b500784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 --2- 172.21.15.37:0/3061869228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b500784e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009be0 tx=0x7f8b4802bef0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 -- 172.21.15.37:0/3061869228 shutdown_connections 2024-03-21T09:09:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 --2- 172.21.15.37:0/3061869228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b500784e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 --2- 172.21.15.37:0/3061869228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b50077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 --2- 172.21.15.37:0/3061869228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 0x7f8b50076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 -- 172.21.15.37:0/3061869228 >> 172.21.15.37:0/3061869228 conn(0x7f8b500feeb0 msgr2=0x7f8b501012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 -- 172.21.15.37:0/3061869228 shutdown_connections 2024-03-21T09:09:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.655+0000 7f8b58000640 1 -- 172.21.15.37:0/3061869228 wait complete. 2024-03-21T09:09:16.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.656+0000 7f8b58000640 1 Processor -- start 2024-03-21T09:09:16.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.656+0000 7f8b58000640 1 -- start start 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b58000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 0x7f8b5019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b58000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b5019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b58000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b501a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b58000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b501139a0 con 0x7f8b500772a0 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b58000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50113820 con 0x7f8b50077c40 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b58000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50113b20 con 0x7f8b50076060 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b54c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b5019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b54c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b5019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44696/0 (socket says 172.21.15.37:44696) 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b54c00640 1 -- 172.21.15.37:0/2593983776 learned_addr learned my addr 172.21.15.37:0/2593983776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b55600640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 0x7f8b5019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.657+0000 7f8b56000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b501a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b54c00640 1 -- 172.21.15.37:0/2593983776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 msgr2=0x7f8b5019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b54c00640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 0x7f8b5019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b54c00640 1 -- 172.21.15.37:0/2593983776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 msgr2=0x7f8b501a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b54c00640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b501a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b54c00640 1 -- 172.21.15.37:0/2593983776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b501a5b70 con 0x7f8b500772a0 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b55600640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 0x7f8b5019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b56000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b501a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b54c00640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b5019e4e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400e9f0 tx=0x7f8b4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400cd80 con 0x7f8b500772a0 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b44010910 con 0x7f8b500772a0 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b501a5e60 con 0x7f8b500772a0 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.658+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440188f0 con 0x7f8b500772a0 2024-03-21T09:09:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.659+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b501a6320 con 0x7f8b500772a0 2024-03-21T09:09:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.660+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b501102b0 con 0x7f8b500772a0 2024-03-21T09:09:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.662+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8b44010ac0 con 0x7f8b500772a0 2024-03-21T09:09:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.664+0000 7f8b4e000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b380a2f20 0x7f8b380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.664+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8b440cfd80 con 0x7f8b500772a0 2024-03-21T09:09:16.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.664+0000 7f8b55600640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b380a2f20 0x7f8b380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:16.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.665+0000 7f8b55600640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b380a2f20 0x7f8b380a53e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b400096c0 tx=0x7f8b40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.666+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b4409c2c0 con 0x7f8b500772a0 2024-03-21T09:09:16.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.906+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b5002d2c0 con 0x7f8b500772a0 2024-03-21T09:09:16.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.909+0000 7f8b4e000640 1 -- 172.21.15.37:0/2593983776 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f8b440a0360 con 0x7f8b500772a0 2024-03-21T09:09:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:16.903 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b380a2f20 msgr2=0x7f8b380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b380a2f20 0x7f8b380a53e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b400096c0 tx=0x7f8b40009260 comp rx=0 tx=0).stop 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 msgr2=0x7f8b5019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b5019e4e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400e9f0 tx=0x7f8b4400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 shutdown_connections 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8b380a2f20 0x7f8b380a53e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b50077c40 0x7f8b501a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b500772a0 0x7f8b5019e4e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 --2- 172.21.15.37:0/2593983776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b50076060 0x7f8b5019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:16.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 >> 172.21.15.37:0/2593983776 conn(0x7f8b500feeb0 msgr2=0x7f8b501009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:16.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 shutdown_connections 2024-03-21T09:09:16.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:16.912+0000 7f8b58000640 1 -- 172.21.15.37:0/2593983776 wait complete. 2024-03-21T09:09:16.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-21T09:09:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 -- 172.21.15.37:0/4093572123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe058102170 msgr2=0x7fe058108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 --2- 172.21.15.37:0/4093572123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe058102170 0x7fe058108a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe050009be0 tx=0x7fe05002bef0 comp rx=0 tx=0).stop 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 -- 172.21.15.37:0/4093572123 shutdown_connections 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 --2- 172.21.15.37:0/4093572123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe058102170 0x7fe058108a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 --2- 172.21.15.37:0/4093572123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0581017b0 0x7fe058101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 --2- 172.21.15.37:0/4093572123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581005c0 0x7fe0581009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 -- 172.21.15.37:0/4093572123 >> 172.21.15.37:0/4093572123 conn(0x7fe0580fbb50 msgr2=0x7fe0580fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 -- 172.21.15.37:0/4093572123 shutdown_connections 2024-03-21T09:09:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.146+0000 7fe05d000640 1 -- 172.21.15.37:0/4093572123 wait complete. 2024-03-21T09:09:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.148+0000 7fe05d000640 1 Processor -- start 2024-03-21T09:09:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.148+0000 7fe05d000640 1 -- start start 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe05d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 0x7fe058196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe05d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581017b0 0x7fe058196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe05d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe058102170 0x7fe05819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe05d000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05810c270 con 0x7fe058102170 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe05d000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05810c0f0 con 0x7fe0581005c0 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe05d000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05810c3f0 con 0x7fe0581017b0 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 0x7fe058196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 0x7fe058196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33380/0 (socket says 172.21.15.37:33380) 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 -- 172.21.15.37:0/3704348596 learned_addr learned my addr 172.21.15.37:0/3704348596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe056800640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe058102170 0x7fe05819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe04fe00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581017b0 0x7fe058196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 -- 172.21.15.37:0/3704348596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581017b0 msgr2=0x7fe058196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581017b0 0x7fe058196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 -- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe058102170 msgr2=0x7fe05819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe058102170 0x7fe05819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe055e00640 1 -- 172.21.15.37:0/3704348596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05819e470 con 0x7fe0581005c0 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.149+0000 7fe04fe00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581017b0 0x7fe058196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.150+0000 7fe055e00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 0x7fe058196850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe04800e9a0 tx=0x7fe04800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.150+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800cd80 con 0x7fe0581005c0 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.150+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe048010910 con 0x7fe0581005c0 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.150+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048018a30 con 0x7fe0581005c0 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.150+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05819e760 con 0x7fe0581005c0 2024-03-21T09:09:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.150+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05819ed00 con 0x7fe0581005c0 2024-03-21T09:09:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.152+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0581078e0 con 0x7fe0581005c0 2024-03-21T09:09:17.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.153+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe048010ac0 con 0x7fe0581005c0 2024-03-21T09:09:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.154+0000 7fe04ea00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe03c0a2fa0 0x7fe03c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.155+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe0480cfa10 con 0x7fe0581005c0 2024-03-21T09:09:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.155+0000 7fe04fe00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe03c0a2fa0 0x7fe03c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.156+0000 7fe04fe00640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe03c0a2fa0 0x7fe03c0a5460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe058197d60 tx=0x7fe040005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.159+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe048053740 con 0x7fe0581005c0 2024-03-21T09:09:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:17 smithi071 ceph-mon[36597]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:17 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/681601132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:17.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:17 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2593983776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:17.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:17 smithi037 ceph-mon[31830]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:17.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:17 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/681601132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:17.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:17 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2593983776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:17.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:17 smithi053 ceph-mon[36169]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:17.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:17 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/681601132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:17.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:17 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2593983776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:17.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:17.501+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe05802d430 con 0x7fe0581005c0 2024-03-21T09:09:18.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.071+0000 7fe04ea00640 1 -- 172.21.15.37:0/3704348596 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe04809c5a0 con 0x7fe0581005c0 2024-03-21T09:09:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe03c0a2fa0 msgr2=0x7fe03c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe03c0a2fa0 0x7fe03c0a5460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe058197d60 tx=0x7fe040005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 msgr2=0x7fe058196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 0x7fe058196850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe04800e9a0 tx=0x7fe04800ee60 comp rx=0 tx=0).stop 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 shutdown_connections 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe03c0a2fa0 0x7fe03c0a5460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe058102170 0x7fe05819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0581017b0 0x7fe058196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 --2- 172.21.15.37:0/3704348596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0581005c0 0x7fe058196850 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.074+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 >> 172.21.15.37:0/3704348596 conn(0x7fe0580fbb50 msgr2=0x7fe058104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.075+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 shutdown_connections 2024-03-21T09:09:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.075+0000 7fe05d000640 1 -- 172.21.15.37:0/3704348596 wait complete. 2024-03-21T09:09:18.092 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-21T09:09:18.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-21T09:09:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 -- 172.21.15.37:0/3778520290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38380756e0 msgr2=0x7f3838073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 --2- 172.21.15.37:0/3778520290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38380756e0 0x7f3838073b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3828009930 tx=0x7f382802bc30 comp rx=0 tx=0).stop 2024-03-21T09:09:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 -- 172.21.15.37:0/3778520290 shutdown_connections 2024-03-21T09:09:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 --2- 172.21.15.37:0/3778520290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f383810dac0 0x7f383810feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 --2- 172.21.15.37:0/3778520290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3838074100 0x7f3838074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 --2- 172.21.15.37:0/3778520290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38380756e0 0x7f3838073b30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 -- 172.21.15.37:0/3778520290 >> 172.21.15.37:0/3778520290 conn(0x7f38380fca60 msgr2=0x7f38380feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 -- 172.21.15.37:0/3778520290 shutdown_connections 2024-03-21T09:09:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.330+0000 7f383da00640 1 -- 172.21.15.37:0/3778520290 wait complete. 2024-03-21T09:09:18.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.331+0000 7f383da00640 1 Processor -- start 2024-03-21T09:09:18.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.331+0000 7f383da00640 1 -- start start 2024-03-21T09:09:18.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.331+0000 7f383da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 0x7f383819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:18.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38380756e0 0x7f383819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f383810dac0 0x7f38381a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838113790 con 0x7f38380756e0 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838113610 con 0x7f383810dac0 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838113910 con 0x7f3838074100 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 0x7f383819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f382fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38380756e0 0x7f383819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 0x7f383819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56644/0 (socket says 172.21.15.37:56644) 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 -- 172.21.15.37:0/217062950 learned_addr learned my addr 172.21.15.37:0/217062950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383d000640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f383810dac0 0x7f38381a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 -- 172.21.15.37:0/217062950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f383810dac0 msgr2=0x7f38381a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f383810dac0 0x7f38381a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 -- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38380756e0 msgr2=0x7f383819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38380756e0 0x7f383819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f3837e00640 1 -- 172.21.15.37:0/217062950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38381a5a60 con 0x7f3838074100 2024-03-21T09:09:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.332+0000 7f383d000640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f383810dac0 0x7f38381a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:18.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.333+0000 7f3837e00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 0x7f383819de40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3828000c00 tx=0x7f382802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:18.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.333+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828043020 con 0x7f3838074100 2024-03-21T09:09:18.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.333+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38381a5cf0 con 0x7f3838074100 2024-03-21T09:09:18.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.333+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38280054e0 con 0x7f3838074100 2024-03-21T09:09:18.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.333+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828003ce0 con 0x7f3838074100 2024-03-21T09:09:18.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.333+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381a61b0 con 0x7f3838074100 2024-03-21T09:09:18.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.334+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3838103c90 con 0x7f3838074100 2024-03-21T09:09:18.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.336+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3828046020 con 0x7f3838074100 2024-03-21T09:09:18.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.337+0000 7f3835600640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f381c0a2f20 0x7f381c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:18.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.338+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f38280f1740 con 0x7f3838074100 2024-03-21T09:09:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.338+0000 7f382fe00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f381c0a2f20 0x7f381c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.339+0000 7f382fe00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f381c0a2f20 0x7f381c0a53e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f383819f350 tx=0x7f3820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.342+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f382804d020 con 0x7f3838074100 2024-03-21T09:09:18.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:18 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3704348596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T09:09:18.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:18 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T09:09:18.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:18 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3704348596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T09:09:18.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:18 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T09:09:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:18 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3704348596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T09:09:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:18 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T09:09:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:18.589+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f383802d070 con 0x7f3838074100 2024-03-21T09:09:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.085+0000 7f3835600640 1 -- 172.21.15.37:0/217062950 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f38280be2d0 con 0x7f3838074100 2024-03-21T09:09:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f381c0a2f20 msgr2=0x7f381c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f381c0a2f20 0x7f381c0a53e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f383819f350 tx=0x7f3820005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 msgr2=0x7f383819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 0x7f383819de40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3828000c00 tx=0x7f382802c6c0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 shutdown_connections 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f381c0a2f20 0x7f381c0a53e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f383810dac0 0x7f38381a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38380756e0 0x7f383819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 --2- 172.21.15.37:0/217062950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3838074100 0x7f383819de40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.090+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 >> 172.21.15.37:0/217062950 conn(0x7f38380fca60 msgr2=0x7f3838100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.091+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 shutdown_connections 2024-03-21T09:09:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.091+0000 7f383da00640 1 -- 172.21.15.37:0/217062950 wait complete. 2024-03-21T09:09:19.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-21T09:09:19.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 -- 172.21.15.37:0/735049759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 msgr2=0x7f71c8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:19.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 --2- 172.21.15.37:0/735049759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c8076480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009860 tx=0x7f71b802bb70 comp rx=0 tx=0).stop 2024-03-21T09:09:19.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 -- 172.21.15.37:0/735049759 shutdown_connections 2024-03-21T09:09:19.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 --2- 172.21.15.37:0/735049759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c8077c40 0x7f71c80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 --2- 172.21.15.37:0/735049759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71c80772a0 0x7f71c8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 --2- 172.21.15.37:0/735049759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c8076480 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 -- 172.21.15.37:0/735049759 >> 172.21.15.37:0/735049759 conn(0x7f71c80fee50 msgr2=0x7f71c8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:19.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 -- 172.21.15.37:0/735049759 shutdown_connections 2024-03-21T09:09:19.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.327+0000 7f71cd400640 1 -- 172.21.15.37:0/735049759 wait complete. 2024-03-21T09:09:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.328+0000 7f71cd400640 1 Processor -- start 2024-03-21T09:09:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 -- start start 2024-03-21T09:09:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80772a0 0x7f71c819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71c8077c40 0x7f71c81a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8113970 con 0x7f71c80772a0 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81137f0 con 0x7f71c8077c40 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.329+0000 7f71cd400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8113af0 con 0x7f71c8076060 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80772a0 0x7f71c819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56656/0 (socket says 172.21.15.37:56656) 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 -- 172.21.15.37:0/2578161321 learned_addr learned my addr 172.21.15.37:0/2578161321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c7200640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71c8077c40 0x7f71c81a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:19.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 -- 172.21.15.37:0/2578161321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71c8077c40 msgr2=0x7f71c81a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:19.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71c8077c40 0x7f71c81a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 -- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80772a0 msgr2=0x7f71c819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:19.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80772a0 0x7f71c819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 -- 172.21.15.37:0/2578161321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c81a5c20 con 0x7f71c8076060 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.330+0000 7f71c6800640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c819e000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005940 tx=0x7f71b8032470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.331+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8041020 con 0x7f71c8076060 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.331+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71b802c9f0 con 0x7f71c8076060 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.331+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8039040 con 0x7f71c8076060 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.331+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c81a5eb0 con 0x7f71c8076060 2024-03-21T09:09:19.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.331+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c81a62c0 con 0x7f71c8076060 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.332+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c8110440 con 0x7f71c8076060 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.334+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f71b8044020 con 0x7f71c8076060 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.335+0000 7f71bf400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f71ac0a2fa0 0x7f71ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.336+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f71b80f1390 con 0x7f71c8076060 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.336+0000 7f71bfe00640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f71ac0a2fa0 0x7f71ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.337+0000 7f71bfe00640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f71ac0a2fa0 0x7f71ac0a5460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71c819f510 tx=0x7f71b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.338+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71b804b020 con 0x7f71c8076060 2024-03-21T09:09:19.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:19 smithi071 ceph-mon[36597]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:19.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:19 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T09:09:19.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:19 smithi071 ceph-mon[36597]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:09:19.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:19 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/217062950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T09:09:19.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:19 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T09:09:19.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:19 smithi037 ceph-mon[31830]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:19 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T09:09:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:19 smithi037 ceph-mon[31830]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:09:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:19 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/217062950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T09:09:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:19 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T09:09:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:19 smithi053 ceph-mon[36169]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T09:09:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:19 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T09:09:19.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:19 smithi053 ceph-mon[36169]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:09:19.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:19 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/217062950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T09:09:19.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:19 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-21T09:09:19.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:19.585+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f71c802d070 con 0x7f71c8076060 2024-03-21T09:09:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.096+0000 7f71bf400640 1 -- 172.21.15.37:0/2578161321 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f71b80bdfa0 con 0x7f71c8076060 2024-03-21T09:09:20.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.100+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f71ac0a2fa0 msgr2=0x7f71ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.100+0000 7f71cd400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f71ac0a2fa0 0x7f71ac0a5460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71c819f510 tx=0x7f71b0005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:20.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.100+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 msgr2=0x7f71c819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.100+0000 7f71cd400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c819e000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005940 tx=0x7f71b8032470 comp rx=0 tx=0).stop 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 shutdown_connections 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f71ac0a2fa0 0x7f71ac0a5460 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71c8077c40 0x7f71c81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c80772a0 0x7f71c819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 --2- 172.21.15.37:0/2578161321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c8076060 0x7f71c819e000 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 >> 172.21.15.37:0/2578161321 conn(0x7f71c80fee50 msgr2=0x7f71c81005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 shutdown_connections 2024-03-21T09:09:20.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.101+0000 7f71cd400640 1 -- 172.21.15.37:0/2578161321 wait complete. 2024-03-21T09:09:20.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.323+0000 7f9077000640 1 -- 172.21.15.37:0/3551387469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 msgr2=0x7f9070104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.323+0000 7f9077000640 1 --2- 172.21.15.37:0/3551387469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f9070104f50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f905c009900 tx=0x7f905c02bc40 comp rx=0 tx=0).stop 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 -- 172.21.15.37:0/3551387469 shutdown_connections 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 --2- 172.21.15.37:0/3551387469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f907010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 --2- 172.21.15.37:0/3551387469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f9070104f50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 --2- 172.21.15.37:0/3551387469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90701038e0 0x7f9070103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 -- 172.21.15.37:0/3551387469 >> 172.21.15.37:0/3551387469 conn(0x7f90700fee50 msgr2=0x7f9070101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 -- 172.21.15.37:0/3551387469 shutdown_connections 2024-03-21T09:09:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.324+0000 7f9077000640 1 -- 172.21.15.37:0/3551387469 wait complete. 2024-03-21T09:09:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 Processor -- start 2024-03-21T09:09:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 -- start start 2024-03-21T09:09:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90701038e0 0x7f9070199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f907019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f90701a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907010f560 con 0x7f9070105490 2024-03-21T09:09:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907010f3e0 con 0x7f9070104ad0 2024-03-21T09:09:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9077000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907010f6e0 con 0x7f90701038e0 2024-03-21T09:09:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9075000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f90701a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f906f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f907019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9075000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f90701a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44778/0 (socket says 172.21.15.37:44778) 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f9075000640 1 -- 172.21.15.37:0/3870562253 learned_addr learned my addr 172.21.15.37:0/3870562253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f906f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f907019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33442/0 (socket says 172.21.15.37:33442) 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.325+0000 7f906fe00640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90701038e0 0x7f9070199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9075000640 1 -- 172.21.15.37:0/3870562253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90701038e0 msgr2=0x7f9070199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9075000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90701038e0 0x7f9070199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9075000640 1 -- 172.21.15.37:0/3870562253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 msgr2=0x7f907019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9075000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f907019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9075000640 1 -- 172.21.15.37:0/3870562253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90701a1830 con 0x7f9070105490 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f906f400640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f907019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9075000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f90701a1190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb10 tx=0x7f906800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068004b20 con 0x7f9070105490 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9068004cc0 con 0x7f9070105490 2024-03-21T09:09:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90701a1b20 con 0x7f9070105490 2024-03-21T09:09:20.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068010760 con 0x7f9070105490 2024-03-21T09:09:20.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.326+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90701a2070 con 0x7f9070105490 2024-03-21T09:09:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.328+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907010ac00 con 0x7f9070105490 2024-03-21T09:09:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.330+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9068005020 con 0x7f9070105490 2024-03-21T09:09:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.331+0000 7f906cc00640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90540a2fa0 0x7f90540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.331+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f90680cfcc0 con 0x7f9070105490 2024-03-21T09:09:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.331+0000 7f906fe00640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90540a2fa0 0x7f90540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.332+0000 7f906fe00640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90540a2fa0 0x7f90540a5460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f907019b0f0 tx=0x7f9060005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:20.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.335+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f90680d6080 con 0x7f9070105490 2024-03-21T09:09:20.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2578161321' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T09:09:20.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:20 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2578161321' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T09:09:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:20 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-21T09:09:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2578161321' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T09:09:20.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:20 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T09:09:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.575+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f9070062340 con 0x7f9070105490 2024-03-21T09:09:20.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.798+0000 7f906cc00640 1 -- 172.21.15.37:0/3870562253 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f906809c850 con 0x7f9070105490 2024-03-21T09:09:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90540a2fa0 msgr2=0x7f90540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90540a2fa0 0x7f90540a5460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f907019b0f0 tx=0x7f9060005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 msgr2=0x7f90701a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f90701a1190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb10 tx=0x7f906800c2d0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 shutdown_connections 2024-03-21T09:09:20.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90540a2fa0 0x7f90540a5460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070105490 0x7f90701a1190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9070104ad0 0x7f907019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 --2- 172.21.15.37:0/3870562253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90701038e0 0x7f9070199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 >> 172.21.15.37:0/3870562253 conn(0x7f90700fee50 msgr2=0x7f9070107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.801+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 shutdown_connections 2024-03-21T09:09:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:20.802+0000 7f9077000640 1 -- 172.21.15.37:0/3870562253 wait complete. 2024-03-21T09:09:20.815 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-21T09:09:20.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-21T09:09:21.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 -- 172.21.15.37:0/1313489439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 msgr2=0x7f6a9c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 --2- 172.21.15.37:0/1313489439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c101900 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009ca0 tx=0x7f6a9402c100 comp rx=0 tx=0).stop 2024-03-21T09:09:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 -- 172.21.15.37:0/1313489439 shutdown_connections 2024-03-21T09:09:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 --2- 172.21.15.37:0/1313489439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a9c103090 0x7f6a9c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 --2- 172.21.15.37:0/1313489439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a9c1026d0 0x7f6a9c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 --2- 172.21.15.37:0/1313489439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c101900 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 -- 172.21.15.37:0/1313489439 >> 172.21.15.37:0/1313489439 conn(0x7f6a9c0fca50 msgr2=0x7f6a9c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 -- 172.21.15.37:0/1313489439 shutdown_connections 2024-03-21T09:09:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.026+0000 7f6a9be00640 1 -- 172.21.15.37:0/1313489439 wait complete. 2024-03-21T09:09:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.027+0000 7f6a9be00640 1 Processor -- start 2024-03-21T09:09:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.027+0000 7f6a9be00640 1 -- start start 2024-03-21T09:09:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a9c1026d0 0x7f6a9c073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a9c103090 0x7f6a9c073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c10d170 con 0x7f6a9c103090 2024-03-21T09:09:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c10cff0 con 0x7f6a9c1026d0 2024-03-21T09:09:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c10d2f0 con 0x7f6a9c1014e0 2024-03-21T09:09:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a9c103090 0x7f6a9c073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56700/0 (socket says 172.21.15.37:56700) 2024-03-21T09:09:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a9c1026d0 0x7f6a9c073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.028+0000 7f6a9aa00640 1 -- 172.21.15.37:0/3100343412 learned_addr learned my addr 172.21.15.37:0/3100343412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9aa00640 1 -- 172.21.15.37:0/3100343412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a9c1026d0 msgr2=0x7f6a9c073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9aa00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a9c1026d0 0x7f6a9c073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9aa00640 1 -- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a9c103090 msgr2=0x7f6a9c073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9aa00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a9c103090 0x7f6a9c073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9aa00640 1 -- 172.21.15.37:0/3100343412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c074860 con 0x7f6a9c1014e0 2024-03-21T09:09:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9aa00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c075450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a94000f80 tx=0x7f6a94035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94039040 con 0x7f6a9c1014e0 2024-03-21T09:09:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a94044070 con 0x7f6a9c1014e0 2024-03-21T09:09:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a940044a0 con 0x7f6a9c1014e0 2024-03-21T09:09:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c1a1880 con 0x7f6a9c1014e0 2024-03-21T09:09:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.029+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c1a1d50 con 0x7f6a9c1014e0 2024-03-21T09:09:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.030+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c108800 con 0x7f6a9c1014e0 2024-03-21T09:09:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.033+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6a94004640 con 0x7f6a9c1014e0 2024-03-21T09:09:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.034+0000 7f6a93400640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6a800a2f90 0x7f6a800a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.034+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6a940f1b90 con 0x7f6a9c1014e0 2024-03-21T09:09:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.035+0000 7f6a9a000640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6a800a2f90 0x7f6a800a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.035+0000 7f6a9a000640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6a800a2f90 0x7f6a800a5450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0751c0 tx=0x7f6a8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.038+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a940be720 con 0x7f6a9c1014e0 2024-03-21T09:09:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.271+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a9c061ae0 con 0x7f6a800a2f90 2024-03-21T09:09:21.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.297+0000 7f6a93400640 1 -- 172.21.15.37:0/3100343412 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6a9c061ae0 con 0x7f6a800a2f90 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6a800a2f90 msgr2=0x7f6a800a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6a800a2f90 0x7f6a800a5450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0751c0 tx=0x7f6a8400b390 comp rx=0 tx=0).stop 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 msgr2=0x7f6a9c075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c075450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a94000f80 tx=0x7f6a94035d10 comp rx=0 tx=0).stop 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 shutdown_connections 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6a800a2f90 0x7f6a800a5450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a9c103090 0x7f6a9c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a9c1026d0 0x7f6a9c073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 --2- 172.21.15.37:0/3100343412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c1014e0 0x7f6a9c075450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 >> 172.21.15.37:0/3100343412 conn(0x7f6a9c0fca50 msgr2=0x7f6a9c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 shutdown_connections 2024-03-21T09:09:21.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.300+0000 7f6a9be00640 1 -- 172.21.15.37:0/3100343412 wait complete. 2024-03-21T09:09:21.312 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-21T09:09:21.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] up:active 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: Cluster is now healthy 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3870562253' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3870562253' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] up:active 2024-03-21T09:09:21.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:21 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:09:21.379 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:09:21.279+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:09:21.279+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:09:21.279+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:09:21.279+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:09:21.279+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] up:active 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T09:09:21.380 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: Cluster is now healthy 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3870562253' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3870562253' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] up:active 2024-03-21T09:09:21.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:09:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-03-21T09:09:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T09:09:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] up:active 2024-03-21T09:09:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: Cluster is now healthy 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3870562253' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3870562253' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] up:active 2024-03-21T09:09:21.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:21 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.521+0000 7fa23e400640 1 -- 172.21.15.37:0/4062670447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 msgr2=0x7fa23810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.521+0000 7fa23e400640 1 --2- 172.21.15.37:0/4062670447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa23810bb40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa230009ca0 tx=0x7fa23002c100 comp rx=0 tx=0).stop 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 -- 172.21.15.37:0/4062670447 shutdown_connections 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 --2- 172.21.15.37:0/4062670447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa23810bb40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 --2- 172.21.15.37:0/4062670447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 0x7fa238104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 --2- 172.21.15.37:0/4062670447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa238103700 0x7fa238103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 -- 172.21.15.37:0/4062670447 >> 172.21.15.37:0/4062670447 conn(0x7fa2380fec70 msgr2=0x7fa2381010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 -- 172.21.15.37:0/4062670447 shutdown_connections 2024-03-21T09:09:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.522+0000 7fa23e400640 1 -- 172.21.15.37:0/4062670447 wait complete. 2024-03-21T09:09:21.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 Processor -- start 2024-03-21T09:09:21.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 -- start start 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa238103700 0x7fa2381999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 0x7fa238199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa2381a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23810f380 con 0x7fa238103700 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23810f200 con 0x7fa2381052b0 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23e400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23810f500 con 0x7fa2381048f0 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.523+0000 7fa23d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa238103700 0x7fa2381999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa23da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa2381a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 0x7fa238199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa23da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa2381a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33476/0 (socket says 172.21.15.37:33476) 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa23da00640 1 -- 172.21.15.37:0/4269174960 learned_addr learned my addr 172.21.15.37:0/4269174960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 -- 172.21.15.37:0/4269174960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 msgr2=0x7fa2381a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa2381a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 -- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa238103700 msgr2=0x7fa2381999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa238103700 0x7fa2381999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 -- 172.21.15.37:0/4269174960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2381a15c0 con 0x7fa2381048f0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa237e00640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 0x7fa238199ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e9f0 tx=0x7fa22400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400ccc0 con 0x7fa2381048f0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa224005020 con 0x7fa2381048f0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240049e0 con 0x7fa2381048f0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.524+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2381a18b0 con 0x7fa2381048f0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.525+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2381a1db0 con 0x7fa2381048f0 2024-03-21T09:09:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.526+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23810aa20 con 0x7fa2381048f0 2024-03-21T09:09:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.528+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa224005560 con 0x7fa2381048f0 2024-03-21T09:09:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.531+0000 7fa235600640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2200a2fa0 0x7fa2200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.531+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa2240cf550 con 0x7fa2381048f0 2024-03-21T09:09:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.531+0000 7fa23d000640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2200a2fa0 0x7fa2200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:21.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.531+0000 7fa23d000640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2200a2fa0 0x7fa2200a5460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa238104750 tx=0x7fa22c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.533+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2240051d0 con 0x7fa2381048f0 2024-03-21T09:09:21.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.797+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fa238061ae0 con 0x7fa2200a2fa0 2024-03-21T09:09:21.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.841+0000 7fa235600640 1 -- 172.21.15.37:0/4269174960 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa238061ae0 con 0x7fa2200a2fa0 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2200a2fa0 msgr2=0x7fa2200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2200a2fa0 0x7fa2200a5460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa238104750 tx=0x7fa22c005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 msgr2=0x7fa238199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 0x7fa238199ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e9f0 tx=0x7fa22400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 shutdown_connections 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2200a2fa0 0x7fa2200a5460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2381052b0 0x7fa2381a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2381048f0 0x7fa238199ee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 --2- 172.21.15.37:0/4269174960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa238103700 0x7fa2381999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 >> 172.21.15.37:0/4269174960 conn(0x7fa2380fec70 msgr2=0x7fa238107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 shutdown_connections 2024-03-21T09:09:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:21.844+0000 7fa23e400640 1 -- 172.21.15.37:0/4269174960 wait complete. 2024-03-21T09:09:21.858 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-21T09:09:21.922 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: 2024-03-21T09:09:21.661+0000 7f4901000700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-21T09:09:21.923 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:09:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: 2024-03-21T09:09:21.661+0000 7f4901000700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-21T09:09:22.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 -- 172.21.15.37:0/1498589499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c074100 msgr2=0x7fd58c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 --2- 172.21.15.37:0/1498589499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c074100 0x7fd58c074580 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd578009890 tx=0x7fd57802bb70 comp rx=0 tx=0).stop 2024-03-21T09:09:22.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 -- 172.21.15.37:0/1498589499 shutdown_connections 2024-03-21T09:09:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 --2- 172.21.15.37:0/1498589499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c10dac0 0x7fd58c10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 --2- 172.21.15.37:0/1498589499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c074100 0x7fd58c074580 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 --2- 172.21.15.37:0/1498589499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c0756e0 0x7fd58c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 -- 172.21.15.37:0/1498589499 >> 172.21.15.37:0/1498589499 conn(0x7fd58c0fca60 msgr2=0x7fd58c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 -- 172.21.15.37:0/1498589499 shutdown_connections 2024-03-21T09:09:22.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.080+0000 7fd592800640 1 -- 172.21.15.37:0/1498589499 wait complete. 2024-03-21T09:09:22.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.081+0000 7fd592800640 1 Processor -- start 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 -- start start 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 0x7fd58c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 0x7fd58c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c10dac0 0x7fd58c1a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c113760 con 0x7fd58c074100 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1135e0 con 0x7fd58c10dac0 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd592800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1138e0 con 0x7fd58c0756e0 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd591400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 0x7fd58c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd590a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 0x7fd58c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd591400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 0x7fd58c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44854/0 (socket says 172.21.15.37:44854) 2024-03-21T09:09:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd591400640 1 -- 172.21.15.37:0/3594748479 learned_addr learned my addr 172.21.15.37:0/3594748479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd590a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 0x7fd58c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56750/0 (socket says 172.21.15.37:56750) 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.082+0000 7fd591e00640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c10dac0 0x7fd58c1a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd590a00640 1 -- 172.21.15.37:0/3594748479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c10dac0 msgr2=0x7fd58c1a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd590a00640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c10dac0 0x7fd58c1a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd590a00640 1 -- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 msgr2=0x7fd58c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd590a00640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 0x7fd58c19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd590a00640 1 -- 172.21.15.37:0/3594748479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c1a5a00 con 0x7fd58c0756e0 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd591e00640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c10dac0 0x7fd58c1a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd591400640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 0x7fd58c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd590a00640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 0x7fd58c19e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd57802c0b0 tx=0x7fd578032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.083+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578041020 con 0x7fd58c0756e0 2024-03-21T09:09:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.084+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57802c9f0 con 0x7fd58c0756e0 2024-03-21T09:09:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.084+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c1a5c90 con 0x7fd58c0756e0 2024-03-21T09:09:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.084+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578039040 con 0x7fd58c0756e0 2024-03-21T09:09:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.084+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c1a61d0 con 0x7fd58c0756e0 2024-03-21T09:09:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.085+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c103c90 con 0x7fd58c0756e0 2024-03-21T09:09:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.088+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd578044020 con 0x7fd58c0756e0 2024-03-21T09:09:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.090+0000 7fd58a000640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a2fa0 0x7fd5740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.090+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd5780f0eb0 con 0x7fd58c0756e0 2024-03-21T09:09:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.090+0000 7fd591400640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a2fa0 0x7fd5740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.091+0000 7fd591400640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a2fa0 0x7fd5740a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd58000f7d0 tx=0x7fd580009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:22.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.094+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd57804b020 con 0x7fd58c0756e0 2024-03-21T09:09:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.115+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd5780b99b0 con 0x7fd58c0756e0 2024-03-21T09:09:22.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.335+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd58c061b70 con 0x7fd5740a2fa0 2024-03-21T09:09:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.341+0000 7fd58a000640 1 -- 172.21.15.37:0/3594748479 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fd58c061b70 con 0x7fd5740a2fa0 2024-03-21T09:09:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/ed51975d-8bac-46fd-a03d-060face92d68 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a2fa0 msgr2=0x7fd5740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a2fa0 0x7fd5740a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd58000f7d0 tx=0x7fd580009500 comp rx=0 tx=0).stop 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 msgr2=0x7fd58c19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 0x7fd58c19e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd57802c0b0 tx=0x7fd578032430 comp rx=0 tx=0).stop 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 shutdown_connections 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a2fa0 0x7fd5740a5460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd58c10dac0 0x7fd58c1a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd58c0756e0 0x7fd58c19e320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 --2- 172.21.15.37:0/3594748479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd58c074100 0x7fd58c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 >> 172.21.15.37:0/3594748479 conn(0x7fd58c0fca60 msgr2=0x7fd58c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 shutdown_connections 2024-03-21T09:09:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.344+0000 7fd592800640 1 -- 172.21.15.37:0/3594748479 wait complete. 2024-03-21T09:09:22.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:22 smithi071 ceph-mon[36597]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:22.354 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:22 smithi037 ceph-mon[31830]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:22.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-21T09:09:22.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:22 smithi053 ceph-mon[36169]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 -- 172.21.15.37:0/813485131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8101510 msgr2=0x7fafe8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 --2- 172.21.15.37:0/813485131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8101510 0x7fafe8101930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009900 tx=0x7fafd802bc30 comp rx=0 tx=0).stop 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 -- 172.21.15.37:0/813485131 shutdown_connections 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 --2- 172.21.15.37:0/813485131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe81030c0 0x7fafe8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 --2- 172.21.15.37:0/813485131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe8102700 0x7fafe8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 --2- 172.21.15.37:0/813485131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8101510 0x7fafe8101930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 -- 172.21.15.37:0/813485131 >> 172.21.15.37:0/813485131 conn(0x7fafe80fca80 msgr2=0x7fafe80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 -- 172.21.15.37:0/813485131 shutdown_connections 2024-03-21T09:09:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.603+0000 7fafed000640 1 -- 172.21.15.37:0/813485131 wait complete. 2024-03-21T09:09:22.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 Processor -- start 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 -- start start 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe8101510 0x7fafe8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 0x7fafe8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe81030c0 0x7fafe81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe810d1f0 con 0x7fafe8101510 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe810d070 con 0x7fafe8102700 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.604+0000 7fafed000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe810d370 con 0x7fafe81030c0 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 0x7fafe8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe8101510 0x7fafe8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 0x7fafe8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33524/0 (socket says 172.21.15.37:33524) 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe81030c0 0x7fafe81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 -- 172.21.15.37:0/3719650427 learned_addr learned my addr 172.21.15.37:0/3719650427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:22.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 -- 172.21.15.37:0/3719650427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe81030c0 msgr2=0x7fafe81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe81030c0 0x7fafe81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 -- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe8101510 msgr2=0x7fafe8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe8101510 0x7fafe8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 -- 172.21.15.37:0/3719650427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe81a16d0 con 0x7fafe8102700 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe7e00640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe81030c0 0x7fafe81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.605+0000 7fafe6a00640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 0x7fafe8199ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fafd400e9f0 tx=0x7fafd400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.606+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400cd80 con 0x7fafe8102700 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.606+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafd4005500 con 0x7fafe8102700 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.606+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe81a19c0 con 0x7fafe8102700 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.606+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4010690 con 0x7fafe8102700 2024-03-21T09:09:22.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.606+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe81a1f10 con 0x7fafe8102700 2024-03-21T09:09:22.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.607+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe81a21a0 con 0x7fafe8102700 2024-03-21T09:09:22.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.609+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fafd40046c0 con 0x7fafe8102700 2024-03-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.611+0000 7fafdfe00640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fafcc0a3030 0x7fafcc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.611+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fafd40cfb90 con 0x7fafe8102700 2024-03-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.611+0000 7fafe7400640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fafcc0a3030 0x7fafcc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:22.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.612+0000 7fafe7400640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fafcc0a3030 0x7fafcc0a54f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafd802c170 tx=0x7fafd802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:22.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.615+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fafd409c5c0 con 0x7fafe8102700 2024-03-21T09:09:22.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.868+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fafe8074ff0 con 0x7fafcc0a3030 2024-03-21T09:09:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.898+0000 7fafdfe00640 1 -- 172.21.15.37:0/3719650427 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fafe8074ff0 con 0x7fafcc0a3030 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fafcc0a3030 msgr2=0x7fafcc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fafcc0a3030 0x7fafcc0a54f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafd802c170 tx=0x7fafd802df90 comp rx=0 tx=0).stop 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 msgr2=0x7fafe8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 0x7fafe8199ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fafd400e9f0 tx=0x7fafd400edf0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 shutdown_connections 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fafcc0a3030 0x7fafcc0a54f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafe81030c0 0x7fafe81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe8102700 0x7fafe8199ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 --2- 172.21.15.37:0/3719650427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe8101510 0x7fafe8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 >> 172.21.15.37:0/3719650427 conn(0x7fafe80fca80 msgr2=0x7fafe80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 shutdown_connections 2024-03-21T09:09:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:22.902+0000 7fafed000640 1 -- 172.21.15.37:0/3719650427 wait complete. 2024-03-21T09:09:22.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-21T09:09:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 -- 172.21.15.37:0/3067933459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381016b0 msgr2=0x7f8538101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 --2- 172.21.15.37:0/3067933459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381016b0 0x7f8538101ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8528009900 tx=0x7f852802bc40 comp rx=0 tx=0).stop 2024-03-21T09:09:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 -- 172.21.15.37:0/3067933459 shutdown_connections 2024-03-21T09:09:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 --2- 172.21.15.37:0/3067933459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8538103260 0x7f8538109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 --2- 172.21.15.37:0/3067933459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f8538102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 --2- 172.21.15.37:0/3067933459 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381016b0 0x7f8538101ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 -- 172.21.15.37:0/3067933459 >> 172.21.15.37:0/3067933459 conn(0x7f85380fcc20 msgr2=0x7f85380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 -- 172.21.15.37:0/3067933459 shutdown_connections 2024-03-21T09:09:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.147+0000 7f853e000640 1 -- 172.21.15.37:0/3067933459 wait complete. 2024-03-21T09:09:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.148+0000 7f853e000640 1 Processor -- start 2024-03-21T09:09:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 -- start start 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85381016b0 0x7f8538199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f853819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8538103260 0x7f85381a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853810d350 con 0x7f85381016b0 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853810d1d0 con 0x7f8538103260 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f853e000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853810d4d0 con 0x7f85381028a0 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f8536a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f853819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f8537400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85381016b0 0x7f8538199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f8537e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8538103260 0x7f85381a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f8536a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f853819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56772/0 (socket says 172.21.15.37:56772) 2024-03-21T09:09:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.149+0000 7f8536a00640 1 -- 172.21.15.37:0/1585264866 learned_addr learned my addr 172.21.15.37:0/1585264866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8537e00640 1 -- 172.21.15.37:0/1585264866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 msgr2=0x7f853819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8537e00640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f853819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8537e00640 1 -- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85381016b0 msgr2=0x7f8538199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8537e00640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85381016b0 0x7f8538199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8537e00640 1 -- 172.21.15.37:0/1585264866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85381a1840 con 0x7f8538103260 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8536a00640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f853819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f8537e00640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8538103260 0x7f85381a11a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8530009fd0 tx=0x7f853000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000ce20 con 0x7f8538103260 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8530005020 con 0x7f8538103260 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530010720 con 0x7f8538103260 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.150+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85381a1b30 con 0x7f8538103260 2024-03-21T09:09:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.151+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85381a2000 con 0x7f8538103260 2024-03-21T09:09:23.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.152+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85381a2230 con 0x7f8538103260 2024-03-21T09:09:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.153+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8530005540 con 0x7f8538103260 2024-03-21T09:09:23.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.156+0000 7f852fe00640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f851c0a2fb0 0x7f851c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.156+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f85300cffb0 con 0x7f8538103260 2024-03-21T09:09:23.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.156+0000 7f8537400640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f851c0a2fb0 0x7f851c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.157+0000 7f8537400640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f851c0a2fb0 0x7f851c0a5470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f852802c180 tx=0x7f852802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.162+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f853009c9e0 con 0x7f8538103260 2024-03-21T09:09:23.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:23 smithi037 ceph-mon[31830]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-03-21T09:09:23.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:23 smithi037 ceph-mon[31830]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:23.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:23 smithi037 ceph-mon[31830]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-21T09:09:23.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:23 smithi037 ceph-mon[31830]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:23.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:23 smithi071 ceph-mon[36597]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-03-21T09:09:23.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:23 smithi071 ceph-mon[36597]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:23.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:23 smithi071 ceph-mon[36597]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-21T09:09:23.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:23 smithi071 ceph-mon[36597]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:23.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.446+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8538074ff0 con 0x7f851c0a2fb0 2024-03-21T09:09:23.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:23 smithi053 ceph-mon[36169]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-03-21T09:09:23.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:23 smithi053 ceph-mon[36169]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:23.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:23 smithi053 ceph-mon[36169]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-21T09:09:23.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:23 smithi053 ceph-mon[36169]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.451+0000 7f852fe00640 1 -- 172.21.15.37:0/1585264866 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f8538074ff0 con 0x7f851c0a2fb0 2024-03-21T09:09:23.446 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/cb278297-306c-49a2-90ad-118e60fdafc6 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f851c0a2fb0 msgr2=0x7f851c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f851c0a2fb0 0x7f851c0a5470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f852802c180 tx=0x7f852802cf90 comp rx=0 tx=0).stop 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8538103260 msgr2=0x7f85381a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8538103260 0x7f85381a11a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8530009fd0 tx=0x7f853000ef10 comp rx=0 tx=0).stop 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 shutdown_connections 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f851c0a2fb0 0x7f851c0a5470 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8538103260 0x7f85381a11a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85381028a0 0x7f853819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 --2- 172.21.15.37:0/1585264866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85381016b0 0x7f8538199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 >> 172.21.15.37:0/1585264866 conn(0x7f85380fcc20 msgr2=0x7f85380738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:23.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 shutdown_connections 2024-03-21T09:09:23.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.455+0000 7f853e000640 1 -- 172.21.15.37:0/1585264866 wait complete. 2024-03-21T09:09:23.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:09:23.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 -- 172.21.15.37:0/3060447642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 msgr2=0x7fd97c101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 --2- 172.21.15.37:0/3060447642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c101b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009900 tx=0x7fd96c02bc50 comp rx=0 tx=0).stop 2024-03-21T09:09:23.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 -- 172.21.15.37:0/3060447642 shutdown_connections 2024-03-21T09:09:23.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 --2- 172.21.15.37:0/3060447642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd97c1032e0 0x7fd97c109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 --2- 172.21.15.37:0/3060447642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c102920 0x7fd97c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 --2- 172.21.15.37:0/3060447642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c101b50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 -- 172.21.15.37:0/3060447642 >> 172.21.15.37:0/3060447642 conn(0x7fd97c0fccc0 msgr2=0x7fd97c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:23.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 -- 172.21.15.37:0/3060447642 shutdown_connections 2024-03-21T09:09:23.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.695+0000 7fd982e00640 1 -- 172.21.15.37:0/3060447642 wait complete. 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.696+0000 7fd982e00640 1 Processor -- start 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.696+0000 7fd982e00640 1 -- start start 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd982e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd982e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd97c102920 0x7fd97c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd982e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 0x7fd97c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd982e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10d360 con 0x7fd97c1032e0 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd982e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10d1e0 con 0x7fd97c101730 2024-03-21T09:09:23.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd982e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10d4e0 con 0x7fd97c102920 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd980e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 0x7fd97c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd97be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd980e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 0x7fd97c1a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44900/0 (socket says 172.21.15.37:44900) 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd980e00640 1 -- 172.21.15.37:0/2326216695 learned_addr learned my addr 172.21.15.37:0/2326216695 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd973e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd97c102920 0x7fd97c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd97be00640 1 -- 172.21.15.37:0/2326216695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd97c102920 msgr2=0x7fd97c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd97be00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd97c102920 0x7fd97c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd97be00640 1 -- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 msgr2=0x7fd97c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd97be00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 0x7fd97c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd97be00640 1 -- 172.21.15.37:0/2326216695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c1a1810 con 0x7fd97c101730 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.697+0000 7fd980e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 0x7fd97c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.698+0000 7fd97be00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c199bf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009230 tx=0x7fd96c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:23.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.698+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c041070 con 0x7fd97c101730 2024-03-21T09:09:23.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.698+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd96c0054e0 con 0x7fd97c101730 2024-03-21T09:09:23.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.698+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c003e60 con 0x7fd97c101730 2024-03-21T09:09:23.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.698+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c1a1aa0 con 0x7fd97c101730 2024-03-21T09:09:23.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.698+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c1a2010 con 0x7fd97c101730 2024-03-21T09:09:23.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.699+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c1a21e0 con 0x7fd97c101730 2024-03-21T09:09:23.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.701+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd96c044070 con 0x7fd97c101730 2024-03-21T09:09:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.702+0000 7fd979600640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd9600a3030 0x7fd9600a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.703+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd96c0f0ff0 con 0x7fd97c101730 2024-03-21T09:09:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.703+0000 7fd973e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd9600a3030 0x7fd9600a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:23.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.703+0000 7fd973e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd9600a3030 0x7fd9600a54f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd97c19b100 tx=0x7fd964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:23.706+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd96c0bd520 con 0x7fd97c101730 2024-03-21T09:09:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.006+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd97c1a23b0 con 0x7fd97c101730 2024-03-21T09:09:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.007+0000 7fd979600640 1 -- 172.21.15.37:0/2326216695 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fd96c0c15c0 con 0x7fd97c101730 2024-03-21T09:09:24.002 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:24.003 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:09:20.795215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:09:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.011+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd9600a3030 msgr2=0x7fd9600a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.011+0000 7fd982e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd9600a3030 0x7fd9600a54f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd97c19b100 tx=0x7fd964005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.011+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 msgr2=0x7fd97c199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.011+0000 7fd982e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c199bf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009230 tx=0x7fd96c02c670 comp rx=0 tx=0).stop 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 shutdown_connections 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd9600a3030 0x7fd9600a54f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd97c1032e0 0x7fd97c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd97c102920 0x7fd97c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 --2- 172.21.15.37:0/2326216695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd97c101730 0x7fd97c199bf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 >> 172.21.15.37:0/2326216695 conn(0x7fd97c0fccc0 msgr2=0x7fd97c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:24.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 shutdown_connections 2024-03-21T09:09:24.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.012+0000 7fd982e00640 1 -- 172.21.15.37:0/2326216695 wait complete. 2024-03-21T09:09:24.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.265+0000 7ffb50c00640 1 -- 172.21.15.37:0/3372182182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1049b0 msgr2=0x7ffb4c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 --2- 172.21.15.37:0/3372182182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1049b0 0x7ffb4c104e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009900 tx=0x7ffb3802bc30 comp rx=0 tx=0).stop 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 -- 172.21.15.37:0/3372182182 shutdown_connections 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 --2- 172.21.15.37:0/3372182182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb4c105370 0x7ffb4c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 --2- 172.21.15.37:0/3372182182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1049b0 0x7ffb4c104e30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 --2- 172.21.15.37:0/3372182182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 -- 172.21.15.37:0/3372182182 >> 172.21.15.37:0/3372182182 conn(0x7ffb4c0fed50 msgr2=0x7ffb4c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 -- 172.21.15.37:0/3372182182 shutdown_connections 2024-03-21T09:09:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.266+0000 7ffb50c00640 1 -- 172.21.15.37:0/3372182182 wait complete. 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 Processor -- start 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 -- start start 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb4c1049b0 0x7ffb4c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c105370 0x7ffb4c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c10f470 con 0x7ffb4c105370 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c10f2f0 con 0x7ffb4c1037c0 2024-03-21T09:09:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.267+0000 7ffb50c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c10f5f0 con 0x7ffb4c1049b0 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:34698/0 (socket says 172.21.15.37:34698) 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 -- 172.21.15.37:0/2283196698 learned_addr learned my addr 172.21.15.37:0/2283196698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4aa00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb4c1049b0 0x7ffb4c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4be00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c105370 0x7ffb4c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 -- 172.21.15.37:0/2283196698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb4c1049b0 msgr2=0x7ffb4c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb4c1049b0 0x7ffb4c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 -- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c105370 msgr2=0x7ffb4c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c105370 0x7ffb4c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 -- 172.21.15.37:0/2283196698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c1a1740 con 0x7ffb4c1037c0 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4be00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c105370 0x7ffb4c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.268+0000 7ffb4b400640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c199b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9c0 tx=0x7ffb3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.269+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00cd80 con 0x7ffb4c1037c0 2024-03-21T09:09:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.269+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb3c0035f0 con 0x7ffb4c1037c0 2024-03-21T09:09:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.269+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c1a1a30 con 0x7ffb4c1037c0 2024-03-21T09:09:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.269+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c010690 con 0x7ffb4c1037c0 2024-03-21T09:09:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.269+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c1a1f50 con 0x7ffb4c1037c0 2024-03-21T09:09:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.270+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c04f470 con 0x7ffb4c1037c0 2024-03-21T09:09:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.273+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7ffb3c0037a0 con 0x7ffb4c1037c0 2024-03-21T09:09:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.275+0000 7ffb43e00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffb300a2fa0 0x7ffb300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.275+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ffb3c0cfcb0 con 0x7ffb4c1037c0 2024-03-21T09:09:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.275+0000 7ffb4aa00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffb300a2fa0 0x7ffb300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.276+0000 7ffb4aa00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffb300a2fa0 0x7ffb300a5460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb38000c00 tx=0x7ffb3802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.279+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffb3c09c6e0 con 0x7ffb4c1037c0 2024-03-21T09:09:24.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:24 smithi037 ceph-mon[31830]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:24.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:24 smithi037 ceph-mon[31830]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:24.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:24 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2326216695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:09:24.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:24 smithi053 ceph-mon[36169]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:24.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:24 smithi053 ceph-mon[36169]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:24.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:24 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2326216695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:09:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.484+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb4c02d200 con 0x7ffb4c1037c0 2024-03-21T09:09:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.486+0000 7ffb43e00640 1 -- 172.21.15.37:0/2283196698 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7ffb3c09bfe0 con 0x7ffb4c1037c0 2024-03-21T09:09:24.481 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:24.481 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffb300a2fa0 msgr2=0x7ffb300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffb300a2fa0 0x7ffb300a5460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb38000c00 tx=0x7ffb3802df90 comp rx=0 tx=0).stop 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 msgr2=0x7ffb4c199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c199b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9c0 tx=0x7ffb3c00ee80 comp rx=0 tx=0).stop 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 shutdown_connections 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ffb300a2fa0 0x7ffb300a5460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb4c105370 0x7ffb4c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb4c1049b0 0x7ffb4c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 --2- 172.21.15.37:0/2283196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c1037c0 0x7ffb4c199b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 >> 172.21.15.37:0/2283196698 conn(0x7ffb4c0fed50 msgr2=0x7ffb4c107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 shutdown_connections 2024-03-21T09:09:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.489+0000 7ffb50c00640 1 -- 172.21.15.37:0/2283196698 wait complete. 2024-03-21T09:09:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-21T09:09:24.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T09:09:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:24 smithi071 ceph-mon[36597]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:24 smithi071 ceph-mon[36597]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:09:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:24 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2326216695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.743+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1136544757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce41037c0 msgr2=0x7f7ce4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.743+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1136544757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce41037c0 0x7f7ce4103be0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009900 tx=0x7f7cd402bc50 comp rx=0 tx=0).stop 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1136544757 shutdown_connections 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1136544757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4105370 0x7f7ce410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1136544757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ce41049b0 0x7f7ce4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1136544757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce41037c0 0x7f7ce4103be0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1136544757 >> 172.21.15.37:0/1136544757 conn(0x7f7ce40fed50 msgr2=0x7f7ce4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1136544757 shutdown_connections 2024-03-21T09:09:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.744+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1136544757 wait complete. 2024-03-21T09:09:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 Processor -- start 2024-03-21T09:09:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 -- start start 2024-03-21T09:09:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 0x7f7ce4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ce41049b0 0x7f7ce4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce4105370 0x7f7ce41a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce410f460 con 0x7f7ce41049b0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce410f2e0 con 0x7f7ce41037c0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce410f5e0 con 0x7f7ce4105370 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 0x7f7ce4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 0x7f7ce4199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:34720/0 (socket says 172.21.15.37:34720) 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce2a00640 1 -- 172.21.15.37:0/1698337669 learned_addr learned my addr 172.21.15.37:0/1698337669 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce2000640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ce41049b0 0x7f7ce4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.745+0000 7f7ce3400640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce4105370 0x7f7ce41a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce2a00640 1 -- 172.21.15.37:0/1698337669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce4105370 msgr2=0x7f7ce41a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce2a00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce4105370 0x7f7ce41a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce2a00640 1 -- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ce41049b0 msgr2=0x7f7ce4199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce2a00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ce41049b0 0x7f7ce4199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce2a00640 1 -- 172.21.15.37:0/1698337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce41a16a0 con 0x7f7ce41037c0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce2a00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 0x7f7ce4199a80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cd402c260 tx=0x7f7cd402c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:24.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4041070 con 0x7f7ce41037c0 2024-03-21T09:09:24.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd4004450 con 0x7f7ce41037c0 2024-03-21T09:09:24.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce41a1930 con 0x7f7ce41037c0 2024-03-21T09:09:24.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.746+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4034440 con 0x7f7ce41037c0 2024-03-21T09:09:24.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.747+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce41a1ed0 con 0x7f7ce41037c0 2024-03-21T09:09:24.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.748+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce410aae0 con 0x7f7ce41037c0 2024-03-21T09:09:24.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.749+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7cd4044070 con 0x7f7ce41037c0 2024-03-21T09:09:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.752+0000 7f7cdb400640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7cc80a3020 0x7f7cc80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.752+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7cd40f1100 con 0x7f7ce41037c0 2024-03-21T09:09:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.752+0000 7f7ce2000640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7cc80a3020 0x7f7cc80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.753+0000 7f7ce2000640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7cc80a3020 0x7f7cc80a54e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ce419af90 tx=0x7f7cd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:24.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.755+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cd40bd5f0 con 0x7f7ce41037c0 2024-03-21T09:09:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.957+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ce4061ae0 con 0x7f7cc80a3020 2024-03-21T09:09:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.971+0000 7f7cdb400640 1 -- 172.21.15.37:0/1698337669 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340620 (secure 0 0 0) 0x7f7ce4061ae0 con 0x7f7cc80a3020 2024-03-21T09:09:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:24.974 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7cc80a3020 msgr2=0x7f7cc80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7cc80a3020 0x7f7cc80a54e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ce419af90 tx=0x7f7cd0005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 msgr2=0x7f7ce4199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 0x7f7ce4199a80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cd402c260 tx=0x7f7cd402c670 comp rx=0 tx=0).stop 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 shutdown_connections 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7cc80a3020 0x7f7cc80a54e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ce4105370 0x7f7ce41a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ce41049b0 0x7f7ce4199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 --2- 172.21.15.37:0/1698337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce41037c0 0x7f7ce4199a80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.977+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 >> 172.21.15.37:0/1698337669 conn(0x7f7ce40fed50 msgr2=0x7f7ce4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.978+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 shutdown_connections 2024-03-21T09:09:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:24.978+0000 7f7ce3e00640 1 -- 172.21.15.37:0/1698337669 wait complete. 2024-03-21T09:09:24.996 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-03-21T09:09:23.781805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615803,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":155,"num_read_kb":146,"num_write":290,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373396,"kb_used_data":11744,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568428,"statfs":{"total":1151940427776,"available":1151558070272,"internally_reserved":0,"allocated":12025856,"data_stored":7617355,"data_compressed":284787,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20936,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":19,"num_read_kb":27,"num_write":61,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006212"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246355+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:08:52.246355+0000","last_peered":"2024-03-21T09:08:52.246355+0000","last_clean":"2024-03-21T09:08:52.246355+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:08:52.246355+0000","last_undegraded":"2024-03-21T09:08:52.246355+0000","last_fullsized":"2024-03-21T09:08:52.246355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247039+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:08:52.247039+0000","last_peered":"2024-03-21T09:08:52.247039+0000","last_clean":"2024-03-21T09:08:52.247039+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:08:52.247039+0000","last_undegraded":"2024-03-21T09:08:52.247039+0000","last_fullsized":"2024-03-21T09:08:52.247039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246844+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:08:52.246844+0000","last_peered":"2024-03-21T09:08:52.246844+0000","last_clean":"2024-03-21T09:08:52.246844+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:08:52.246844+0000","last_undegraded":"2024-03-21T09:08:52.246844+0000","last_fullsized":"2024-03-21T09:08:52.246844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121949+0000","last_change":"2024-03-21T09:08:46.824151+0000","last_active":"2024-03-21T09:08:52.121949+0000","last_peered":"2024-03-21T09:08:52.121949+0000","last_clean":"2024-03-21T09:08:52.121949+0000","last_became_active":"2024-03-21T09:08:46.822770+0000","last_became_peered":"2024-03-21T09:08:46.822770+0000","last_unstale":"2024-03-21T09:08:52.121949+0000","last_undegraded":"2024-03-21T09:08:52.121949+0000","last_fullsized":"2024-03-21T09:08:52.121949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:34:26.969292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245999+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:08:52.245999+0000","last_peered":"2024-03-21T09:08:52.245999+0000","last_clean":"2024-03-21T09:08:52.245999+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:08:52.245999+0000","last_undegraded":"2024-03-21T09:08:52.245999+0000","last_fullsized":"2024-03-21T09:08:52.245999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244325+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:08:52.244325+0000","last_peered":"2024-03-21T09:08:52.244325+0000","last_clean":"2024-03-21T09:08:52.244325+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:08:52.244325+0000","last_undegraded":"2024-03-21T09:08:52.244325+0000","last_fullsized":"2024-03-21T09:08:52.244325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244861+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:08:52.244861+0000","last_peered":"2024-03-21T09:08:52.244861+0000","last_clean":"2024-03-21T09:08:52.244861+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:08:52.244861+0000","last_undegraded":"2024-03-21T09:08:52.244861+0000","last_fullsized":"2024-03-21T09:08:52.244861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961241+0000","last_change":"2024-03-21T09:08:46.851011+0000","last_active":"2024-03-21T09:08:51.961241+0000","last_peered":"2024-03-21T09:08:51.961241+0000","last_clean":"2024-03-21T09:08:51.961241+0000","last_became_active":"2024-03-21T09:08:46.850599+0000","last_became_peered":"2024-03-21T09:08:46.850599+0000","last_unstale":"2024-03-21T09:08:51.961241+0000","last_undegraded":"2024-03-21T09:08:51.961241+0000","last_fullsized":"2024-03-21T09:08:51.961241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:50:30.925086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960373+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:08:51.960373+0000","last_peered":"2024-03-21T09:08:51.960373+0000","last_clean":"2024-03-21T09:08:51.960373+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:08:51.960373+0000","last_undegraded":"2024-03-21T09:08:51.960373+0000","last_fullsized":"2024-03-21T09:08:51.960373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245102+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:08:52.245102+0000","last_peered":"2024-03-21T09:08:52.245102+0000","last_clean":"2024-03-21T09:08:52.245102+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:08:52.245102+0000","last_undegraded":"2024-03-21T09:08:52.245102+0000","last_fullsized":"2024-03-21T09:08:52.245102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245486+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:08:52.245486+0000","last_peered":"2024-03-21T09:08:52.245486+0000","last_clean":"2024-03-21T09:08:52.245486+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:08:52.245486+0000","last_undegraded":"2024-03-21T09:08:52.245486+0000","last_fullsized":"2024-03-21T09:08:52.245486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121788+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:08:52.121788+0000","last_peered":"2024-03-21T09:08:52.121788+0000","last_clean":"2024-03-21T09:08:52.121788+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:08:52.121788+0000","last_undegraded":"2024-03-21T09:08:52.121788+0000","last_fullsized":"2024-03-21T09:08:52.121788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245277+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:08:52.245277+0000","last_peered":"2024-03-21T09:08:52.245277+0000","last_clean":"2024-03-21T09:08:52.245277+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:08:52.245277+0000","last_undegraded":"2024-03-21T09:08:52.245277+0000","last_fullsized":"2024-03-21T09:08:52.245277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246438+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:08:52.246438+0000","last_peered":"2024-03-21T09:08:52.246438+0000","last_clean":"2024-03-21T09:08:52.246438+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:08:52.246438+0000","last_undegraded":"2024-03-21T09:08:52.246438+0000","last_fullsized":"2024-03-21T09:08:52.246438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246374+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:08:52.246374+0000","last_peered":"2024-03-21T09:08:52.246374+0000","last_clean":"2024-03-21T09:08:52.246374+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:08:52.246374+0000","last_undegraded":"2024-03-21T09:08:52.246374+0000","last_fullsized":"2024-03-21T09:08:52.246374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.959525+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:08:51.959525+0000","last_peered":"2024-03-21T09:08:51.959525+0000","last_clean":"2024-03-21T09:08:51.959525+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:08:51.959525+0000","last_undegraded":"2024-03-21T09:08:51.959525+0000","last_fullsized":"2024-03-21T09:08:51.959525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246166+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:08:52.246166+0000","last_peered":"2024-03-21T09:08:52.246166+0000","last_clean":"2024-03-21T09:08:52.246166+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:08:52.246166+0000","last_undegraded":"2024-03-21T09:08:52.246166+0000","last_fullsized":"2024-03-21T09:08:52.246166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.964924+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:08:51.964924+0000","last_peered":"2024-03-21T09:08:51.964924+0000","last_clean":"2024-03-21T09:08:51.964924+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:08:51.964924+0000","last_undegraded":"2024-03-21T09:08:51.964924+0000","last_fullsized":"2024-03-21T09:08:51.964924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245967+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:08:52.245967+0000","last_peered":"2024-03-21T09:08:52.245967+0000","last_clean":"2024-03-21T09:08:52.245967+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:08:52.245967+0000","last_undegraded":"2024-03-21T09:08:52.245967+0000","last_fullsized":"2024-03-21T09:08:52.245967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245855+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:08:52.245855+0000","last_peered":"2024-03-21T09:08:52.245855+0000","last_clean":"2024-03-21T09:08:52.245855+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:08:52.245855+0000","last_undegraded":"2024-03-21T09:08:52.245855+0000","last_fullsized":"2024-03-21T09:08:52.245855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965910+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:08:51.965910+0000","last_peered":"2024-03-21T09:08:51.965910+0000","last_clean":"2024-03-21T09:08:51.965910+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:08:51.965910+0000","last_undegraded":"2024-03-21T09:08:51.965910+0000","last_fullsized":"2024-03-21T09:08:51.965910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.962946+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:08:51.962946+0000","last_peered":"2024-03-21T09:08:51.962946+0000","last_clean":"2024-03-21T09:08:51.962946+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:08:51.962946+0000","last_undegraded":"2024-03-21T09:08:51.962946+0000","last_fullsized":"2024-03-21T09:08:51.962946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121684+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:08:52.121684+0000","last_peered":"2024-03-21T09:08:52.121684+0000","last_clean":"2024-03-21T09:08:52.121684+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:08:52.121684+0000","last_undegraded":"2024-03-21T09:08:52.121684+0000","last_fullsized":"2024-03-21T09:08:52.121684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963121+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:08:51.963121+0000","last_peered":"2024-03-21T09:08:51.963121+0000","last_clean":"2024-03-21T09:08:51.963121+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:08:51.963121+0000","last_undegraded":"2024-03-21T09:08:51.963121+0000","last_fullsized":"2024-03-21T09:08:51.963121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247523+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:08:52.247523+0000","last_peered":"2024-03-21T09:08:52.247523+0000","last_clean":"2024-03-21T09:08:52.247523+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:08:52.247523+0000","last_undegraded":"2024-03-21T09:08:52.247523+0000","last_fullsized":"2024-03-21T09:08:52.247523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121479+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:08:52.121479+0000","last_peered":"2024-03-21T09:08:52.121479+0000","last_clean":"2024-03-21T09:08:52.121479+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:08:52.121479+0000","last_undegraded":"2024-03-21T09:08:52.121479+0000","last_fullsized":"2024-03-21T09:08:52.121479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247348+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:08:52.247348+0000","last_peered":"2024-03-21T09:08:52.247348+0000","last_clean":"2024-03-21T09:08:52.247348+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:08:52.247348+0000","last_undegraded":"2024-03-21T09:08:52.247348+0000","last_fullsized":"2024-03-21T09:08:52.247348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.959811+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:08:51.959811+0000","last_peered":"2024-03-21T09:08:51.959811+0000","last_clean":"2024-03-21T09:08:51.959811+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:08:51.959811+0000","last_undegraded":"2024-03-21T09:08:51.959811+0000","last_fullsized":"2024-03-21T09:08:51.959811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121407+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:08:52.121407+0000","last_peered":"2024-03-21T09:08:52.121407+0000","last_clean":"2024-03-21T09:08:52.121407+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:08:52.121407+0000","last_undegraded":"2024-03-21T09:08:52.121407+0000","last_fullsized":"2024-03-21T09:08:52.121407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246430+0000","last_change":"2024-03-21T09:08:45.818384+0000","last_active":"2024-03-21T09:08:52.246430+0000","last_peered":"2024-03-21T09:08:52.246430+0000","last_clean":"2024-03-21T09:08:52.246430+0000","last_became_active":"2024-03-21T09:08:45.818028+0000","last_became_peered":"2024-03-21T09:08:45.818028+0000","last_unstale":"2024-03-21T09:08:52.246430+0000","last_undegraded":"2024-03-21T09:08:52.246430+0000","last_fullsized":"2024-03-21T09:08:52.246430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:04.433350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247132+0000","last_change":"2024-03-21T09:08:46.818779+0000","last_active":"2024-03-21T09:08:52.247132+0000","last_peered":"2024-03-21T09:08:52.247132+0000","last_clean":"2024-03-21T09:08:52.247132+0000","last_became_active":"2024-03-21T09:08:46.818412+0000","last_became_peered":"2024-03-21T09:08:46.818412+0000","last_unstale":"2024-03-21T09:08:52.247132+0000","last_undegraded":"2024-03-21T09:08:52.247132+0000","last_fullsized":"2024-03-21T09:08:52.247132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:47:37.234884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245826+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:08:52.245826+0000","last_peered":"2024-03-21T09:08:52.245826+0000","last_clean":"2024-03-21T09:08:52.245826+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:08:52.245826+0000","last_undegraded":"2024-03-21T09:08:52.245826+0000","last_fullsized":"2024-03-21T09:08:52.245826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:21.809871+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:09:21.809871+0000","last_peered":"2024-03-21T09:09:21.809871+0000","last_clean":"2024-03-21T09:09:21.809871+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:09:21.809871+0000","last_undegraded":"2024-03-21T09:09:21.809871+0000","last_fullsized":"2024-03-21T09:09:21.809871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245107+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:08:52.245107+0000","last_peered":"2024-03-21T09:08:52.245107+0000","last_clean":"2024-03-21T09:08:52.245107+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:08:52.245107+0000","last_undegraded":"2024-03-21T09:08:52.245107+0000","last_fullsized":"2024-03-21T09:08:52.245107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245459+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:08:52.245459+0000","last_peered":"2024-03-21T09:08:52.245459+0000","last_clean":"2024-03-21T09:08:52.245459+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:08:52.245459+0000","last_undegraded":"2024-03-21T09:08:52.245459+0000","last_fullsized":"2024-03-21T09:08:52.245459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.123215+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:09:20.123215+0000","last_peered":"2024-03-21T09:09:20.123215+0000","last_clean":"2024-03-21T09:09:20.123215+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:09:20.123215+0000","last_undegraded":"2024-03-21T09:09:20.123215+0000","last_fullsized":"2024-03-21T09:09:20.123215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246873+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:08:52.246873+0000","last_peered":"2024-03-21T09:08:52.246873+0000","last_clean":"2024-03-21T09:08:52.246873+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:08:52.246873+0000","last_undegraded":"2024-03-21T09:08:52.246873+0000","last_fullsized":"2024-03-21T09:08:52.246873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246123+0000","last_change":"2024-03-21T09:08:50.962949+0000","last_active":"2024-03-21T09:08:52.246123+0000","last_peered":"2024-03-21T09:08:52.246123+0000","last_clean":"2024-03-21T09:08:52.246123+0000","last_became_active":"2024-03-21T09:08:50.962527+0000","last_became_peered":"2024-03-21T09:08:50.962527+0000","last_unstale":"2024-03-21T09:08:52.246123+0000","last_undegraded":"2024-03-21T09:08:52.246123+0000","last_fullsized":"2024-03-21T09:08:52.246123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:28.557444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.123319+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:09:20.123319+0000","last_peered":"2024-03-21T09:09:20.123319+0000","last_clean":"2024-03-21T09:09:20.123319+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:09:20.123319+0000","last_undegraded":"2024-03-21T09:09:20.123319+0000","last_fullsized":"2024-03-21T09:09:20.123319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961158+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:08:51.961158+0000","last_peered":"2024-03-21T09:08:51.961158+0000","last_clean":"2024-03-21T09:08:51.961158+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:08:51.961158+0000","last_undegraded":"2024-03-21T09:08:51.961158+0000","last_fullsized":"2024-03-21T09:08:51.961158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247083+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:08:52.247083+0000","last_peered":"2024-03-21T09:08:52.247083+0000","last_clean":"2024-03-21T09:08:52.247083+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:08:52.247083+0000","last_undegraded":"2024-03-21T09:08:52.247083+0000","last_fullsized":"2024-03-21T09:08:52.247083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245952+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:08:52.245952+0000","last_peered":"2024-03-21T09:08:52.245952+0000","last_clean":"2024-03-21T09:08:52.245952+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:08:52.245952+0000","last_undegraded":"2024-03-21T09:08:52.245952+0000","last_fullsized":"2024-03-21T09:08:52.245952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963598+0000","last_change":"2024-03-21T09:08:46.834298+0000","last_active":"2024-03-21T09:08:51.963598+0000","last_peered":"2024-03-21T09:08:51.963598+0000","last_clean":"2024-03-21T09:08:51.963598+0000","last_became_active":"2024-03-21T09:08:46.833681+0000","last_became_peered":"2024-03-21T09:08:46.833681+0000","last_unstale":"2024-03-21T09:08:51.963598+0000","last_undegraded":"2024-03-21T09:08:51.963598+0000","last_fullsized":"2024-03-21T09:08:51.963598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:41.342590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961571+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:08:51.961571+0000","last_peered":"2024-03-21T09:08:51.961571+0000","last_clean":"2024-03-21T09:08:51.961571+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:08:51.961571+0000","last_undegraded":"2024-03-21T09:08:51.961571+0000","last_fullsized":"2024-03-21T09:08:51.961571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.807067+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:09:20.807067+0000","last_peered":"2024-03-21T09:09:20.807067+0000","last_clean":"2024-03-21T09:09:20.807067+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:09:20.807067+0000","last_undegraded":"2024-03-21T09:09:20.807067+0000","last_fullsized":"2024-03-21T09:09:20.807067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960381+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:08:51.960381+0000","last_peered":"2024-03-21T09:08:51.960381+0000","last_clean":"2024-03-21T09:08:51.960381+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:08:51.960381+0000","last_undegraded":"2024-03-21T09:08:51.960381+0000","last_fullsized":"2024-03-21T09:08:51.960381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963372+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:08:51.963372+0000","last_peered":"2024-03-21T09:08:51.963372+0000","last_clean":"2024-03-21T09:08:51.963372+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:08:51.963372+0000","last_undegraded":"2024-03-21T09:08:51.963372+0000","last_fullsized":"2024-03-21T09:08:51.963372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103063+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:09:19.103063+0000","last_peered":"2024-03-21T09:09:19.103063+0000","last_clean":"2024-03-21T09:09:19.103063+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:09:19.103063+0000","last_undegraded":"2024-03-21T09:09:19.103063+0000","last_fullsized":"2024-03-21T09:09:19.103063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247384+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:08:52.247384+0000","last_peered":"2024-03-21T09:08:52.247384+0000","last_clean":"2024-03-21T09:08:52.247384+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:08:52.247384+0000","last_undegraded":"2024-03-21T09:08:52.247384+0000","last_fullsized":"2024-03-21T09:08:52.247384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121866+0000","last_change":"2024-03-21T09:08:50.974540+0000","last_active":"2024-03-21T09:08:52.121866+0000","last_peered":"2024-03-21T09:08:52.121866+0000","last_clean":"2024-03-21T09:08:52.121866+0000","last_became_active":"2024-03-21T09:08:50.973639+0000","last_became_peered":"2024-03-21T09:08:50.973639+0000","last_unstale":"2024-03-21T09:08:52.121866+0000","last_undegraded":"2024-03-21T09:08:52.121866+0000","last_fullsized":"2024-03-21T09:08:52.121866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:47.377024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.104653+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:09:19.104653+0000","last_peered":"2024-03-21T09:09:19.104653+0000","last_clean":"2024-03-21T09:09:19.104653+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:09:19.104653+0000","last_undegraded":"2024-03-21T09:09:19.104653+0000","last_fullsized":"2024-03-21T09:09:19.104653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960102+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:08:51.960102+0000","last_peered":"2024-03-21T09:08:51.960102+0000","last_clean":"2024-03-21T09:08:51.960102+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:08:51.960102+0000","last_undegraded":"2024-03-21T09:08:51.960102+0000","last_fullsized":"2024-03-21T09:08:51.960102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.964027+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:08:51.964027+0000","last_peered":"2024-03-21T09:08:51.964027+0000","last_clean":"2024-03-21T09:08:51.964027+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:08:51.964027+0000","last_undegraded":"2024-03-21T09:08:51.964027+0000","last_fullsized":"2024-03-21T09:08:51.964027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'20","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:21.821673+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:09:21.821673+0000","last_peered":"2024-03-21T09:09:21.821673+0000","last_clean":"2024-03-21T09:09:21.821673+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:09:21.821673+0000","last_undegraded":"2024-03-21T09:09:21.821673+0000","last_fullsized":"2024-03-21T09:09:21.821673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":18,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247161+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:08:52.247161+0000","last_peered":"2024-03-21T09:08:52.247161+0000","last_clean":"2024-03-21T09:08:52.247161+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:08:52.247161+0000","last_undegraded":"2024-03-21T09:08:52.247161+0000","last_fullsized":"2024-03-21T09:08:52.247161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247755+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:08:52.247755+0000","last_peered":"2024-03-21T09:08:52.247755+0000","last_clean":"2024-03-21T09:08:52.247755+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:08:52.247755+0000","last_undegraded":"2024-03-21T09:08:52.247755+0000","last_fullsized":"2024-03-21T09:08:52.247755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103901+0000","last_change":"2024-03-21T09:08:45.826707+0000","last_active":"2024-03-21T09:09:19.103901+0000","last_peered":"2024-03-21T09:09:19.103901+0000","last_clean":"2024-03-21T09:09:19.103901+0000","last_became_active":"2024-03-21T09:08:45.826364+0000","last_became_peered":"2024-03-21T09:08:45.826364+0000","last_unstale":"2024-03-21T09:09:19.103901+0000","last_undegraded":"2024-03-21T09:09:19.103901+0000","last_fullsized":"2024-03-21T09:09:19.103901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:38:48.129131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245058+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:08:52.245058+0000","last_peered":"2024-03-21T09:08:52.245058+0000","last_clean":"2024-03-21T09:08:52.245058+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:08:52.245058+0000","last_undegraded":"2024-03-21T09:08:52.245058+0000","last_fullsized":"2024-03-21T09:08:52.245058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245056+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:08:52.245056+0000","last_peered":"2024-03-21T09:08:52.245056+0000","last_clean":"2024-03-21T09:08:52.245056+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:08:52.245056+0000","last_undegraded":"2024-03-21T09:08:52.245056+0000","last_fullsized":"2024-03-21T09:08:52.245056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103955+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:09:19.103955+0000","last_peered":"2024-03-21T09:09:19.103955+0000","last_clean":"2024-03-21T09:09:19.103955+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:09:19.103955+0000","last_undegraded":"2024-03-21T09:09:19.103955+0000","last_fullsized":"2024-03-21T09:09:19.103955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244930+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:08:52.244930+0000","last_peered":"2024-03-21T09:08:52.244930+0000","last_clean":"2024-03-21T09:08:52.244930+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:08:52.244930+0000","last_undegraded":"2024-03-21T09:08:52.244930+0000","last_fullsized":"2024-03-21T09:08:52.244930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965580+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:08:51.965580+0000","last_peered":"2024-03-21T09:08:51.965580+0000","last_clean":"2024-03-21T09:08:51.965580+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:08:51.965580+0000","last_undegraded":"2024-03-21T09:08:51.965580+0000","last_fullsized":"2024-03-21T09:08:51.965580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121392+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:08:52.121392+0000","last_peered":"2024-03-21T09:08:52.121392+0000","last_clean":"2024-03-21T09:08:52.121392+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:08:52.121392+0000","last_undegraded":"2024-03-21T09:08:52.121392+0000","last_fullsized":"2024-03-21T09:08:52.121392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244633+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:08:52.244633+0000","last_peered":"2024-03-21T09:08:52.244633+0000","last_clean":"2024-03-21T09:08:52.244633+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:08:52.244633+0000","last_undegraded":"2024-03-21T09:08:52.244633+0000","last_fullsized":"2024-03-21T09:08:52.244633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245809+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:08:52.245809+0000","last_peered":"2024-03-21T09:08:52.245809+0000","last_clean":"2024-03-21T09:08:52.245809+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:08:52.245809+0000","last_undegraded":"2024-03-21T09:08:52.245809+0000","last_fullsized":"2024-03-21T09:08:52.245809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.106502+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:09:19.106502+0000","last_peered":"2024-03-21T09:09:19.106502+0000","last_clean":"2024-03-21T09:09:19.106502+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:09:19.106502+0000","last_undegraded":"2024-03-21T09:09:19.106502+0000","last_fullsized":"2024-03-21T09:09:19.106502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246351+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:08:52.246351+0000","last_peered":"2024-03-21T09:08:52.246351+0000","last_clean":"2024-03-21T09:08:52.246351+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:08:52.246351+0000","last_undegraded":"2024-03-21T09:08:52.246351+0000","last_fullsized":"2024-03-21T09:08:52.246351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246302+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:08:52.246302+0000","last_peered":"2024-03-21T09:08:52.246302+0000","last_clean":"2024-03-21T09:08:52.246302+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:08:52.246302+0000","last_undegraded":"2024-03-21T09:08:52.246302+0000","last_fullsized":"2024-03-21T09:08:52.246302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247063+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:08:52.247063+0000","last_peered":"2024-03-21T09:08:52.247063+0000","last_clean":"2024-03-21T09:08:52.247063+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:08:52.247063+0000","last_undegraded":"2024-03-21T09:08:52.247063+0000","last_fullsized":"2024-03-21T09:08:52.247063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245451+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:08:52.245451+0000","last_peered":"2024-03-21T09:08:52.245451+0000","last_clean":"2024-03-21T09:08:52.245451+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:08:52.245451+0000","last_undegraded":"2024-03-21T09:08:52.245451+0000","last_fullsized":"2024-03-21T09:08:52.245451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245049+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:08:52.245049+0000","last_peered":"2024-03-21T09:08:52.245049+0000","last_clean":"2024-03-21T09:08:52.245049+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:08:52.245049+0000","last_undegraded":"2024-03-21T09:08:52.245049+0000","last_fullsized":"2024-03-21T09:08:52.245049+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.964264+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:08:51.964264+0000","last_peered":"2024-03-21T09:08:51.964264+0000","last_clean":"2024-03-21T09:08:51.964264+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:08:51.964264+0000","last_undegraded":"2024-03-21T09:08:51.964264+0000","last_fullsized":"2024-03-21T09:08:51.964264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244758+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:08:52.244758+0000","last_peered":"2024-03-21T09:08:52.244758+0000","last_clean":"2024-03-21T09:08:52.244758+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:08:52.244758+0000","last_undegraded":"2024-03-21T09:08:52.244758+0000","last_fullsized":"2024-03-21T09:08:52.244758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246000+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:08:52.246000+0000","last_peered":"2024-03-21T09:08:52.246000+0000","last_clean":"2024-03-21T09:08:52.246000+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:08:52.246000+0000","last_undegraded":"2024-03-21T09:08:52.246000+0000","last_fullsized":"2024-03-21T09:08:52.246000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245074+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:08:52.245074+0000","last_peered":"2024-03-21T09:08:52.245074+0000","last_clean":"2024-03-21T09:08:52.245074+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:08:52.245074+0000","last_undegraded":"2024-03-21T09:08:52.245074+0000","last_fullsized":"2024-03-21T09:08:52.245074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244581+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:08:52.244581+0000","last_peered":"2024-03-21T09:08:52.244581+0000","last_clean":"2024-03-21T09:08:52.244581+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:08:52.244581+0000","last_undegraded":"2024-03-21T09:08:52.244581+0000","last_fullsized":"2024-03-21T09:08:52.244581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245652+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:08:52.245652+0000","last_peered":"2024-03-21T09:08:52.245652+0000","last_clean":"2024-03-21T09:08:52.245652+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:08:52.245652+0000","last_undegraded":"2024-03-21T09:08:52.245652+0000","last_fullsized":"2024-03-21T09:08:52.245652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246029+0000","last_change":"2024-03-21T09:08:46.817136+0000","last_active":"2024-03-21T09:08:52.246029+0000","last_peered":"2024-03-21T09:08:52.246029+0000","last_clean":"2024-03-21T09:08:52.246029+0000","last_became_active":"2024-03-21T09:08:46.816790+0000","last_became_peered":"2024-03-21T09:08:46.816790+0000","last_unstale":"2024-03-21T09:08:52.246029+0000","last_undegraded":"2024-03-21T09:08:52.246029+0000","last_fullsized":"2024-03-21T09:08:52.246029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:56:22.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:21.809039+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:09:21.809039+0000","last_peered":"2024-03-21T09:09:21.809039+0000","last_clean":"2024-03-21T09:09:21.809039+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:09:21.809039+0000","last_undegraded":"2024-03-21T09:09:21.809039+0000","last_fullsized":"2024-03-21T09:09:21.809039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245457+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:08:52.245457+0000","last_peered":"2024-03-21T09:08:52.245457+0000","last_clean":"2024-03-21T09:08:52.245457+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:08:52.245457+0000","last_undegraded":"2024-03-21T09:08:52.245457+0000","last_fullsized":"2024-03-21T09:08:52.245457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.962059+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:08:51.962059+0000","last_peered":"2024-03-21T09:08:51.962059+0000","last_clean":"2024-03-21T09:08:51.962059+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:08:51.962059+0000","last_undegraded":"2024-03-21T09:08:51.962059+0000","last_fullsized":"2024-03-21T09:08:51.962059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103497+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:09:19.103497+0000","last_peered":"2024-03-21T09:09:19.103497+0000","last_clean":"2024-03-21T09:09:19.103497+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:09:19.103497+0000","last_undegraded":"2024-03-21T09:09:19.103497+0000","last_fullsized":"2024-03-21T09:09:19.103497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244316+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:08:52.244316+0000","last_peered":"2024-03-21T09:08:52.244316+0000","last_clean":"2024-03-21T09:08:52.244316+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:08:52.244316+0000","last_undegraded":"2024-03-21T09:08:52.244316+0000","last_fullsized":"2024-03-21T09:08:52.244316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247500+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:08:52.247500+0000","last_peered":"2024-03-21T09:08:52.247500+0000","last_clean":"2024-03-21T09:08:52.247500+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:08:52.247500+0000","last_undegraded":"2024-03-21T09:08:52.247500+0000","last_fullsized":"2024-03-21T09:08:52.247500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245555+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:08:52.245555+0000","last_peered":"2024-03-21T09:08:52.245555+0000","last_clean":"2024-03-21T09:08:52.245555+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:08:52.245555+0000","last_undegraded":"2024-03-21T09:08:52.245555+0000","last_fullsized":"2024-03-21T09:08:52.245555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.114485+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:09:20.114485+0000","last_peered":"2024-03-21T09:09:20.114485+0000","last_clean":"2024-03-21T09:09:20.114485+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:09:20.114485+0000","last_undegraded":"2024-03-21T09:09:20.114485+0000","last_fullsized":"2024-03-21T09:09:20.114485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246286+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:08:52.246286+0000","last_peered":"2024-03-21T09:08:52.246286+0000","last_clean":"2024-03-21T09:08:52.246286+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:08:52.246286+0000","last_undegraded":"2024-03-21T09:08:52.246286+0000","last_fullsized":"2024-03-21T09:08:52.246286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247280+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:08:52.247280+0000","last_peered":"2024-03-21T09:08:52.247280+0000","last_clean":"2024-03-21T09:08:52.247280+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:08:52.247280+0000","last_undegraded":"2024-03-21T09:08:52.247280+0000","last_fullsized":"2024-03-21T09:08:52.247280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245381+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:08:52.245381+0000","last_peered":"2024-03-21T09:08:52.245381+0000","last_clean":"2024-03-21T09:08:52.245381+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:08:52.245381+0000","last_undegraded":"2024-03-21T09:08:52.245381+0000","last_fullsized":"2024-03-21T09:08:52.245381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960999+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:08:51.960999+0000","last_peered":"2024-03-21T09:08:51.960999+0000","last_clean":"2024-03-21T09:08:51.960999+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:08:51.960999+0000","last_undegraded":"2024-03-21T09:08:51.960999+0000","last_fullsized":"2024-03-21T09:08:51.960999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965064+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:08:51.965064+0000","last_peered":"2024-03-21T09:08:51.965064+0000","last_clean":"2024-03-21T09:08:51.965064+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:08:51.965064+0000","last_undegraded":"2024-03-21T09:08:51.965064+0000","last_fullsized":"2024-03-21T09:08:51.965064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.962914+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:08:51.962914+0000","last_peered":"2024-03-21T09:08:51.962914+0000","last_clean":"2024-03-21T09:08:51.962914+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:08:51.962914+0000","last_undegraded":"2024-03-21T09:08:51.962914+0000","last_fullsized":"2024-03-21T09:08:51.962914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245642+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:08:52.245642+0000","last_peered":"2024-03-21T09:08:52.245642+0000","last_clean":"2024-03-21T09:08:52.245642+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:08:52.245642+0000","last_undegraded":"2024-03-21T09:08:52.245642+0000","last_fullsized":"2024-03-21T09:08:52.245642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103639+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:09:19.103639+0000","last_peered":"2024-03-21T09:09:19.103639+0000","last_clean":"2024-03-21T09:09:19.103639+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:09:19.103639+0000","last_undegraded":"2024-03-21T09:09:19.103639+0000","last_fullsized":"2024-03-21T09:09:19.103639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247118+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:08:52.247118+0000","last_peered":"2024-03-21T09:08:52.247118+0000","last_clean":"2024-03-21T09:08:52.247118+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:08:52.247118+0000","last_undegraded":"2024-03-21T09:08:52.247118+0000","last_fullsized":"2024-03-21T09:08:52.247118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121780+0000","last_change":"2024-03-21T09:08:51.220889+0000","last_active":"2024-03-21T09:08:52.121780+0000","last_peered":"2024-03-21T09:08:52.121780+0000","last_clean":"2024-03-21T09:08:52.121780+0000","last_became_active":"2024-03-21T09:08:51.220186+0000","last_became_peered":"2024-03-21T09:08:51.220186+0000","last_unstale":"2024-03-21T09:08:52.121780+0000","last_undegraded":"2024-03-21T09:08:52.121780+0000","last_fullsized":"2024-03-21T09:08:52.121780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:56.603716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.108127+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:09:19.108127+0000","last_peered":"2024-03-21T09:09:19.108127+0000","last_clean":"2024-03-21T09:09:19.108127+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:09:19.108127+0000","last_undegraded":"2024-03-21T09:09:19.108127+0000","last_fullsized":"2024-03-21T09:09:19.108127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244688+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:08:52.244688+0000","last_peered":"2024-03-21T09:08:52.244688+0000","last_clean":"2024-03-21T09:08:52.244688+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:08:52.244688+0000","last_undegraded":"2024-03-21T09:08:52.244688+0000","last_fullsized":"2024-03-21T09:08:52.244688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965843+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:08:51.965843+0000","last_peered":"2024-03-21T09:08:51.965843+0000","last_clean":"2024-03-21T09:08:51.965843+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:08:51.965843+0000","last_undegraded":"2024-03-21T09:08:51.965843+0000","last_fullsized":"2024-03-21T09:08:51.965843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.107699+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:09:19.107699+0000","last_peered":"2024-03-21T09:09:19.107699+0000","last_clean":"2024-03-21T09:09:19.107699+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:09:19.107699+0000","last_undegraded":"2024-03-21T09:09:19.107699+0000","last_fullsized":"2024-03-21T09:09:19.107699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245081+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:08:52.245081+0000","last_peered":"2024-03-21T09:08:52.245081+0000","last_clean":"2024-03-21T09:08:52.245081+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:08:52.245081+0000","last_undegraded":"2024-03-21T09:08:52.245081+0000","last_fullsized":"2024-03-21T09:08:52.245081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121856+0000","last_change":"2024-03-21T09:08:50.974359+0000","last_active":"2024-03-21T09:08:52.121856+0000","last_peered":"2024-03-21T09:08:52.121856+0000","last_clean":"2024-03-21T09:08:52.121856+0000","last_became_active":"2024-03-21T09:08:50.973570+0000","last_became_peered":"2024-03-21T09:08:50.973570+0000","last_unstale":"2024-03-21T09:08:52.121856+0000","last_undegraded":"2024-03-21T09:08:52.121856+0000","last_fullsized":"2024-03-21T09:08:52.121856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:29.515148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245077+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:08:52.245077+0000","last_peered":"2024-03-21T09:08:52.245077+0000","last_clean":"2024-03-21T09:08:52.245077+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:08:52.245077+0000","last_undegraded":"2024-03-21T09:08:52.245077+0000","last_fullsized":"2024-03-21T09:08:52.245077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961987+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:08:51.961987+0000","last_peered":"2024-03-21T09:08:51.961987+0000","last_clean":"2024-03-21T09:08:51.961987+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:08:51.961987+0000","last_undegraded":"2024-03-21T09:08:51.961987+0000","last_fullsized":"2024-03-21T09:08:51.961987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121864+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:08:52.121864+0000","last_peered":"2024-03-21T09:08:52.121864+0000","last_clean":"2024-03-21T09:08:52.121864+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:08:52.121864+0000","last_undegraded":"2024-03-21T09:08:52.121864+0000","last_fullsized":"2024-03-21T09:08:52.121864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103356+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:09:19.103356+0000","last_peered":"2024-03-21T09:09:19.103356+0000","last_clean":"2024-03-21T09:09:19.103356+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:09:19.103356+0000","last_undegraded":"2024-03-21T09:09:19.103356+0000","last_fullsized":"2024-03-21T09:09:19.103356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245101+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:08:52.245101+0000","last_peered":"2024-03-21T09:08:52.245101+0000","last_clean":"2024-03-21T09:08:52.245101+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:08:52.245101+0000","last_undegraded":"2024-03-21T09:08:52.245101+0000","last_fullsized":"2024-03-21T09:08:52.245101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.959779+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:08:51.959779+0000","last_peered":"2024-03-21T09:08:51.959779+0000","last_clean":"2024-03-21T09:08:51.959779+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:08:51.959779+0000","last_undegraded":"2024-03-21T09:08:51.959779+0000","last_fullsized":"2024-03-21T09:08:51.959779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246166+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:08:52.246166+0000","last_peered":"2024-03-21T09:08:52.246166+0000","last_clean":"2024-03-21T09:08:52.246166+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:08:52.246166+0000","last_undegraded":"2024-03-21T09:08:52.246166+0000","last_fullsized":"2024-03-21T09:08:52.246166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245624+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:08:52.245624+0000","last_peered":"2024-03-21T09:08:52.245624+0000","last_clean":"2024-03-21T09:08:52.245624+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:08:52.245624+0000","last_undegraded":"2024-03-21T09:08:52.245624+0000","last_fullsized":"2024-03-21T09:08:52.245624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246629+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:08:52.246629+0000","last_peered":"2024-03-21T09:08:52.246629+0000","last_clean":"2024-03-21T09:08:52.246629+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:08:52.246629+0000","last_undegraded":"2024-03-21T09:08:52.246629+0000","last_fullsized":"2024-03-21T09:08:52.246629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247321+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:08:52.247321+0000","last_peered":"2024-03-21T09:08:52.247321+0000","last_clean":"2024-03-21T09:08:52.247321+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:08:52.247321+0000","last_undegraded":"2024-03-21T09:08:52.247321+0000","last_fullsized":"2024-03-21T09:08:52.247321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245494+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:08:52.245494+0000","last_peered":"2024-03-21T09:08:52.245494+0000","last_clean":"2024-03-21T09:08:52.245494+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:08:52.245494+0000","last_undegraded":"2024-03-21T09:08:52.245494+0000","last_fullsized":"2024-03-21T09:08:52.245494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245772+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:08:52.245772+0000","last_peered":"2024-03-21T09:08:52.245772+0000","last_clean":"2024-03-21T09:08:52.245772+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:08:52.245772+0000","last_undegraded":"2024-03-21T09:08:52.245772+0000","last_fullsized":"2024-03-21T09:08:52.245772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245563+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:08:52.245563+0000","last_peered":"2024-03-21T09:08:52.245563+0000","last_clean":"2024-03-21T09:08:52.245563+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:08:52.245563+0000","last_undegraded":"2024-03-21T09:08:52.245563+0000","last_fullsized":"2024-03-21T09:08:52.245563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245406+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:08:52.245406+0000","last_peered":"2024-03-21T09:08:52.245406+0000","last_clean":"2024-03-21T09:08:52.245406+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:08:52.245406+0000","last_undegraded":"2024-03-21T09:08:52.245406+0000","last_fullsized":"2024-03-21T09:08:52.245406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245199+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:08:52.245199+0000","last_peered":"2024-03-21T09:08:52.245199+0000","last_clean":"2024-03-21T09:08:52.245199+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:08:52.245199+0000","last_undegraded":"2024-03-21T09:08:52.245199+0000","last_fullsized":"2024-03-21T09:08:52.245199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245851+0000","last_change":"2024-03-21T09:08:46.820749+0000","last_active":"2024-03-21T09:08:52.245851+0000","last_peered":"2024-03-21T09:08:52.245851+0000","last_clean":"2024-03-21T09:08:52.245851+0000","last_became_active":"2024-03-21T09:08:46.820219+0000","last_became_peered":"2024-03-21T09:08:46.820219+0000","last_unstale":"2024-03-21T09:08:52.245851+0000","last_undegraded":"2024-03-21T09:08:52.245851+0000","last_fullsized":"2024-03-21T09:08:52.245851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:38.570982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244944+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:08:52.244944+0000","last_peered":"2024-03-21T09:08:52.244944+0000","last_clean":"2024-03-21T09:08:52.244944+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:08:52.244944+0000","last_undegraded":"2024-03-21T09:08:52.244944+0000","last_fullsized":"2024-03-21T09:08:52.244944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246702+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:08:52.246702+0000","last_peered":"2024-03-21T09:08:52.246702+0000","last_clean":"2024-03-21T09:08:52.246702+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:08:52.246702+0000","last_undegraded":"2024-03-21T09:08:52.246702+0000","last_fullsized":"2024-03-21T09:08:52.246702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245486+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:08:52.245486+0000","last_peered":"2024-03-21T09:08:52.245486+0000","last_clean":"2024-03-21T09:08:52.245486+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:08:52.245486+0000","last_undegraded":"2024-03-21T09:08:52.245486+0000","last_fullsized":"2024-03-21T09:08:52.245486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245733+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:08:52.245733+0000","last_peered":"2024-03-21T09:08:52.245733+0000","last_clean":"2024-03-21T09:08:52.245733+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:08:52.245733+0000","last_undegraded":"2024-03-21T09:08:52.245733+0000","last_fullsized":"2024-03-21T09:08:52.245733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245476+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:08:52.245476+0000","last_peered":"2024-03-21T09:08:52.245476+0000","last_clean":"2024-03-21T09:08:52.245476+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:08:52.245476+0000","last_undegraded":"2024-03-21T09:08:52.245476+0000","last_fullsized":"2024-03-21T09:08:52.245476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963102+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:08:51.963102+0000","last_peered":"2024-03-21T09:08:51.963102+0000","last_clean":"2024-03-21T09:08:51.963102+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:08:51.963102+0000","last_undegraded":"2024-03-21T09:08:51.963102+0000","last_fullsized":"2024-03-21T09:08:51.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245909+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:08:52.245909+0000","last_peered":"2024-03-21T09:08:52.245909+0000","last_clean":"2024-03-21T09:08:52.245909+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:08:52.245909+0000","last_undegraded":"2024-03-21T09:08:52.245909+0000","last_fullsized":"2024-03-21T09:08:52.245909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960953+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:08:51.960953+0000","last_peered":"2024-03-21T09:08:51.960953+0000","last_clean":"2024-03-21T09:08:51.960953+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:08:51.960953+0000","last_undegraded":"2024-03-21T09:08:51.960953+0000","last_fullsized":"2024-03-21T09:08:51.960953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960007+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:08:51.960007+0000","last_peered":"2024-03-21T09:08:51.960007+0000","last_clean":"2024-03-21T09:08:51.960007+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:08:51.960007+0000","last_undegraded":"2024-03-21T09:08:51.960007+0000","last_fullsized":"2024-03-21T09:08:51.960007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245200+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:08:52.245200+0000","last_peered":"2024-03-21T09:08:52.245200+0000","last_clean":"2024-03-21T09:08:52.245200+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:08:52.245200+0000","last_undegraded":"2024-03-21T09:08:52.245200+0000","last_fullsized":"2024-03-21T09:08:52.245200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245207+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:08:52.245207+0000","last_peered":"2024-03-21T09:08:52.245207+0000","last_clean":"2024-03-21T09:08:52.245207+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:08:52.245207+0000","last_undegraded":"2024-03-21T09:08:52.245207+0000","last_fullsized":"2024-03-21T09:08:52.245207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25435,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":19,"num_read_kb":27,"num_write":109,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":46400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":488548,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612635,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":1093205,"data_compressed":50952,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":484452,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":484474,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1075442,"data_compressed":50952,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495665,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":483960,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":487712,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483960,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":487746,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":484154,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":483892,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1079810,"data_compressed":50952,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:09:25.001 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.266+0000 7f2f2fe00640 1 -- 172.21.15.37:0/99902996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 msgr2=0x7f2f30106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.266+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/99902996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f30106d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009ca0 tx=0x7f2f1802c100 comp rx=0 tx=0).stop 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 -- 172.21.15.37:0/99902996 shutdown_connections 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/99902996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f30107290 0x7f2f301096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/99902996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f30106d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/99902996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f30102030 0x7f2f30104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 -- 172.21.15.37:0/99902996 >> 172.21.15.37:0/99902996 conn(0x7f2f300fba40 msgr2=0x7f2f300fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 -- 172.21.15.37:0/99902996 shutdown_connections 2024-03-21T09:09:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.267+0000 7f2f2fe00640 1 -- 172.21.15.37:0/99902996 wait complete. 2024-03-21T09:09:25.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.268+0000 7f2f2fe00640 1 Processor -- start 2024-03-21T09:09:25.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.268+0000 7f2f2fe00640 1 -- start start 2024-03-21T09:09:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f30102030 0x7f2f30199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f3019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f30107290 0x7f2f301a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3010d240 con 0x7f2f30102030 2024-03-21T09:09:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3010d0c0 con 0x7f2f30107290 2024-03-21T09:09:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3010d3c0 con 0x7f2f30104960 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f3019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f30102030 0x7f2f30199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f3019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50322/0 (socket says 172.21.15.37:50322) 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 -- 172.21.15.37:0/2034090462 learned_addr learned my addr 172.21.15.37:0/2034090462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f2f400640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f30107290 0x7f2f301a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 -- 172.21.15.37:0/2034090462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f30107290 msgr2=0x7f2f301a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f30107290 0x7f2f301a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 -- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f30102030 msgr2=0x7f2f30199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f30102030 0x7f2f30199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.269+0000 7f2f26000640 1 -- 172.21.15.37:0/2034090462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f301a1740 con 0x7f2f30104960 2024-03-21T09:09:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.270+0000 7f2f26000640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f3019a060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000f80 tx=0x7f2f18035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.270+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18039040 con 0x7f2f30104960 2024-03-21T09:09:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.270+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f18044070 con 0x7f2f30104960 2024-03-21T09:09:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.270+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f301a19d0 con 0x7f2f30104960 2024-03-21T09:09:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.270+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18004430 con 0x7f2f30104960 2024-03-21T09:09:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.270+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f301a1ee0 con 0x7f2f30104960 2024-03-21T09:09:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.271+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f30108de0 con 0x7f2f30104960 2024-03-21T09:09:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.273+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2f180045d0 con 0x7f2f30104960 2024-03-21T09:09:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.275+0000 7f2f34a00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2f140a3020 0x7f2f140a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.275+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2f180f1970 con 0x7f2f30104960 2024-03-21T09:09:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.275+0000 7f2f2ea00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2f140a3020 0x7f2f140a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.276+0000 7f2f2ea00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2f140a3020 0x7f2f140a54e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009bd0 tx=0x7f2f20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:25.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.283+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f1804e020 con 0x7f2f30104960 2024-03-21T09:09:25.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:25 smithi037 ceph-mon[31830]: pgmap v339: 129 pgs: 129 active+clean; 601 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-03-21T09:09:25.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:25 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2283196698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:25.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:25 smithi053 ceph-mon[36169]: pgmap v339: 129 pgs: 129 active+clean; 601 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-03-21T09:09:25.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:25 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2283196698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:25.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.484+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f30061ae0 con 0x7f2f140a3020 2024-03-21T09:09:25.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.513+0000 7f2f34a00640 1 -- 172.21.15.37:0/2034090462 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340620 (secure 0 0 0) 0x7f2f30061ae0 con 0x7f2f140a3020 2024-03-21T09:09:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:25.515 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:09:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2f140a3020 msgr2=0x7f2f140a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2f140a3020 0x7f2f140a54e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009bd0 tx=0x7f2f20005f90 comp rx=0 tx=0).stop 2024-03-21T09:09:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 msgr2=0x7f2f3019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f3019a060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000f80 tx=0x7f2f18035d10 comp rx=0 tx=0).stop 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 shutdown_connections 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2f140a3020 0x7f2f140a54e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f30107290 0x7f2f301a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f30104960 0x7f2f3019a060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 --2- 172.21.15.37:0/2034090462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f30102030 0x7f2f30199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 >> 172.21.15.37:0/2034090462 conn(0x7f2f300fba40 msgr2=0x7f2f301017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 shutdown_connections 2024-03-21T09:09:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:25.519+0000 7f2f2fe00640 1 -- 172.21.15.37:0/2034090462 wait complete. 2024-03-21T09:09:25.534 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-03-21T09:09:23.781805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615803,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":155,"num_read_kb":146,"num_write":290,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373396,"kb_used_data":11744,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568428,"statfs":{"total":1151940427776,"available":1151558070272,"internally_reserved":0,"allocated":12025856,"data_stored":7617355,"data_compressed":284787,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20936,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":19,"num_read_kb":27,"num_write":61,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006212"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246355+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:08:52.246355+0000","last_peered":"2024-03-21T09:08:52.246355+0000","last_clean":"2024-03-21T09:08:52.246355+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:08:52.246355+0000","last_undegraded":"2024-03-21T09:08:52.246355+0000","last_fullsized":"2024-03-21T09:08:52.246355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247039+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:08:52.247039+0000","last_peered":"2024-03-21T09:08:52.247039+0000","last_clean":"2024-03-21T09:08:52.247039+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:08:52.247039+0000","last_undegraded":"2024-03-21T09:08:52.247039+0000","last_fullsized":"2024-03-21T09:08:52.247039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246844+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:08:52.246844+0000","last_peered":"2024-03-21T09:08:52.246844+0000","last_clean":"2024-03-21T09:08:52.246844+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:08:52.246844+0000","last_undegraded":"2024-03-21T09:08:52.246844+0000","last_fullsized":"2024-03-21T09:08:52.246844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121949+0000","last_change":"2024-03-21T09:08:46.824151+0000","last_active":"2024-03-21T09:08:52.121949+0000","last_peered":"2024-03-21T09:08:52.121949+0000","last_clean":"2024-03-21T09:08:52.121949+0000","last_became_active":"2024-03-21T09:08:46.822770+0000","last_became_peered":"2024-03-21T09:08:46.822770+0000","last_unstale":"2024-03-21T09:08:52.121949+0000","last_undegraded":"2024-03-21T09:08:52.121949+0000","last_fullsized":"2024-03-21T09:08:52.121949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:34:26.969292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245999+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:08:52.245999+0000","last_peered":"2024-03-21T09:08:52.245999+0000","last_clean":"2024-03-21T09:08:52.245999+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:08:52.245999+0000","last_undegraded":"2024-03-21T09:08:52.245999+0000","last_fullsized":"2024-03-21T09:08:52.245999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244325+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:08:52.244325+0000","last_peered":"2024-03-21T09:08:52.244325+0000","last_clean":"2024-03-21T09:08:52.244325+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:08:52.244325+0000","last_undegraded":"2024-03-21T09:08:52.244325+0000","last_fullsized":"2024-03-21T09:08:52.244325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244861+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:08:52.244861+0000","last_peered":"2024-03-21T09:08:52.244861+0000","last_clean":"2024-03-21T09:08:52.244861+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:08:52.244861+0000","last_undegraded":"2024-03-21T09:08:52.244861+0000","last_fullsized":"2024-03-21T09:08:52.244861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961241+0000","last_change":"2024-03-21T09:08:46.851011+0000","last_active":"2024-03-21T09:08:51.961241+0000","last_peered":"2024-03-21T09:08:51.961241+0000","last_clean":"2024-03-21T09:08:51.961241+0000","last_became_active":"2024-03-21T09:08:46.850599+0000","last_became_peered":"2024-03-21T09:08:46.850599+0000","last_unstale":"2024-03-21T09:08:51.961241+0000","last_undegraded":"2024-03-21T09:08:51.961241+0000","last_fullsized":"2024-03-21T09:08:51.961241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:50:30.925086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960373+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:08:51.960373+0000","last_peered":"2024-03-21T09:08:51.960373+0000","last_clean":"2024-03-21T09:08:51.960373+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:08:51.960373+0000","last_undegraded":"2024-03-21T09:08:51.960373+0000","last_fullsized":"2024-03-21T09:08:51.960373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245102+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:08:52.245102+0000","last_peered":"2024-03-21T09:08:52.245102+0000","last_clean":"2024-03-21T09:08:52.245102+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:08:52.245102+0000","last_undegraded":"2024-03-21T09:08:52.245102+0000","last_fullsized":"2024-03-21T09:08:52.245102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245486+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:08:52.245486+0000","last_peered":"2024-03-21T09:08:52.245486+0000","last_clean":"2024-03-21T09:08:52.245486+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:08:52.245486+0000","last_undegraded":"2024-03-21T09:08:52.245486+0000","last_fullsized":"2024-03-21T09:08:52.245486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121788+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:08:52.121788+0000","last_peered":"2024-03-21T09:08:52.121788+0000","last_clean":"2024-03-21T09:08:52.121788+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:08:52.121788+0000","last_undegraded":"2024-03-21T09:08:52.121788+0000","last_fullsized":"2024-03-21T09:08:52.121788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245277+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:08:52.245277+0000","last_peered":"2024-03-21T09:08:52.245277+0000","last_clean":"2024-03-21T09:08:52.245277+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:08:52.245277+0000","last_undegraded":"2024-03-21T09:08:52.245277+0000","last_fullsized":"2024-03-21T09:08:52.245277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246438+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:08:52.246438+0000","last_peered":"2024-03-21T09:08:52.246438+0000","last_clean":"2024-03-21T09:08:52.246438+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:08:52.246438+0000","last_undegraded":"2024-03-21T09:08:52.246438+0000","last_fullsized":"2024-03-21T09:08:52.246438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246374+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:08:52.246374+0000","last_peered":"2024-03-21T09:08:52.246374+0000","last_clean":"2024-03-21T09:08:52.246374+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:08:52.246374+0000","last_undegraded":"2024-03-21T09:08:52.246374+0000","last_fullsized":"2024-03-21T09:08:52.246374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.959525+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:08:51.959525+0000","last_peered":"2024-03-21T09:08:51.959525+0000","last_clean":"2024-03-21T09:08:51.959525+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:08:51.959525+0000","last_undegraded":"2024-03-21T09:08:51.959525+0000","last_fullsized":"2024-03-21T09:08:51.959525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246166+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:08:52.246166+0000","last_peered":"2024-03-21T09:08:52.246166+0000","last_clean":"2024-03-21T09:08:52.246166+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:08:52.246166+0000","last_undegraded":"2024-03-21T09:08:52.246166+0000","last_fullsized":"2024-03-21T09:08:52.246166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.964924+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:08:51.964924+0000","last_peered":"2024-03-21T09:08:51.964924+0000","last_clean":"2024-03-21T09:08:51.964924+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:08:51.964924+0000","last_undegraded":"2024-03-21T09:08:51.964924+0000","last_fullsized":"2024-03-21T09:08:51.964924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245967+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:08:52.245967+0000","last_peered":"2024-03-21T09:08:52.245967+0000","last_clean":"2024-03-21T09:08:52.245967+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:08:52.245967+0000","last_undegraded":"2024-03-21T09:08:52.245967+0000","last_fullsized":"2024-03-21T09:08:52.245967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245855+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:08:52.245855+0000","last_peered":"2024-03-21T09:08:52.245855+0000","last_clean":"2024-03-21T09:08:52.245855+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:08:52.245855+0000","last_undegraded":"2024-03-21T09:08:52.245855+0000","last_fullsized":"2024-03-21T09:08:52.245855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965910+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:08:51.965910+0000","last_peered":"2024-03-21T09:08:51.965910+0000","last_clean":"2024-03-21T09:08:51.965910+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:08:51.965910+0000","last_undegraded":"2024-03-21T09:08:51.965910+0000","last_fullsized":"2024-03-21T09:08:51.965910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.962946+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:08:51.962946+0000","last_peered":"2024-03-21T09:08:51.962946+0000","last_clean":"2024-03-21T09:08:51.962946+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:08:51.962946+0000","last_undegraded":"2024-03-21T09:08:51.962946+0000","last_fullsized":"2024-03-21T09:08:51.962946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121684+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:08:52.121684+0000","last_peered":"2024-03-21T09:08:52.121684+0000","last_clean":"2024-03-21T09:08:52.121684+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:08:52.121684+0000","last_undegraded":"2024-03-21T09:08:52.121684+0000","last_fullsized":"2024-03-21T09:08:52.121684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963121+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:08:51.963121+0000","last_peered":"2024-03-21T09:08:51.963121+0000","last_clean":"2024-03-21T09:08:51.963121+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:08:51.963121+0000","last_undegraded":"2024-03-21T09:08:51.963121+0000","last_fullsized":"2024-03-21T09:08:51.963121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247523+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:08:52.247523+0000","last_peered":"2024-03-21T09:08:52.247523+0000","last_clean":"2024-03-21T09:08:52.247523+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:08:52.247523+0000","last_undegraded":"2024-03-21T09:08:52.247523+0000","last_fullsized":"2024-03-21T09:08:52.247523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121479+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:08:52.121479+0000","last_peered":"2024-03-21T09:08:52.121479+0000","last_clean":"2024-03-21T09:08:52.121479+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:08:52.121479+0000","last_undegraded":"2024-03-21T09:08:52.121479+0000","last_fullsized":"2024-03-21T09:08:52.121479+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247348+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:08:52.247348+0000","last_peered":"2024-03-21T09:08:52.247348+0000","last_clean":"2024-03-21T09:08:52.247348+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:08:52.247348+0000","last_undegraded":"2024-03-21T09:08:52.247348+0000","last_fullsized":"2024-03-21T09:08:52.247348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.959811+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:08:51.959811+0000","last_peered":"2024-03-21T09:08:51.959811+0000","last_clean":"2024-03-21T09:08:51.959811+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:08:51.959811+0000","last_undegraded":"2024-03-21T09:08:51.959811+0000","last_fullsized":"2024-03-21T09:08:51.959811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121407+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:08:52.121407+0000","last_peered":"2024-03-21T09:08:52.121407+0000","last_clean":"2024-03-21T09:08:52.121407+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:08:52.121407+0000","last_undegraded":"2024-03-21T09:08:52.121407+0000","last_fullsized":"2024-03-21T09:08:52.121407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246430+0000","last_change":"2024-03-21T09:08:45.818384+0000","last_active":"2024-03-21T09:08:52.246430+0000","last_peered":"2024-03-21T09:08:52.246430+0000","last_clean":"2024-03-21T09:08:52.246430+0000","last_became_active":"2024-03-21T09:08:45.818028+0000","last_became_peered":"2024-03-21T09:08:45.818028+0000","last_unstale":"2024-03-21T09:08:52.246430+0000","last_undegraded":"2024-03-21T09:08:52.246430+0000","last_fullsized":"2024-03-21T09:08:52.246430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:04.433350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247132+0000","last_change":"2024-03-21T09:08:46.818779+0000","last_active":"2024-03-21T09:08:52.247132+0000","last_peered":"2024-03-21T09:08:52.247132+0000","last_clean":"2024-03-21T09:08:52.247132+0000","last_became_active":"2024-03-21T09:08:46.818412+0000","last_became_peered":"2024-03-21T09:08:46.818412+0000","last_unstale":"2024-03-21T09:08:52.247132+0000","last_undegraded":"2024-03-21T09:08:52.247132+0000","last_fullsized":"2024-03-21T09:08:52.247132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:47:37.234884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245826+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:08:52.245826+0000","last_peered":"2024-03-21T09:08:52.245826+0000","last_clean":"2024-03-21T09:08:52.245826+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:08:52.245826+0000","last_undegraded":"2024-03-21T09:08:52.245826+0000","last_fullsized":"2024-03-21T09:08:52.245826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:21.809871+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:09:21.809871+0000","last_peered":"2024-03-21T09:09:21.809871+0000","last_clean":"2024-03-21T09:09:21.809871+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:09:21.809871+0000","last_undegraded":"2024-03-21T09:09:21.809871+0000","last_fullsized":"2024-03-21T09:09:21.809871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245107+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:08:52.245107+0000","last_peered":"2024-03-21T09:08:52.245107+0000","last_clean":"2024-03-21T09:08:52.245107+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:08:52.245107+0000","last_undegraded":"2024-03-21T09:08:52.245107+0000","last_fullsized":"2024-03-21T09:08:52.245107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245459+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:08:52.245459+0000","last_peered":"2024-03-21T09:08:52.245459+0000","last_clean":"2024-03-21T09:08:52.245459+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:08:52.245459+0000","last_undegraded":"2024-03-21T09:08:52.245459+0000","last_fullsized":"2024-03-21T09:08:52.245459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.123215+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:09:20.123215+0000","last_peered":"2024-03-21T09:09:20.123215+0000","last_clean":"2024-03-21T09:09:20.123215+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:09:20.123215+0000","last_undegraded":"2024-03-21T09:09:20.123215+0000","last_fullsized":"2024-03-21T09:09:20.123215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246873+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:08:52.246873+0000","last_peered":"2024-03-21T09:08:52.246873+0000","last_clean":"2024-03-21T09:08:52.246873+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:08:52.246873+0000","last_undegraded":"2024-03-21T09:08:52.246873+0000","last_fullsized":"2024-03-21T09:08:52.246873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246123+0000","last_change":"2024-03-21T09:08:50.962949+0000","last_active":"2024-03-21T09:08:52.246123+0000","last_peered":"2024-03-21T09:08:52.246123+0000","last_clean":"2024-03-21T09:08:52.246123+0000","last_became_active":"2024-03-21T09:08:50.962527+0000","last_became_peered":"2024-03-21T09:08:50.962527+0000","last_unstale":"2024-03-21T09:08:52.246123+0000","last_undegraded":"2024-03-21T09:08:52.246123+0000","last_fullsized":"2024-03-21T09:08:52.246123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:28.557444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.123319+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:09:20.123319+0000","last_peered":"2024-03-21T09:09:20.123319+0000","last_clean":"2024-03-21T09:09:20.123319+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:09:20.123319+0000","last_undegraded":"2024-03-21T09:09:20.123319+0000","last_fullsized":"2024-03-21T09:09:20.123319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961158+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:08:51.961158+0000","last_peered":"2024-03-21T09:08:51.961158+0000","last_clean":"2024-03-21T09:08:51.961158+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:08:51.961158+0000","last_undegraded":"2024-03-21T09:08:51.961158+0000","last_fullsized":"2024-03-21T09:08:51.961158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247083+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:08:52.247083+0000","last_peered":"2024-03-21T09:08:52.247083+0000","last_clean":"2024-03-21T09:08:52.247083+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:08:52.247083+0000","last_undegraded":"2024-03-21T09:08:52.247083+0000","last_fullsized":"2024-03-21T09:08:52.247083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245952+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:08:52.245952+0000","last_peered":"2024-03-21T09:08:52.245952+0000","last_clean":"2024-03-21T09:08:52.245952+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:08:52.245952+0000","last_undegraded":"2024-03-21T09:08:52.245952+0000","last_fullsized":"2024-03-21T09:08:52.245952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963598+0000","last_change":"2024-03-21T09:08:46.834298+0000","last_active":"2024-03-21T09:08:51.963598+0000","last_peered":"2024-03-21T09:08:51.963598+0000","last_clean":"2024-03-21T09:08:51.963598+0000","last_became_active":"2024-03-21T09:08:46.833681+0000","last_became_peered":"2024-03-21T09:08:46.833681+0000","last_unstale":"2024-03-21T09:08:51.963598+0000","last_undegraded":"2024-03-21T09:08:51.963598+0000","last_fullsized":"2024-03-21T09:08:51.963598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:54:41.342590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961571+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:08:51.961571+0000","last_peered":"2024-03-21T09:08:51.961571+0000","last_clean":"2024-03-21T09:08:51.961571+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:08:51.961571+0000","last_undegraded":"2024-03-21T09:08:51.961571+0000","last_fullsized":"2024-03-21T09:08:51.961571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.807067+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:09:20.807067+0000","last_peered":"2024-03-21T09:09:20.807067+0000","last_clean":"2024-03-21T09:09:20.807067+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:09:20.807067+0000","last_undegraded":"2024-03-21T09:09:20.807067+0000","last_fullsized":"2024-03-21T09:09:20.807067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960381+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:08:51.960381+0000","last_peered":"2024-03-21T09:08:51.960381+0000","last_clean":"2024-03-21T09:08:51.960381+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:08:51.960381+0000","last_undegraded":"2024-03-21T09:08:51.960381+0000","last_fullsized":"2024-03-21T09:08:51.960381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963372+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:08:51.963372+0000","last_peered":"2024-03-21T09:08:51.963372+0000","last_clean":"2024-03-21T09:08:51.963372+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:08:51.963372+0000","last_undegraded":"2024-03-21T09:08:51.963372+0000","last_fullsized":"2024-03-21T09:08:51.963372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103063+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:09:19.103063+0000","last_peered":"2024-03-21T09:09:19.103063+0000","last_clean":"2024-03-21T09:09:19.103063+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:09:19.103063+0000","last_undegraded":"2024-03-21T09:09:19.103063+0000","last_fullsized":"2024-03-21T09:09:19.103063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247384+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:08:52.247384+0000","last_peered":"2024-03-21T09:08:52.247384+0000","last_clean":"2024-03-21T09:08:52.247384+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:08:52.247384+0000","last_undegraded":"2024-03-21T09:08:52.247384+0000","last_fullsized":"2024-03-21T09:08:52.247384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121866+0000","last_change":"2024-03-21T09:08:50.974540+0000","last_active":"2024-03-21T09:08:52.121866+0000","last_peered":"2024-03-21T09:08:52.121866+0000","last_clean":"2024-03-21T09:08:52.121866+0000","last_became_active":"2024-03-21T09:08:50.973639+0000","last_became_peered":"2024-03-21T09:08:50.973639+0000","last_unstale":"2024-03-21T09:08:52.121866+0000","last_undegraded":"2024-03-21T09:08:52.121866+0000","last_fullsized":"2024-03-21T09:08:52.121866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:47.377024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.104653+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:09:19.104653+0000","last_peered":"2024-03-21T09:09:19.104653+0000","last_clean":"2024-03-21T09:09:19.104653+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:09:19.104653+0000","last_undegraded":"2024-03-21T09:09:19.104653+0000","last_fullsized":"2024-03-21T09:09:19.104653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960102+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:08:51.960102+0000","last_peered":"2024-03-21T09:08:51.960102+0000","last_clean":"2024-03-21T09:08:51.960102+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:08:51.960102+0000","last_undegraded":"2024-03-21T09:08:51.960102+0000","last_fullsized":"2024-03-21T09:08:51.960102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.964027+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:08:51.964027+0000","last_peered":"2024-03-21T09:08:51.964027+0000","last_clean":"2024-03-21T09:08:51.964027+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:08:51.964027+0000","last_undegraded":"2024-03-21T09:08:51.964027+0000","last_fullsized":"2024-03-21T09:08:51.964027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'20","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:21.821673+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:09:21.821673+0000","last_peered":"2024-03-21T09:09:21.821673+0000","last_clean":"2024-03-21T09:09:21.821673+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:09:21.821673+0000","last_undegraded":"2024-03-21T09:09:21.821673+0000","last_fullsized":"2024-03-21T09:09:21.821673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":18,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247161+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:08:52.247161+0000","last_peered":"2024-03-21T09:08:52.247161+0000","last_clean":"2024-03-21T09:08:52.247161+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:08:52.247161+0000","last_undegraded":"2024-03-21T09:08:52.247161+0000","last_fullsized":"2024-03-21T09:08:52.247161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247755+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:08:52.247755+0000","last_peered":"2024-03-21T09:08:52.247755+0000","last_clean":"2024-03-21T09:08:52.247755+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:08:52.247755+0000","last_undegraded":"2024-03-21T09:08:52.247755+0000","last_fullsized":"2024-03-21T09:08:52.247755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103901+0000","last_change":"2024-03-21T09:08:45.826707+0000","last_active":"2024-03-21T09:09:19.103901+0000","last_peered":"2024-03-21T09:09:19.103901+0000","last_clean":"2024-03-21T09:09:19.103901+0000","last_became_active":"2024-03-21T09:08:45.826364+0000","last_became_peered":"2024-03-21T09:08:45.826364+0000","last_unstale":"2024-03-21T09:09:19.103901+0000","last_undegraded":"2024-03-21T09:09:19.103901+0000","last_fullsized":"2024-03-21T09:09:19.103901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:38:48.129131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245058+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:08:52.245058+0000","last_peered":"2024-03-21T09:08:52.245058+0000","last_clean":"2024-03-21T09:08:52.245058+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:08:52.245058+0000","last_undegraded":"2024-03-21T09:08:52.245058+0000","last_fullsized":"2024-03-21T09:08:52.245058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245056+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:08:52.245056+0000","last_peered":"2024-03-21T09:08:52.245056+0000","last_clean":"2024-03-21T09:08:52.245056+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:08:52.245056+0000","last_undegraded":"2024-03-21T09:08:52.245056+0000","last_fullsized":"2024-03-21T09:08:52.245056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103955+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:09:19.103955+0000","last_peered":"2024-03-21T09:09:19.103955+0000","last_clean":"2024-03-21T09:09:19.103955+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:09:19.103955+0000","last_undegraded":"2024-03-21T09:09:19.103955+0000","last_fullsized":"2024-03-21T09:09:19.103955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244930+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:08:52.244930+0000","last_peered":"2024-03-21T09:08:52.244930+0000","last_clean":"2024-03-21T09:08:52.244930+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:08:52.244930+0000","last_undegraded":"2024-03-21T09:08:52.244930+0000","last_fullsized":"2024-03-21T09:08:52.244930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965580+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:08:51.965580+0000","last_peered":"2024-03-21T09:08:51.965580+0000","last_clean":"2024-03-21T09:08:51.965580+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:08:51.965580+0000","last_undegraded":"2024-03-21T09:08:51.965580+0000","last_fullsized":"2024-03-21T09:08:51.965580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121392+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:08:52.121392+0000","last_peered":"2024-03-21T09:08:52.121392+0000","last_clean":"2024-03-21T09:08:52.121392+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:08:52.121392+0000","last_undegraded":"2024-03-21T09:08:52.121392+0000","last_fullsized":"2024-03-21T09:08:52.121392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244633+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:08:52.244633+0000","last_peered":"2024-03-21T09:08:52.244633+0000","last_clean":"2024-03-21T09:08:52.244633+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:08:52.244633+0000","last_undegraded":"2024-03-21T09:08:52.244633+0000","last_fullsized":"2024-03-21T09:08:52.244633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245809+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:08:52.245809+0000","last_peered":"2024-03-21T09:08:52.245809+0000","last_clean":"2024-03-21T09:08:52.245809+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:08:52.245809+0000","last_undegraded":"2024-03-21T09:08:52.245809+0000","last_fullsized":"2024-03-21T09:08:52.245809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.106502+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:09:19.106502+0000","last_peered":"2024-03-21T09:09:19.106502+0000","last_clean":"2024-03-21T09:09:19.106502+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:09:19.106502+0000","last_undegraded":"2024-03-21T09:09:19.106502+0000","last_fullsized":"2024-03-21T09:09:19.106502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246351+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:08:52.246351+0000","last_peered":"2024-03-21T09:08:52.246351+0000","last_clean":"2024-03-21T09:08:52.246351+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:08:52.246351+0000","last_undegraded":"2024-03-21T09:08:52.246351+0000","last_fullsized":"2024-03-21T09:08:52.246351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246302+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:08:52.246302+0000","last_peered":"2024-03-21T09:08:52.246302+0000","last_clean":"2024-03-21T09:08:52.246302+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:08:52.246302+0000","last_undegraded":"2024-03-21T09:08:52.246302+0000","last_fullsized":"2024-03-21T09:08:52.246302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247063+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:08:52.247063+0000","last_peered":"2024-03-21T09:08:52.247063+0000","last_clean":"2024-03-21T09:08:52.247063+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:08:52.247063+0000","last_undegraded":"2024-03-21T09:08:52.247063+0000","last_fullsized":"2024-03-21T09:08:52.247063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245451+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:08:52.245451+0000","last_peered":"2024-03-21T09:08:52.245451+0000","last_clean":"2024-03-21T09:08:52.245451+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:08:52.245451+0000","last_undegraded":"2024-03-21T09:08:52.245451+0000","last_fullsized":"2024-03-21T09:08:52.245451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245049+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:08:52.245049+0000","last_peered":"2024-03-21T09:08:52.245049+0000","last_clean":"2024-03-21T09:08:52.245049+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:08:52.245049+0000","last_undegraded":"2024-03-21T09:08:52.245049+0000","last_fullsized":"2024-03-21T09:08:52.245049+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.964264+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:08:51.964264+0000","last_peered":"2024-03-21T09:08:51.964264+0000","last_clean":"2024-03-21T09:08:51.964264+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:08:51.964264+0000","last_undegraded":"2024-03-21T09:08:51.964264+0000","last_fullsized":"2024-03-21T09:08:51.964264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244758+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:08:52.244758+0000","last_peered":"2024-03-21T09:08:52.244758+0000","last_clean":"2024-03-21T09:08:52.244758+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:08:52.244758+0000","last_undegraded":"2024-03-21T09:08:52.244758+0000","last_fullsized":"2024-03-21T09:08:52.244758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246000+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:08:52.246000+0000","last_peered":"2024-03-21T09:08:52.246000+0000","last_clean":"2024-03-21T09:08:52.246000+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:08:52.246000+0000","last_undegraded":"2024-03-21T09:08:52.246000+0000","last_fullsized":"2024-03-21T09:08:52.246000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245074+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:08:52.245074+0000","last_peered":"2024-03-21T09:08:52.245074+0000","last_clean":"2024-03-21T09:08:52.245074+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:08:52.245074+0000","last_undegraded":"2024-03-21T09:08:52.245074+0000","last_fullsized":"2024-03-21T09:08:52.245074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244581+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:08:52.244581+0000","last_peered":"2024-03-21T09:08:52.244581+0000","last_clean":"2024-03-21T09:08:52.244581+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:08:52.244581+0000","last_undegraded":"2024-03-21T09:08:52.244581+0000","last_fullsized":"2024-03-21T09:08:52.244581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245652+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:08:52.245652+0000","last_peered":"2024-03-21T09:08:52.245652+0000","last_clean":"2024-03-21T09:08:52.245652+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:08:52.245652+0000","last_undegraded":"2024-03-21T09:08:52.245652+0000","last_fullsized":"2024-03-21T09:08:52.245652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246029+0000","last_change":"2024-03-21T09:08:46.817136+0000","last_active":"2024-03-21T09:08:52.246029+0000","last_peered":"2024-03-21T09:08:52.246029+0000","last_clean":"2024-03-21T09:08:52.246029+0000","last_became_active":"2024-03-21T09:08:46.816790+0000","last_became_peered":"2024-03-21T09:08:46.816790+0000","last_unstale":"2024-03-21T09:08:52.246029+0000","last_undegraded":"2024-03-21T09:08:52.246029+0000","last_fullsized":"2024-03-21T09:08:52.246029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:56:22.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:21.809039+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:09:21.809039+0000","last_peered":"2024-03-21T09:09:21.809039+0000","last_clean":"2024-03-21T09:09:21.809039+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:09:21.809039+0000","last_undegraded":"2024-03-21T09:09:21.809039+0000","last_fullsized":"2024-03-21T09:09:21.809039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245457+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:08:52.245457+0000","last_peered":"2024-03-21T09:08:52.245457+0000","last_clean":"2024-03-21T09:08:52.245457+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:08:52.245457+0000","last_undegraded":"2024-03-21T09:08:52.245457+0000","last_fullsized":"2024-03-21T09:08:52.245457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.962059+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:08:51.962059+0000","last_peered":"2024-03-21T09:08:51.962059+0000","last_clean":"2024-03-21T09:08:51.962059+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:08:51.962059+0000","last_undegraded":"2024-03-21T09:08:51.962059+0000","last_fullsized":"2024-03-21T09:08:51.962059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103497+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:09:19.103497+0000","last_peered":"2024-03-21T09:09:19.103497+0000","last_clean":"2024-03-21T09:09:19.103497+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:09:19.103497+0000","last_undegraded":"2024-03-21T09:09:19.103497+0000","last_fullsized":"2024-03-21T09:09:19.103497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244316+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:08:52.244316+0000","last_peered":"2024-03-21T09:08:52.244316+0000","last_clean":"2024-03-21T09:08:52.244316+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:08:52.244316+0000","last_undegraded":"2024-03-21T09:08:52.244316+0000","last_fullsized":"2024-03-21T09:08:52.244316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247500+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:08:52.247500+0000","last_peered":"2024-03-21T09:08:52.247500+0000","last_clean":"2024-03-21T09:08:52.247500+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:08:52.247500+0000","last_undegraded":"2024-03-21T09:08:52.247500+0000","last_fullsized":"2024-03-21T09:08:52.247500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245555+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:08:52.245555+0000","last_peered":"2024-03-21T09:08:52.245555+0000","last_clean":"2024-03-21T09:08:52.245555+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:08:52.245555+0000","last_undegraded":"2024-03-21T09:08:52.245555+0000","last_fullsized":"2024-03-21T09:08:52.245555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:20.114485+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:09:20.114485+0000","last_peered":"2024-03-21T09:09:20.114485+0000","last_clean":"2024-03-21T09:09:20.114485+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:09:20.114485+0000","last_undegraded":"2024-03-21T09:09:20.114485+0000","last_fullsized":"2024-03-21T09:09:20.114485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246286+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:08:52.246286+0000","last_peered":"2024-03-21T09:08:52.246286+0000","last_clean":"2024-03-21T09:08:52.246286+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:08:52.246286+0000","last_undegraded":"2024-03-21T09:08:52.246286+0000","last_fullsized":"2024-03-21T09:08:52.246286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247280+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:08:52.247280+0000","last_peered":"2024-03-21T09:08:52.247280+0000","last_clean":"2024-03-21T09:08:52.247280+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:08:52.247280+0000","last_undegraded":"2024-03-21T09:08:52.247280+0000","last_fullsized":"2024-03-21T09:08:52.247280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245381+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:08:52.245381+0000","last_peered":"2024-03-21T09:08:52.245381+0000","last_clean":"2024-03-21T09:08:52.245381+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:08:52.245381+0000","last_undegraded":"2024-03-21T09:08:52.245381+0000","last_fullsized":"2024-03-21T09:08:52.245381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960999+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:08:51.960999+0000","last_peered":"2024-03-21T09:08:51.960999+0000","last_clean":"2024-03-21T09:08:51.960999+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:08:51.960999+0000","last_undegraded":"2024-03-21T09:08:51.960999+0000","last_fullsized":"2024-03-21T09:08:51.960999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965064+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:08:51.965064+0000","last_peered":"2024-03-21T09:08:51.965064+0000","last_clean":"2024-03-21T09:08:51.965064+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:08:51.965064+0000","last_undegraded":"2024-03-21T09:08:51.965064+0000","last_fullsized":"2024-03-21T09:08:51.965064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.962914+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:08:51.962914+0000","last_peered":"2024-03-21T09:08:51.962914+0000","last_clean":"2024-03-21T09:08:51.962914+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:08:51.962914+0000","last_undegraded":"2024-03-21T09:08:51.962914+0000","last_fullsized":"2024-03-21T09:08:51.962914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245642+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:08:52.245642+0000","last_peered":"2024-03-21T09:08:52.245642+0000","last_clean":"2024-03-21T09:08:52.245642+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:08:52.245642+0000","last_undegraded":"2024-03-21T09:08:52.245642+0000","last_fullsized":"2024-03-21T09:08:52.245642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103639+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:09:19.103639+0000","last_peered":"2024-03-21T09:09:19.103639+0000","last_clean":"2024-03-21T09:09:19.103639+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:09:19.103639+0000","last_undegraded":"2024-03-21T09:09:19.103639+0000","last_fullsized":"2024-03-21T09:09:19.103639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247118+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:08:52.247118+0000","last_peered":"2024-03-21T09:08:52.247118+0000","last_clean":"2024-03-21T09:08:52.247118+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:08:52.247118+0000","last_undegraded":"2024-03-21T09:08:52.247118+0000","last_fullsized":"2024-03-21T09:08:52.247118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121780+0000","last_change":"2024-03-21T09:08:51.220889+0000","last_active":"2024-03-21T09:08:52.121780+0000","last_peered":"2024-03-21T09:08:52.121780+0000","last_clean":"2024-03-21T09:08:52.121780+0000","last_became_active":"2024-03-21T09:08:51.220186+0000","last_became_peered":"2024-03-21T09:08:51.220186+0000","last_unstale":"2024-03-21T09:08:52.121780+0000","last_undegraded":"2024-03-21T09:08:52.121780+0000","last_fullsized":"2024-03-21T09:08:52.121780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:56.603716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.108127+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:09:19.108127+0000","last_peered":"2024-03-21T09:09:19.108127+0000","last_clean":"2024-03-21T09:09:19.108127+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:09:19.108127+0000","last_undegraded":"2024-03-21T09:09:19.108127+0000","last_fullsized":"2024-03-21T09:09:19.108127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244688+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:08:52.244688+0000","last_peered":"2024-03-21T09:08:52.244688+0000","last_clean":"2024-03-21T09:08:52.244688+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:08:52.244688+0000","last_undegraded":"2024-03-21T09:08:52.244688+0000","last_fullsized":"2024-03-21T09:08:52.244688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.965843+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:08:51.965843+0000","last_peered":"2024-03-21T09:08:51.965843+0000","last_clean":"2024-03-21T09:08:51.965843+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:08:51.965843+0000","last_undegraded":"2024-03-21T09:08:51.965843+0000","last_fullsized":"2024-03-21T09:08:51.965843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.107699+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:09:19.107699+0000","last_peered":"2024-03-21T09:09:19.107699+0000","last_clean":"2024-03-21T09:09:19.107699+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:09:19.107699+0000","last_undegraded":"2024-03-21T09:09:19.107699+0000","last_fullsized":"2024-03-21T09:09:19.107699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245081+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:08:52.245081+0000","last_peered":"2024-03-21T09:08:52.245081+0000","last_clean":"2024-03-21T09:08:52.245081+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:08:52.245081+0000","last_undegraded":"2024-03-21T09:08:52.245081+0000","last_fullsized":"2024-03-21T09:08:52.245081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121856+0000","last_change":"2024-03-21T09:08:50.974359+0000","last_active":"2024-03-21T09:08:52.121856+0000","last_peered":"2024-03-21T09:08:52.121856+0000","last_clean":"2024-03-21T09:08:52.121856+0000","last_became_active":"2024-03-21T09:08:50.973570+0000","last_became_peered":"2024-03-21T09:08:50.973570+0000","last_unstale":"2024-03-21T09:08:52.121856+0000","last_undegraded":"2024-03-21T09:08:52.121856+0000","last_fullsized":"2024-03-21T09:08:52.121856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:29.515148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245077+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:08:52.245077+0000","last_peered":"2024-03-21T09:08:52.245077+0000","last_clean":"2024-03-21T09:08:52.245077+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:08:52.245077+0000","last_undegraded":"2024-03-21T09:08:52.245077+0000","last_fullsized":"2024-03-21T09:08:52.245077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.961987+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:08:51.961987+0000","last_peered":"2024-03-21T09:08:51.961987+0000","last_clean":"2024-03-21T09:08:51.961987+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:08:51.961987+0000","last_undegraded":"2024-03-21T09:08:51.961987+0000","last_fullsized":"2024-03-21T09:08:51.961987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.121864+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:08:52.121864+0000","last_peered":"2024-03-21T09:08:52.121864+0000","last_clean":"2024-03-21T09:08:52.121864+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:08:52.121864+0000","last_undegraded":"2024-03-21T09:08:52.121864+0000","last_fullsized":"2024-03-21T09:08:52.121864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:09:19.103356+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:09:19.103356+0000","last_peered":"2024-03-21T09:09:19.103356+0000","last_clean":"2024-03-21T09:09:19.103356+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:09:19.103356+0000","last_undegraded":"2024-03-21T09:09:19.103356+0000","last_fullsized":"2024-03-21T09:09:19.103356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245101+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:08:52.245101+0000","last_peered":"2024-03-21T09:08:52.245101+0000","last_clean":"2024-03-21T09:08:52.245101+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:08:52.245101+0000","last_undegraded":"2024-03-21T09:08:52.245101+0000","last_fullsized":"2024-03-21T09:08:52.245101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.959779+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:08:51.959779+0000","last_peered":"2024-03-21T09:08:51.959779+0000","last_clean":"2024-03-21T09:08:51.959779+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:08:51.959779+0000","last_undegraded":"2024-03-21T09:08:51.959779+0000","last_fullsized":"2024-03-21T09:08:51.959779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246166+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:08:52.246166+0000","last_peered":"2024-03-21T09:08:52.246166+0000","last_clean":"2024-03-21T09:08:52.246166+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:08:52.246166+0000","last_undegraded":"2024-03-21T09:08:52.246166+0000","last_fullsized":"2024-03-21T09:08:52.246166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245624+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:08:52.245624+0000","last_peered":"2024-03-21T09:08:52.245624+0000","last_clean":"2024-03-21T09:08:52.245624+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:08:52.245624+0000","last_undegraded":"2024-03-21T09:08:52.245624+0000","last_fullsized":"2024-03-21T09:08:52.245624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246629+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:08:52.246629+0000","last_peered":"2024-03-21T09:08:52.246629+0000","last_clean":"2024-03-21T09:08:52.246629+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:08:52.246629+0000","last_undegraded":"2024-03-21T09:08:52.246629+0000","last_fullsized":"2024-03-21T09:08:52.246629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.247321+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:08:52.247321+0000","last_peered":"2024-03-21T09:08:52.247321+0000","last_clean":"2024-03-21T09:08:52.247321+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:08:52.247321+0000","last_undegraded":"2024-03-21T09:08:52.247321+0000","last_fullsized":"2024-03-21T09:08:52.247321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245494+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:08:52.245494+0000","last_peered":"2024-03-21T09:08:52.245494+0000","last_clean":"2024-03-21T09:08:52.245494+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:08:52.245494+0000","last_undegraded":"2024-03-21T09:08:52.245494+0000","last_fullsized":"2024-03-21T09:08:52.245494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245772+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:08:52.245772+0000","last_peered":"2024-03-21T09:08:52.245772+0000","last_clean":"2024-03-21T09:08:52.245772+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:08:52.245772+0000","last_undegraded":"2024-03-21T09:08:52.245772+0000","last_fullsized":"2024-03-21T09:08:52.245772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245563+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:08:52.245563+0000","last_peered":"2024-03-21T09:08:52.245563+0000","last_clean":"2024-03-21T09:08:52.245563+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:08:52.245563+0000","last_undegraded":"2024-03-21T09:08:52.245563+0000","last_fullsized":"2024-03-21T09:08:52.245563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245406+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:08:52.245406+0000","last_peered":"2024-03-21T09:08:52.245406+0000","last_clean":"2024-03-21T09:08:52.245406+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:08:52.245406+0000","last_undegraded":"2024-03-21T09:08:52.245406+0000","last_fullsized":"2024-03-21T09:08:52.245406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245199+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:08:52.245199+0000","last_peered":"2024-03-21T09:08:52.245199+0000","last_clean":"2024-03-21T09:08:52.245199+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:08:52.245199+0000","last_undegraded":"2024-03-21T09:08:52.245199+0000","last_fullsized":"2024-03-21T09:08:52.245199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245851+0000","last_change":"2024-03-21T09:08:46.820749+0000","last_active":"2024-03-21T09:08:52.245851+0000","last_peered":"2024-03-21T09:08:52.245851+0000","last_clean":"2024-03-21T09:08:52.245851+0000","last_became_active":"2024-03-21T09:08:46.820219+0000","last_became_peered":"2024-03-21T09:08:46.820219+0000","last_unstale":"2024-03-21T09:08:52.245851+0000","last_undegraded":"2024-03-21T09:08:52.245851+0000","last_fullsized":"2024-03-21T09:08:52.245851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:38.570982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.244944+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:08:52.244944+0000","last_peered":"2024-03-21T09:08:52.244944+0000","last_clean":"2024-03-21T09:08:52.244944+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:08:52.244944+0000","last_undegraded":"2024-03-21T09:08:52.244944+0000","last_fullsized":"2024-03-21T09:08:52.244944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.246702+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:08:52.246702+0000","last_peered":"2024-03-21T09:08:52.246702+0000","last_clean":"2024-03-21T09:08:52.246702+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:08:52.246702+0000","last_undegraded":"2024-03-21T09:08:52.246702+0000","last_fullsized":"2024-03-21T09:08:52.246702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245486+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:08:52.245486+0000","last_peered":"2024-03-21T09:08:52.245486+0000","last_clean":"2024-03-21T09:08:52.245486+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:08:52.245486+0000","last_undegraded":"2024-03-21T09:08:52.245486+0000","last_fullsized":"2024-03-21T09:08:52.245486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245733+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:08:52.245733+0000","last_peered":"2024-03-21T09:08:52.245733+0000","last_clean":"2024-03-21T09:08:52.245733+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:08:52.245733+0000","last_undegraded":"2024-03-21T09:08:52.245733+0000","last_fullsized":"2024-03-21T09:08:52.245733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245476+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:08:52.245476+0000","last_peered":"2024-03-21T09:08:52.245476+0000","last_clean":"2024-03-21T09:08:52.245476+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:08:52.245476+0000","last_undegraded":"2024-03-21T09:08:52.245476+0000","last_fullsized":"2024-03-21T09:08:52.245476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.963102+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:08:51.963102+0000","last_peered":"2024-03-21T09:08:51.963102+0000","last_clean":"2024-03-21T09:08:51.963102+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:08:51.963102+0000","last_undegraded":"2024-03-21T09:08:51.963102+0000","last_fullsized":"2024-03-21T09:08:51.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245909+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:08:52.245909+0000","last_peered":"2024-03-21T09:08:52.245909+0000","last_clean":"2024-03-21T09:08:52.245909+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:08:52.245909+0000","last_undegraded":"2024-03-21T09:08:52.245909+0000","last_fullsized":"2024-03-21T09:08:52.245909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960953+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:08:51.960953+0000","last_peered":"2024-03-21T09:08:51.960953+0000","last_clean":"2024-03-21T09:08:51.960953+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:08:51.960953+0000","last_undegraded":"2024-03-21T09:08:51.960953+0000","last_fullsized":"2024-03-21T09:08:51.960953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:51.960007+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:08:51.960007+0000","last_peered":"2024-03-21T09:08:51.960007+0000","last_clean":"2024-03-21T09:08:51.960007+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:08:51.960007+0000","last_undegraded":"2024-03-21T09:08:51.960007+0000","last_fullsized":"2024-03-21T09:08:51.960007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245200+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:08:52.245200+0000","last_peered":"2024-03-21T09:08:52.245200+0000","last_clean":"2024-03-21T09:08:52.245200+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:08:52.245200+0000","last_undegraded":"2024-03-21T09:08:52.245200+0000","last_fullsized":"2024-03-21T09:08:52.245200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-21T09:08:52.245207+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:08:52.245207+0000","last_peered":"2024-03-21T09:08:52.245207+0000","last_clean":"2024-03-21T09:08:52.245207+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:08:52.245207+0000","last_undegraded":"2024-03-21T09:08:52.245207+0000","last_fullsized":"2024-03-21T09:08:52.245207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25435,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":19,"num_read_kb":27,"num_write":109,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":46400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":488548,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612635,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":1093205,"data_compressed":50952,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":484452,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":484474,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1075442,"data_compressed":50952,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495665,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":483960,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":487712,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":483960,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":487746,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":484154,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":483892,"data_compressed":14659,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1079810,"data_compressed":50952,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:09:25.538 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-21T09:09:25.539 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T09:09:25.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T09:09:25.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:25 smithi071 ceph-mon[36597]: pgmap v339: 129 pgs: 129 active+clean; 601 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-03-21T09:09:25.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:25 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2283196698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:26.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:26 smithi037 ceph-mon[31830]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:09:26.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:26 smithi037 ceph-mon[31830]: from='client.24889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:09:26.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:26 smithi053 ceph-mon[36169]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:09:26.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:26 smithi053 ceph-mon[36169]: from='client.24889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:09:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:26 smithi071 ceph-mon[36597]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:09:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:26 smithi071 ceph-mon[36597]: from='client.24889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:09:27.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:27 smithi037 ceph-mon[31830]: pgmap v340: 129 pgs: 129 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-21T09:09:27.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:27 smithi053 ceph-mon[36169]: pgmap v340: 129 pgs: 129 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-21T09:09:27.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:27 smithi071 ceph-mon[36597]: pgmap v340: 129 pgs: 129 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 -- 172.21.15.37:0/1368770394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38103410 msgr2=0x7f2e38103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 --2- 172.21.15.37:0/1368770394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38103410 0x7f2e38103870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2e240099c0 tx=0x7f2e24009cd0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 -- 172.21.15.37:0/1368770394 shutdown_connections 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 --2- 172.21.15.37:0/1368770394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38103db0 0x7f2e3810a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 --2- 172.21.15.37:0/1368770394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38102210 0x7f2e38102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 --2- 172.21.15.37:0/1368770394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38103410 0x7f2e38103870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 -- 172.21.15.37:0/1368770394 >> 172.21.15.37:0/1368770394 conn(0x7f2e380fd7b0 msgr2=0x7f2e380ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:28.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 -- 172.21.15.37:0/1368770394 shutdown_connections 2024-03-21T09:09:28.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.406+0000 7f2e3f800700 1 -- 172.21.15.37:0/1368770394 wait complete. 2024-03-21T09:09:28.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 Processor -- start 2024-03-21T09:09:28.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 -- start start 2024-03-21T09:09:28.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 0x7f2e38198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:28.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38103410 0x7f2e381988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:28.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38103db0 0x7f2e3819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:28.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3810de40 con 0x7f2e38103db0 2024-03-21T09:09:28.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3810dcc0 con 0x7f2e38102210 2024-03-21T09:09:28.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.408+0000 7f2e3f800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3810dfc0 con 0x7f2e38103410 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 0x7f2e38198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 0x7f2e38198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:34752/0 (socket says 172.21.15.37:34752) 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 -- 172.21.15.37:0/3000919281 learned_addr learned my addr 172.21.15.37:0/3000919281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e37e00700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38103410 0x7f2e381988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3d800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38103db0 0x7f2e3819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 -- 172.21.15.37:0/3000919281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38103410 msgr2=0x7f2e381988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38103410 0x7f2e381988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 -- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38103db0 msgr2=0x7f2e3819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38103db0 0x7f2e3819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3ce00700 1 -- 172.21.15.37:0/3000919281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3819ffc0 con 0x7f2e38102210 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.409+0000 7f2e3d800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38103db0 0x7f2e3819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.410+0000 7f2e3ce00700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 0x7f2e38198360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00d980 tx=0x7f2e2c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.410+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c017030 con 0x7f2e38102210 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.410+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c004500 con 0x7f2e38102210 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.410+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c003aa0 con 0x7f2e38102210 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.410+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e381a02b0 con 0x7f2e38102210 2024-03-21T09:09:28.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.410+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e381a07a0 con 0x7f2e38102210 2024-03-21T09:09:28.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.411+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e380506a0 con 0x7f2e38102210 2024-03-21T09:09:28.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.413+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2e2c01a070 con 0x7f2e38102210 2024-03-21T09:09:28.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.415+0000 7f2e35600700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2e200a3170 0x7f2e200a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:28.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.415+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2e2c0cf110 con 0x7f2e38102210 2024-03-21T09:09:28.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.415+0000 7f2e37e00700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2e200a3170 0x7f2e200a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:28.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.416+0000 7f2e37e00700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2e200a3170 0x7f2e200a5630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:28.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.419+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e2c0232f0 con 0x7f2e38102210 2024-03-21T09:09:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.769+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2e3802dbd0 con 0x7f2e38102210 2024-03-21T09:09:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.775+0000 7f2e35600700 1 -- 172.21.15.37:0/3000919281 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2e3802dbd0 con 0x7f2e38102210 2024-03-21T09:09:28.770 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-03-21T09:09:28.771 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBI+ftlTjkHLhAAcMDGxBHgMmjcOM5YJoQnIQ== 2024-03-21T09:09:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.778+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2e200a3170 msgr2=0x7f2e200a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.778+0000 7f2e3f800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2e200a3170 0x7f2e200a5630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000c00 tx=0x7f2e24005fb0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.778+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 msgr2=0x7f2e38198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.778+0000 7f2e3f800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 0x7f2e38198360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00d980 tx=0x7f2e2c00dc90 comp rx=0 tx=0).stop 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 shutdown_connections 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e38103db0 0x7f2e3819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f2e200a3170 0x7f2e200a5630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e38102210 0x7f2e38198360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 --2- 172.21.15.37:0/3000919281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38103410 0x7f2e381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 >> 172.21.15.37:0/3000919281 conn(0x7f2e380fd7b0 msgr2=0x7f2e381066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 shutdown_connections 2024-03-21T09:09:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:28.779+0000 7f2e3f800700 1 -- 172.21.15.37:0/3000919281 wait complete. 2024-03-21T09:09:29.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:29 smithi037 ceph-mon[31830]: pgmap v341: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-03-21T09:09:29.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:29 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3000919281' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T09:09:29.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:29 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T09:09:29.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:29 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T09:09:29.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:29 smithi053 ceph-mon[36169]: pgmap v341: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-03-21T09:09:29.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:29 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3000919281' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T09:09:29.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:29 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T09:09:29.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:29 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T09:09:29.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:29 smithi071 ceph-mon[36597]: pgmap v341: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2024-03-21T09:09:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:29 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3000919281' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T09:09:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:29 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T09:09:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:29 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T09:09:29.609 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:09:29.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T09:09:29.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T09:09:29.781 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T09:09:29.782 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T09:09:29.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T09:09:30.308 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:31.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:31 smithi037 ceph-mon[31830]: pgmap v342: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-21T09:09:31.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:31 smithi071 ceph-mon[36597]: pgmap v342: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-21T09:09:31.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:31 smithi053 ceph-mon[36169]: pgmap v342: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 -- 172.21.15.37:0/3970239327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc1070d0 msgr2=0x7f79bc1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 --2- 172.21.15.37:0/3970239327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc1070d0 0x7f79bc1074b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009990 tx=0x7f79b0009ca0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 -- 172.21.15.37:0/3970239327 shutdown_connections 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 --2- 172.21.15.37:0/3970239327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 0x7f79bc10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 --2- 172.21.15.37:0/3970239327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc1070d0 0x7f79bc1074b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 --2- 172.21.15.37:0/3970239327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79bc101090 0x7f79bc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 -- 172.21.15.37:0/3970239327 >> 172.21.15.37:0/3970239327 conn(0x7f79bc077fc0 msgr2=0x7f79bc0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 -- 172.21.15.37:0/3970239327 shutdown_connections 2024-03-21T09:09:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.658+0000 7f79c3800700 1 -- 172.21.15.37:0/3970239327 wait complete. 2024-03-21T09:09:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.660+0000 7f79c3800700 1 Processor -- start 2024-03-21T09:09:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 -- start start 2024-03-21T09:09:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 0x7f79bc1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 0x7f79bc1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79bc1070d0 0x7f79bc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc111360 con 0x7f79bc1070d0 2024-03-21T09:09:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1111e0 con 0x7f79bc101a30 2024-03-21T09:09:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c3800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1114e0 con 0x7f79bc101090 2024-03-21T09:09:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 0x7f79bc1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 0x7f79bc1a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50356/0 (socket says 172.21.15.37:50356) 2024-03-21T09:09:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79c0e00700 1 -- 172.21.15.37:0/3917835477 learned_addr learned my addr 172.21.15.37:0/3917835477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.661+0000 7f79bbe00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 0x7f79bc1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c1800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79bc1070d0 0x7f79bc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c0e00700 1 -- 172.21.15.37:0/3917835477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 msgr2=0x7f79bc1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c0e00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 0x7f79bc1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c0e00700 1 -- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79bc1070d0 msgr2=0x7f79bc19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c0e00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79bc1070d0 0x7f79bc19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c0e00700 1 -- 172.21.15.37:0/3917835477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc19b0d0 con 0x7f79bc101090 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79bbe00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 0x7f79bc1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.662+0000 7f79c0e00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 0x7f79bc1a07b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005910 tx=0x7f79b000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.663+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0027070 con 0x7f79bc101090 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.663+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79b0004650 con 0x7f79bc101090 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.663+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000f460 con 0x7f79bc101090 2024-03-21T09:09:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.663+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc19b360 con 0x7f79bc101090 2024-03-21T09:09:32.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.663+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc1a7680 con 0x7f79bc101090 2024-03-21T09:09:32.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.664+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc068e50 con 0x7f79bc101090 2024-03-21T09:09:32.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.667+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f79b002a070 con 0x7f79bc101090 2024-03-21T09:09:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.668+0000 7f79b9600700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f79a40a30c0 0x7f79a40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.668+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f79b00d0670 con 0x7f79bc101090 2024-03-21T09:09:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.668+0000 7f79bbe00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f79a40a30c0 0x7f79a40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.669+0000 7f79bbe00700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f79a40a30c0 0x7f79a40a5580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79bc101030 tx=0x7f79a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.672+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79b005e740 con 0x7f79bc101090 2024-03-21T09:09:32.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.970+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f79bc1a7850 con 0x7f79bc101090 2024-03-21T09:09:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.977+0000 7f79b9600700 1 -- 172.21.15.37:0/3917835477 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+263552 (secure 0 0 0) 0x7f79b0013070 con 0x7f79bc101090 2024-03-21T09:09:32.975 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f79a40a30c0 msgr2=0x7f79a40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f79a40a30c0 0x7f79a40a5580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79bc101030 tx=0x7f79a8009260 comp rx=0 tx=0).stop 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 msgr2=0x7f79bc1a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 0x7f79bc1a07b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005910 tx=0x7f79b000bab0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 shutdown_connections 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79bc1070d0 0x7f79bc19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79bc101090 0x7f79bc1a07b0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005910 tx=0x7f79b000bab0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f79a40a30c0 0x7f79a40a5580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 --2- 172.21.15.37:0/3917835477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79bc101a30 0x7f79bc1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:32.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.982+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 >> 172.21.15.37:0/3917835477 conn(0x7f79bc077fc0 msgr2=0x7f79bc069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:32.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.983+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 shutdown_connections 2024-03-21T09:09:32.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:32.983+0000 7f79c3800700 1 -- 172.21.15.37:0/3917835477 wait complete. 2024-03-21T09:09:33.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:33 smithi037 ceph-mon[31830]: pgmap v343: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-21T09:09:33.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:33 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3917835477' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T09:09:33.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:33 smithi053 ceph-mon[36169]: pgmap v343: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-21T09:09:33.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:33 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3917835477' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T09:09:33.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:33 smithi071 ceph-mon[36597]: pgmap v343: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-21T09:09:33.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:33 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3917835477' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T09:09:33.741 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1300592626},{"type":"v1","addr":"172.21.15.37:6801","nonce":1300592626}]},"active_addr":"172.21.15.37:6801/1300592626","active_change":"2024-03-21T09:00:45.623903+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1173483074}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2805488546}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2204787263}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1759352225}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3473655253}]}]} 2024-03-21T09:09:33.747 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T09:09:33.747 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T09:09:33.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T09:09:34.172 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:35.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:35 smithi037 ceph-mon[31830]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-21T09:09:35.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:35 smithi053 ceph-mon[36169]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-21T09:09:35.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:35 smithi071 ceph-mon[36597]: pgmap v344: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-21T09:09:36.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.452+0000 7f6c80a00700 1 -- 172.21.15.37:0/3871322042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c1074f0 msgr2=0x7f6c7c10f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:36.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.452+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3871322042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c1074f0 0x7f6c7c10f0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400b0a0 tx=0x7f6c7400b3b0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.452+0000 7f6c80a00700 1 -- 172.21.15.37:0/3871322042 shutdown_connections 2024-03-21T09:09:36.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.452+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3871322042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c1074f0 0x7f6c7c10f0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.453+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3871322042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c7c102410 0x7f6c7c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.453+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3871322042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.453+0000 7f6c80a00700 1 -- 172.21.15.37:0/3871322042 >> 172.21.15.37:0/3871322042 conn(0x7f6c7c0fdc60 msgr2=0x7f6c7c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:36.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.453+0000 7f6c80a00700 1 -- 172.21.15.37:0/3871322042 shutdown_connections 2024-03-21T09:09:36.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.453+0000 7f6c80a00700 1 -- 172.21.15.37:0/3871322042 wait complete. 2024-03-21T09:09:36.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 Processor -- start 2024-03-21T09:09:36.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 -- start start 2024-03-21T09:09:36.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c7c102410 0x7f6c7c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:36.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:36.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c7c1074f0 0x7f6c7c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c112530 con 0x7f6c7c102410 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1123b0 con 0x7f6c7c102dc0 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.454+0000 7f6c80a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1126b0 con 0x7f6c7c1074f0 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c79e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c7c102410 0x7f6c7c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c1a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47726/0 (socket says 172.21.15.37:47726) 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 -- 172.21.15.37:0/3863065584 learned_addr learned my addr 172.21.15.37:0/3863065584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c7a800700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c7c1074f0 0x7f6c7c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 -- 172.21.15.37:0/3863065584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c7c1074f0 msgr2=0x7f6c7c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c7c1074f0 0x7f6c7c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 -- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c7c102410 msgr2=0x7f6c7c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c7c102410 0x7f6c7c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.455+0000 7f6c71400700 1 -- 172.21.15.37:0/3863065584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c19b090 con 0x7f6c7c102dc0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.456+0000 7f6c71400700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c1a0cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e970 tx=0x7f6c6400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.456+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004a30 con 0x7f6c7c102dc0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.456+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6400ccc0 con 0x7f6c7c102dc0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.456+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c19b380 con 0x7f6c7c102dc0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.456+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64005650 con 0x7f6c7c102dc0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.456+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c1a7640 con 0x7f6c7c102dc0 2024-03-21T09:09:36.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.458+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c068e50 con 0x7f6c7c102dc0 2024-03-21T09:09:36.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.459+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6c64004bd0 con 0x7f6c7c102dc0 2024-03-21T09:09:36.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.460+0000 7f6c73400700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6c600a30c0 0x7f6c600a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:36.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.460+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6c640cf910 con 0x7f6c7c102dc0 2024-03-21T09:09:36.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.461+0000 7f6c79e00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6c600a30c0 0x7f6c600a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:36.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.461+0000 7f6c79e00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6c600a30c0 0x7f6c600a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c19ba70 tx=0x7f6c6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.464+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c6409c340 con 0x7f6c7c102dc0 2024-03-21T09:09:36.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.767+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c7c1a78d0 con 0x7f6c7c102dc0 2024-03-21T09:09:36.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.769+0000 7f6c73400700 1 -- 172.21.15.37:0/3863065584 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f6c6409bc40 con 0x7f6c7c102dc0 2024-03-21T09:09:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:36.764 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:36.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6c600a30c0 msgr2=0x7f6c600a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:36.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6c600a30c0 0x7f6c600a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c19ba70 tx=0x7f6c6c009260 comp rx=0 tx=0).stop 2024-03-21T09:09:36.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 msgr2=0x7f6c7c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:36.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c1a0cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e970 tx=0x7f6c6400ec80 comp rx=0 tx=0).stop 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 shutdown_connections 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c7c1074f0 0x7f6c7c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c7c102410 0x7f6c7c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6c600a30c0 0x7f6c600a5580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 --2- 172.21.15.37:0/3863065584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c7c102dc0 0x7f6c7c1a0cb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 >> 172.21.15.37:0/3863065584 conn(0x7f6c7c0fdc60 msgr2=0x7f6c7c10d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.773+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 shutdown_connections 2024-03-21T09:09:36.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:36.774+0000 7f6c80a00700 1 -- 172.21.15.37:0/3863065584 wait complete. 2024-03-21T09:09:37.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:37 smithi037 ceph-mon[31830]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T09:09:37.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3863065584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:37.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:37 smithi053 ceph-mon[36169]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T09:09:37.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3863065584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:37.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:37 smithi071 ceph-mon[36597]: pgmap v345: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T09:09:37.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3863065584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:37.620 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T09:09:37.621 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T09:09:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:39.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:39 smithi037 ceph-mon[31830]: pgmap v346: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-21T09:09:39.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:39 smithi053 ceph-mon[36169]: pgmap v346: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-21T09:09:39.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:39 smithi071 ceph-mon[36597]: pgmap v346: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-21T09:09:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 -- 172.21.15.37:0/3184096212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c5406a7e0 msgr2=0x7f3c5410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 --2- 172.21.15.37:0/3184096212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c5406a7e0 0x7f3c5410f020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00b0a0 tx=0x7f3c4c00b3b0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 -- 172.21.15.37:0/3184096212 shutdown_connections 2024-03-21T09:09:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 --2- 172.21.15.37:0/3184096212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c5406a7e0 0x7f3c5410f020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 --2- 172.21.15.37:0/3184096212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c541081e0 0x7f3c541085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 --2- 172.21.15.37:0/3184096212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c5406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 -- 172.21.15.37:0/3184096212 >> 172.21.15.37:0/3184096212 conn(0x7f3c54077fc0 msgr2=0x7f3c540783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 -- 172.21.15.37:0/3184096212 shutdown_connections 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.418+0000 7f3c5a800700 1 -- 172.21.15.37:0/3184096212 wait complete. 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 Processor -- start 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 -- start start 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c541a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c5406a7e0 0x7f3c541a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c541081e0 0x7f3c5419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54112480 con 0x7f3c5406a7e0 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54112300 con 0x7f3c541081e0 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.419+0000 7f3c5a800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54112600 con 0x7f3c54069e40 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c541a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c52a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c5406a7e0 0x7f3c541a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c541a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40412/0 (socket says 172.21.15.37:40412) 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 -- 172.21.15.37:0/1138776523 learned_addr learned my addr 172.21.15.37:0/1138776523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53e00700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c541081e0 0x7f3c5419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 -- 172.21.15.37:0/1138776523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c541081e0 msgr2=0x7f3c5419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c541081e0 0x7f3c5419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 -- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c5406a7e0 msgr2=0x7f3c541a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c5406a7e0 0x7f3c541a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.420+0000 7f3c53400700 1 -- 172.21.15.37:0/1138776523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5419b0d0 con 0x7f3c54069e40 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.421+0000 7f3c53400700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c541a07b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400d980 tx=0x7f3c4400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.421+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44016070 con 0x7f3c54069e40 2024-03-21T09:09:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.421+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c44004d10 con 0x7f3c54069e40 2024-03-21T09:09:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.421+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5419b3c0 con 0x7f3c54069e40 2024-03-21T09:09:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.421+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c541a7680 con 0x7f3c54069e40 2024-03-21T09:09:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.421+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44005670 con 0x7f3c54069e40 2024-03-21T09:09:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.422+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c34005330 con 0x7f3c54069e40 2024-03-21T09:09:40.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.424+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3c44004220 con 0x7f3c54069e40 2024-03-21T09:09:40.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.425+0000 7f3c4be00700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3c380a30c0 0x7f3c380a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:40.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.426+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3c440cf3d0 con 0x7f3c54069e40 2024-03-21T09:09:40.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.426+0000 7f3c52a00700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3c380a30c0 0x7f3c380a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:40.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.426+0000 7f3c52a00700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3c380a30c0 0x7f3c380a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c54103030 tx=0x7f3c40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:40.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.429+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c4401f020 con 0x7f3c54069e40 2024-03-21T09:09:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.658+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c340058f0 con 0x7f3c54069e40 2024-03-21T09:09:40.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.660+0000 7f3c4be00700 1 -- 172.21.15.37:0/1138776523 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f3c44019e40 con 0x7f3c54069e40 2024-03-21T09:09:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:09:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:08:51.441766+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:09:40.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3c380a30c0 msgr2=0x7f3c380a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:40.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3c380a30c0 0x7f3c380a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c54103030 tx=0x7f3c40009260 comp rx=0 tx=0).stop 2024-03-21T09:09:40.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 msgr2=0x7f3c541a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:40.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c541a07b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400d980 tx=0x7f3c4400dc90 comp rx=0 tx=0).stop 2024-03-21T09:09:40.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 shutdown_connections 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3c380a30c0 0x7f3c380a5580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c541081e0 0x7f3c5419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c54069e40 0x7f3c541a07b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 --2- 172.21.15.37:0/1138776523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c5406a7e0 0x7f3c541a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 >> 172.21.15.37:0/1138776523 conn(0x7f3c54077fc0 msgr2=0x7f3c5410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.663+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 shutdown_connections 2024-03-21T09:09:40.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:40.664+0000 7f3c5a800700 1 -- 172.21.15.37:0/1138776523 wait complete. 2024-03-21T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:41 smithi037 ceph-mon[31830]: pgmap v347: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-21T09:09:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:41 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1138776523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:41.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:41 smithi053 ceph-mon[36169]: pgmap v347: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-21T09:09:41.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:41 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1138776523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:41.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T09:09:41.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T09:09:41.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T09:09:41.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T09:09:41.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T09:09:41.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T09:09:41.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T09:09:41.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T09:09:41.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-21T09:09:41.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-21T09:09:41.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-21T09:09:41.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-21T09:09:41.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:41 smithi071 ceph-mon[36597]: pgmap v347: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-21T09:09:41.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:41 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1138776523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:09:42.700 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:42.723 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:43.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:43 smithi037 ceph-mon[31830]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T09:09:43.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:43 smithi053 ceph-mon[36169]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T09:09:43.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:43 smithi071 ceph-mon[36597]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T09:09:45.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:45 smithi037 ceph-mon[31830]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:09:45.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:45 smithi053 ceph-mon[36169]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:09:45.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:45 smithi071 ceph-mon[36597]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-21T09:09:46.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:09:46.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-21T09:09:46.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:09:46.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-21T09:09:46.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-21T09:09:47.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:47 smithi037 ceph-mon[31830]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T09:09:47.332 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.338 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.343 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.347 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.348 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-21T09:09:47.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-21T09:09:47.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-21T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:47 smithi071 ceph-mon[36597]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T09:09:47.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-21T09:09:47.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:47 smithi053 ceph-mon[36169]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T09:09:47.571 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.575 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:47.604 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:09:48.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:48 smithi037 ceph-mon[31830]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T09:09:48.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:48 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:48 smithi071 ceph-mon[36597]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T09:09:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:48 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.633+0000 7f56ab200700 1 -- 172.21.15.37:0/615017353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103280 msgr2=0x7f56ac103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:48.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.633+0000 7f56ab200700 1 --2- 172.21.15.37:0/615017353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103280 0x7f56ac103660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f569c009990 tx=0x7f569c009ca0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 -- 172.21.15.37:0/615017353 shutdown_connections 2024-03-21T09:09:48.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 --2- 172.21.15.37:0/615017353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac10c6b0 0x7f56ac10eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 --2- 172.21.15.37:0/615017353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103280 0x7f56ac103660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 --2- 172.21.15.37:0/615017353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56ac103ba0 0x7f56ac10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 -- 172.21.15.37:0/615017353 >> 172.21.15.37:0/615017353 conn(0x7f56ac0fca50 msgr2=0x7f56ac0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:48.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 -- 172.21.15.37:0/615017353 shutdown_connections 2024-03-21T09:09:48.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.634+0000 7f56ab200700 1 -- 172.21.15.37:0/615017353 wait complete. 2024-03-21T09:09:48.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.635+0000 7f56ab200700 1 Processor -- start 2024-03-21T09:09:48.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 -- start start 2024-03-21T09:09:48.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac103280 0x7f56ac1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:48.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103ba0 0x7f56ac1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:48.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56ac10c6b0 0x7f56ac19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac1123f0 con 0x7f56ac10c6b0 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac112270 con 0x7f56ac103280 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56ab200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac112570 con 0x7f56ac103ba0 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.636+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac103280 0x7f56ac1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103ba0 0x7f56ac1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac103280 0x7f56ac1a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57160/0 (socket says 172.21.15.37:57160) 2024-03-21T09:09:48.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 -- 172.21.15.37:0/3264289336 learned_addr learned my addr 172.21.15.37:0/3264289336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:48.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56aa800700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56ac10c6b0 0x7f56ac19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:48.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 -- 172.21.15.37:0/3264289336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103ba0 msgr2=0x7f56ac1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:48.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56ac103ba0 0x7f56ac1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 -- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56ac10c6b0 msgr2=0x7f56ac19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56ac10c6b0 0x7f56ac19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56a9e00700 1 -- 172.21.15.37:0/3264289336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac19b040 con 0x7f56ac103280 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.637+0000 7f56aa800700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56ac10c6b0 0x7f56ac19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.638+0000 7f56a9e00700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac103280 0x7f56ac1a0720 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.638+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c027070 con 0x7f56ac103280 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.638+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f569c003d70 con 0x7f56ac103280 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.638+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac19b2d0 con 0x7f56ac103280 2024-03-21T09:09:48.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.638+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00f460 con 0x7f56ac103280 2024-03-21T09:09:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.638+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac1a75f0 con 0x7f56ac103280 2024-03-21T09:09:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.639+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f56ac102a00 con 0x7f56ac103280 2024-03-21T09:09:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.641+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f569c02a070 con 0x7f56ac103280 2024-03-21T09:09:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.642+0000 7f56a2a00700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f56900a30c0 0x7f56900a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.643+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f569c0d0600 con 0x7f56ac103280 2024-03-21T09:09:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.643+0000 7f56a2a00700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] conn(0x7f56900ae220 0x7f56900b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.643+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 --> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f569c001d30 con 0x7f56900ae220 2024-03-21T09:09:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.643+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f569c0d7050 con 0x7f56ac103280 2024-03-21T09:09:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.643+0000 7f56a9400700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f56900a30c0 0x7f56900a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.643+0000 7f56aa800700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] conn(0x7f56900ae220 0x7f56900b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:48.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.644+0000 7f56aa800700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] conn(0x7f56900ae220 0x7f56900b06c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:48.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.644+0000 7f56a9400700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f56900a30c0 0x7f56900a5580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56ac10df10 tx=0x7f5698009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:48.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.645+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== osd.4 v2:172.21.15.53:6808/1561544362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f569c001d30 con 0x7f56900ae220 2024-03-21T09:09:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.673+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 --> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f56ac06af70 con 0x7f56900ae220 2024-03-21T09:09:48.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.673+0000 7f56a2a00700 1 -- 172.21.15.37:0/3264289336 <== osd.4 v2:172.21.15.53:6808/1561544362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f56ac06af70 con 0x7f56900ae220 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] conn(0x7f56900ae220 msgr2=0x7f56900b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] conn(0x7f56900ae220 0x7f56900b06c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f56900a30c0 msgr2=0x7f56900a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f56900a30c0 0x7f56900a5580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56ac10df10 tx=0x7f5698009260 comp rx=0 tx=0).stop 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac103280 msgr2=0x7f56ac1a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 --2- 172.21.15.37:0/3264289336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac103280 0x7f56ac1a0720 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c00bab0 comp rx=0 tx=0).stop 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56aa800700 1 -- 172.21.15.37:0/3264289336 reap_dead start 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 shutdown_connections 2024-03-21T09:09:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 >> 172.21.15.37:0/3264289336 conn(0x7f56ac0fca50 msgr2=0x7f56ac0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.674+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 shutdown_connections 2024-03-21T09:09:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:48.675+0000 7f56ab200700 1 -- 172.21.15.37:0/3264289336 wait complete. 2024-03-21T09:09:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:48 smithi053 ceph-mon[36169]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T09:09:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:48 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:09:49.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:49 smithi037 ceph-mon[31830]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T09:09:49.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:49 smithi071 ceph-mon[36597]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T09:09:49.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:49 smithi053 ceph-mon[36169]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T09:09:51.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:51 smithi037 ceph-mon[31830]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:51.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:51 smithi071 ceph-mon[36597]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:51.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:51 smithi053 ceph-mon[36169]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:53.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:53 smithi037 ceph-mon[31830]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:53.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:53 smithi071 ceph-mon[36597]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:53.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:53 smithi053 ceph-mon[36169]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 -- 172.21.15.37:0/701233554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101030 msgr2=0x7f0520101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 --2- 172.21.15.37:0/701233554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101030 0x7f0520101410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0514009990 tx=0x7f0514009ca0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 -- 172.21.15.37:0/701233554 shutdown_connections 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 --2- 172.21.15.37:0/701233554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520106440 0x7f05201068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 --2- 172.21.15.37:0/701233554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101030 0x7f0520101410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 --2- 172.21.15.37:0/701233554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520101950 0x7f0520105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 -- 172.21.15.37:0/701233554 >> 172.21.15.37:0/701233554 conn(0x7f05200fca50 msgr2=0x7f05200fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 -- 172.21.15.37:0/701233554 shutdown_connections 2024-03-21T09:09:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.598+0000 7f0526c00700 1 -- 172.21.15.37:0/701233554 wait complete. 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.599+0000 7f0526c00700 1 Processor -- start 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 -- start start 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520101030 0x7f052019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101950 0x7f052019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520106440 0x7f05201a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520118a00 con 0x7f0520106440 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520118880 con 0x7f0520101030 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0526c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520118b80 con 0x7f0520101950 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0525800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520101030 0x7f052019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0525800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520101030 0x7f052019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58396/0 (socket says 172.21.15.37:58396) 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0525800700 1 -- 172.21.15.37:0/3047973160 learned_addr learned my addr 172.21.15.37:0/3047973160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.600+0000 7f0524e00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101950 0x7f052019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0526200700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520106440 0x7f05201a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0525800700 1 -- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101950 msgr2=0x7f052019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0525800700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101950 0x7f052019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0525800700 1 -- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520106440 msgr2=0x7f05201a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0525800700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520106440 0x7f05201a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0525800700 1 -- 172.21.15.37:0/3047973160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05201a3e80 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0524e00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520101950 0x7f052019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0526200700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520106440 0x7f05201a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.601+0000 7f0525800700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520101030 0x7f052019ee70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0514000c00 tx=0x7f051400ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.602+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514027070 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.602+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0514003d70 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.602+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05201a4110 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.602+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400f460 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.602+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05201ab9f0 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.603+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0520105860 con 0x7f0520101030 2024-03-21T09:09:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.605+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f051402a070 con 0x7f0520101030 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.606+0000 7f051e000700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a3040 0x7f05080a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.606+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f05140d0f00 con 0x7f0520101030 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.606+0000 7f0524e00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a3040 0x7f05080a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.606+0000 7f051e000700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] conn(0x7f05080ae1a0 0x7f05080b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.606+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 --> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0514018e50 con 0x7f05080ae1a0 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.606+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05140d12b0 con 0x7f0520101030 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.607+0000 7f0526200700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] conn(0x7f05080ae1a0 0x7f05080b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.607+0000 7f0524e00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a3040 0x7f05080a5500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0520100040 tx=0x7f050c00b360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.607+0000 7f0526200700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] conn(0x7f05080ae1a0 0x7f05080b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.609+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== osd.2 v2:172.21.15.71:6800/1968250372 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0514018e50 con 0x7f05080ae1a0 2024-03-21T09:09:54.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.630+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 --> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f052006afb0 con 0x7f05080ae1a0 2024-03-21T09:09:54.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f051e000700 1 -- 172.21.15.37:0/3047973160 <== osd.2 v2:172.21.15.71:6800/1968250372 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f052006afb0 con 0x7f05080ae1a0 2024-03-21T09:09:54.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] conn(0x7f05080ae1a0 msgr2=0x7f05080b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:54.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f0526c00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] conn(0x7f05080ae1a0 0x7f05080b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:54.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a3040 msgr2=0x7f05080a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f0526c00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a3040 0x7f05080a5500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0520100040 tx=0x7f050c00b360 comp rx=0 tx=0).stop 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520101030 msgr2=0x7f052019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.631+0000 7f0526c00700 1 --2- 172.21.15.37:0/3047973160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520101030 0x7f052019ee70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0514000c00 tx=0x7f051400ba90 comp rx=0 tx=0).stop 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.632+0000 7f0526200700 1 -- 172.21.15.37:0/3047973160 reap_dead start 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.632+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 shutdown_connections 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.632+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 >> 172.21.15.37:0/3047973160 conn(0x7f05200fca50 msgr2=0x7f0520103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.632+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 shutdown_connections 2024-03-21T09:09:54.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:54.632+0000 7f0526c00700 1 -- 172.21.15.37:0/3047973160 wait complete. 2024-03-21T09:09:55.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:55 smithi037 ceph-mon[31830]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:55.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:55 smithi071 ceph-mon[36597]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:55.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:55 smithi053 ceph-mon[36169]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.056+0000 7f34de200700 1 -- 172.21.15.37:0/2113314691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 msgr2=0x7f34d81026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.056+0000 7f34de200700 1 --2- 172.21.15.37:0/2113314691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 0x7f34d81026c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f34c80099c0 tx=0x7f34c8009cd0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 -- 172.21.15.37:0/2113314691 shutdown_connections 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 --2- 172.21.15.37:0/2113314691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d81073c0 0x7f34d810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 --2- 172.21.15.37:0/2113314691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 0x7f34d81026c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 --2- 172.21.15.37:0/2113314691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34d8102c90 0x7f34d8106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 -- 172.21.15.37:0/2113314691 >> 172.21.15.37:0/2113314691 conn(0x7f34d80fdb90 msgr2=0x7f34d80fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 -- 172.21.15.37:0/2113314691 shutdown_connections 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.057+0000 7f34de200700 1 -- 172.21.15.37:0/2113314691 wait complete. 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.058+0000 7f34de200700 1 Processor -- start 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.058+0000 7f34de200700 1 -- start start 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.058+0000 7f34de200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 0x7f34d8107f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.058+0000 7f34de200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8102c90 0x7f34d8108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34de200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34d81073c0 0x7f34d8108990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34de200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81123f0 con 0x7f34d81073c0 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34de200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8112270 con 0x7f34d8102c90 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34de200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8112570 con 0x7f34d81022e0 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34dce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 0x7f34d8107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34cfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8102c90 0x7f34d8108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34dce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 0x7f34d8107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47354/0 (socket says 172.21.15.37:47354) 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34dce00700 1 -- 172.21.15.37:0/3133007179 learned_addr learned my addr 172.21.15.37:0/3133007179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:56.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34dd800700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34d81073c0 0x7f34d8108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34cfe00700 1 -- 172.21.15.37:0/3133007179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 msgr2=0x7f34d8107f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34cfe00700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d81022e0 0x7f34d8107f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34cfe00700 1 -- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34d81073c0 msgr2=0x7f34d8108990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34cfe00700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34d81073c0 0x7f34d8108990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.059+0000 7f34cfe00700 1 -- 172.21.15.37:0/3133007179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d81ab0f0 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.060+0000 7f34cfe00700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8102c90 0x7f34d8108450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34c000e9f0 tx=0x7f34c000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.060+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c000cbd0 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.060+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d81ab3e0 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.060+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34c000cd70 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.060+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c0018710 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.060+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d81ab920 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.061+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34d810ecc0 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.063+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f34c0010430 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.064+0000 7f34d6000700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34bc0a3040 0x7f34bc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.064+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f34c00d0300 con 0x7f34d8102c90 2024-03-21T09:09:56.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.065+0000 7f34dce00700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34bc0a3040 0x7f34bc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.065+0000 7f34d6000700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] conn(0x7f34bc0ae1a0 0x7f34bc0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.065+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 --> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34bc0b0d10 con 0x7f34bc0ae1a0 2024-03-21T09:09:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.065+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34c00d06b0 con 0x7f34d8102c90 2024-03-21T09:09:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.065+0000 7f34dd800700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] conn(0x7f34bc0ae1a0 0x7f34bc0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.065+0000 7f34dce00700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34bc0a3040 0x7f34bc0a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34c8000c00 tx=0x7f34c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:56.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.066+0000 7f34dd800700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] conn(0x7f34bc0ae1a0 0x7f34bc0b0640 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.067+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== osd.0 v2:172.21.15.37:6802/2693651580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f34bc0b0d10 con 0x7f34bc0ae1a0 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.085+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 --> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34d81037c0 con 0x7f34bc0ae1a0 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.085+0000 7f34d6000700 1 -- 172.21.15.37:0/3133007179 <== osd.0 v2:172.21.15.37:6802/2693651580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f34d81037c0 con 0x7f34bc0ae1a0 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] conn(0x7f34bc0ae1a0 msgr2=0x7f34bc0b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] conn(0x7f34bc0ae1a0 0x7f34bc0b0640 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34bc0a3040 msgr2=0x7f34bc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34bc0a3040 0x7f34bc0a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34c8000c00 tx=0x7f34c8005fb0 comp rx=0 tx=0).stop 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8102c90 msgr2=0x7f34d8108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 --2- 172.21.15.37:0/3133007179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34d8102c90 0x7f34d8108450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34c000e9f0 tx=0x7f34c000ed00 comp rx=0 tx=0).stop 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34dd800700 1 -- 172.21.15.37:0/3133007179 reap_dead start 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 shutdown_connections 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 >> 172.21.15.37:0/3133007179 conn(0x7f34d80fdb90 msgr2=0x7f34d810e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 shutdown_connections 2024-03-21T09:09:56.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:52.086+0000 7f34de200700 1 -- 172.21.15.37:0/3133007179 wait complete. 2024-03-21T09:09:57.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:57 smithi037 ceph-mon[31830]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-03-21T09:09:57.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:57 smithi071 ceph-mon[36597]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-03-21T09:09:57.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:57 smithi053 ceph-mon[36169]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-03-21T09:09:58.355 INFO:teuthology.orchestra.run.smithi037.stdout:77309411408 2024-03-21T09:09:58.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T09:09:59.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:09:59 smithi037 ceph-mon[31830]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-21T09:09:59.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:09:59 smithi071 ceph-mon[36597]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-21T09:09:59.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:09:59 smithi053 ceph-mon[36169]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 -- 172.21.15.37:0/739204656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a81081e0 msgr2=0x7f90a81085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 --2- 172.21.15.37:0/739204656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a81081e0 0x7f90a81085c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f909c009990 tx=0x7f909c009ca0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 -- 172.21.15.37:0/739204656 shutdown_connections 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 --2- 172.21.15.37:0/739204656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90a806a7e0 0x7f90a810f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 --2- 172.21.15.37:0/739204656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a81081e0 0x7f90a81085c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 --2- 172.21.15.37:0/739204656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8069e40 0x7f90a806a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.718+0000 7f90af400700 1 -- 172.21.15.37:0/739204656 >> 172.21.15.37:0/739204656 conn(0x7f90a8077fc0 msgr2=0x7f90a80783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.719+0000 7f90af400700 1 -- 172.21.15.37:0/739204656 shutdown_connections 2024-03-21T09:09:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.719+0000 7f90af400700 1 -- 172.21.15.37:0/739204656 wait complete. 2024-03-21T09:09:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 Processor -- start 2024-03-21T09:09:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 -- start start 2024-03-21T09:09:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90a8069e40 0x7f90a81a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a806a7e0 0x7f90a81a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a81081e0 0x7f90a819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8112470 con 0x7f90a8069e40 2024-03-21T09:09:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81122f0 con 0x7f90a806a7e0 2024-03-21T09:09:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.721+0000 7f90af400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81125f0 con 0x7f90a81081e0 2024-03-21T09:09:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a806a7e0 0x7f90a81a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a806a7e0 0x7f90a81a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58402/0 (socket says 172.21.15.37:58402) 2024-03-21T09:09:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 -- 172.21.15.37:0/2095324381 learned_addr learned my addr 172.21.15.37:0/2095324381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:09:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90aca00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90a8069e40 0x7f90a81a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90ad400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a81081e0 0x7f90a819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 -- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a81081e0 msgr2=0x7f90a819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a81081e0 0x7f90a819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 -- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90a8069e40 msgr2=0x7f90a81a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90a8069e40 0x7f90a81a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90a7e00700 1 -- 172.21.15.37:0/2095324381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a819b150 con 0x7f90a806a7e0 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90aca00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90a8069e40 0x7f90a81a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.722+0000 7f90ad400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a81081e0 0x7f90a819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.723+0000 7f90a7e00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a806a7e0 0x7f90a81a0d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f909400d980 tx=0x7f909400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.723+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094016070 con 0x7f90a806a7e0 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.723+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9094004500 con 0x7f90a806a7e0 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.723+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a819b440 con 0x7f90a806a7e0 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.723+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094004d00 con 0x7f90a806a7e0 2024-03-21T09:09:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.723+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a81a7700 con 0x7f90a806a7e0 2024-03-21T09:09:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.724+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90a810dd50 con 0x7f90a806a7e0 2024-03-21T09:09:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.726+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f909400bbf0 con 0x7f90a806a7e0 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.727+0000 7f90a5600700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90900a30c0 0x7f90900a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.727+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f90940cf700 con 0x7f90a806a7e0 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.728+0000 7f90aca00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90900a30c0 0x7f90900a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.728+0000 7f90a5600700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] conn(0x7f90900ae220 0x7f90900b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.728+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 --> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90900b0d90 con 0x7f90900ae220 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.728+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90940d6050 con 0x7f90a806a7e0 2024-03-21T09:09:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.728+0000 7f90ad400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] conn(0x7f90900ae220 0x7f90900b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:09:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.728+0000 7f90aca00700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90900a30c0 0x7f90900a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f909c000c00 tx=0x7f909c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.729+0000 7f90ad400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] conn(0x7f90900ae220 0x7f90900b06c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:09:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.730+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== osd.8 v2:172.21.15.71:6816/4216652236 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f90900b0d90 con 0x7f90900ae220 2024-03-21T09:09:59.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.748+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 --> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90a8103890 con 0x7f90900ae220 2024-03-21T09:09:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.748+0000 7f90a5600700 1 -- 172.21.15.37:0/2095324381 <== osd.8 v2:172.21.15.71:6816/4216652236 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f90a8103890 con 0x7f90900ae220 2024-03-21T09:09:59.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.748+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] conn(0x7f90900ae220 msgr2=0x7f90900b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:59.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.748+0000 7f90af400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] conn(0x7f90900ae220 0x7f90900b06c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90900a30c0 msgr2=0x7f90900a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:59.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90900a30c0 0x7f90900a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f909c000c00 tx=0x7f909c005fb0 comp rx=0 tx=0).stop 2024-03-21T09:09:59.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a806a7e0 msgr2=0x7f90a81a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:09:59.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 --2- 172.21.15.37:0/2095324381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90a806a7e0 0x7f90a81a0d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f909400d980 tx=0x7f909400dd40 comp rx=0 tx=0).stop 2024-03-21T09:09:59.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90ad400700 1 -- 172.21.15.37:0/2095324381 reap_dead start 2024-03-21T09:09:59.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 shutdown_connections 2024-03-21T09:09:59.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 >> 172.21.15.37:0/2095324381 conn(0x7f90a8077fc0 msgr2=0x7f90a810d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:09:59.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 shutdown_connections 2024-03-21T09:09:59.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:09:59.749+0000 7f90af400700 1 -- 172.21.15.37:0/2095324381 wait complete. 2024-03-21T09:10:00.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:00 smithi037 ceph-mon[31830]: overall HEALTH_OK 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 -- 172.21.15.37:0/3913926202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 msgr2=0x7f159806a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 --2- 172.21.15.37:0/3913926202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 0x7f159806a2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f15800099c0 tx=0x7f1580009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 -- 172.21.15.37:0/3913926202 shutdown_connections 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 --2- 172.21.15.37:0/3913926202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 0x7f159810f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 --2- 172.21.15.37:0/3913926202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15981081e0 0x7f15981085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 --2- 172.21.15.37:0/3913926202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 0x7f159806a2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 -- 172.21.15.37:0/3913926202 >> 172.21.15.37:0/3913926202 conn(0x7f1598077fc0 msgr2=0x7f15980783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 -- 172.21.15.37:0/3913926202 shutdown_connections 2024-03-21T09:10:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.523+0000 7f159d600700 1 -- 172.21.15.37:0/3913926202 wait complete. 2024-03-21T09:10:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.524+0000 7f159d600700 1 Processor -- start 2024-03-21T09:10:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.524+0000 7f159d600700 1 -- start start 2024-03-21T09:10:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f159d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 0x7f15981a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f159d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 0x7f15981a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:00.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f159d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15981081e0 0x7f159819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:00.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f159d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15981124b0 con 0x7f15981081e0 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f159d600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598112330 con 0x7f159806a7e0 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f159d600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598112630 con 0x7f1598069e40 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 0x7f15981a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f1596a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 0x7f15981a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 0x7f15981a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58422/0 (socket says 172.21.15.37:58422) 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f1596a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 0x7f15981a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52866/0 (socket says 172.21.15.37:52866) 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 -- 172.21.15.37:0/3892326755 learned_addr learned my addr 172.21.15.37:0/3892326755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f1597400700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15981081e0 0x7f159819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 -- 172.21.15.37:0/3892326755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 msgr2=0x7f15981a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1598069e40 0x7f15981a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 -- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15981081e0 msgr2=0x7f159819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15981081e0 0x7f159819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.525+0000 7f158e000700 1 -- 172.21.15.37:0/3892326755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159819b0f0 con 0x7f159806a7e0 2024-03-21T09:10:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.526+0000 7f158e000700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 0x7f15981a0d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15800098e0 tx=0x7f1580003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:00.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.526+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800050c0 con 0x7f159806a7e0 2024-03-21T09:10:00.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.526+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f158000f460 con 0x7f159806a7e0 2024-03-21T09:10:00.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.526+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159819b380 con 0x7f159806a7e0 2024-03-21T09:10:00.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.526+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580017610 con 0x7f159806a7e0 2024-03-21T09:10:00.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.526+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15981a76a0 con 0x7f159806a7e0 2024-03-21T09:10:00.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.528+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1598100510 con 0x7f159806a7e0 2024-03-21T09:10:00.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.529+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f158000f610 con 0x7f159806a7e0 2024-03-21T09:10:00.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.530+0000 7f158fe00700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f157c0a30c0 0x7f157c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:00.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.531+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f15800d05a0 con 0x7f159806a7e0 2024-03-21T09:10:00.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.531+0000 7f1596a00700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f157c0a30c0 0x7f157c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:00.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.531+0000 7f158fe00700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] conn(0x7f157c0ae220 0x7f157c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:00.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.531+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 --> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f158001ee80 con 0x7f157c0ae220 2024-03-21T09:10:00.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.531+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15800d7050 con 0x7f159806a7e0 2024-03-21T09:10:00.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.531+0000 7f1597400700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] conn(0x7f157c0ae220 0x7f157c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:00.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.532+0000 7f1596a00700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f157c0a30c0 0x7f157c0a5580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f159819bad0 tx=0x7f158800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:00.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.532+0000 7f1597400700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] conn(0x7f157c0ae220 0x7f157c0b06c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:00.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.533+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== osd.10 v2:172.21.15.53:6824/3560446684 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f158001ee80 con 0x7f157c0ae220 2024-03-21T09:10:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.551+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 --> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1598103890 con 0x7f157c0ae220 2024-03-21T09:10:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f158fe00700 1 -- 172.21.15.37:0/3892326755 <== osd.10 v2:172.21.15.53:6824/3560446684 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1598103890 con 0x7f157c0ae220 2024-03-21T09:10:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] conn(0x7f157c0ae220 msgr2=0x7f157c0b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] conn(0x7f157c0ae220 0x7f157c0b06c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f157c0a30c0 msgr2=0x7f157c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f157c0a30c0 0x7f157c0a5580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f159819bad0 tx=0x7f158800b040 comp rx=0 tx=0).stop 2024-03-21T09:10:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 msgr2=0x7f15981a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 --2- 172.21.15.37:0/3892326755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159806a7e0 0x7f15981a0d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15800098e0 tx=0x7f1580003a90 comp rx=0 tx=0).stop 2024-03-21T09:10:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f1597400700 1 -- 172.21.15.37:0/3892326755 reap_dead start 2024-03-21T09:10:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 shutdown_connections 2024-03-21T09:10:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.552+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 >> 172.21.15.37:0/3892326755 conn(0x7f1598077fc0 msgr2=0x7f15980ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.553+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 shutdown_connections 2024-03-21T09:10:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:00.553+0000 7f159d600700 1 -- 172.21.15.37:0/3892326755 wait complete. 2024-03-21T09:10:00.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:00 smithi071 ceph-mon[36597]: overall HEALTH_OK 2024-03-21T09:10:00.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:00 smithi053 ceph-mon[36169]: overall HEALTH_OK 2024-03-21T09:10:01.503 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:01 smithi071 ceph-mon[36597]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-03-21T09:10:01.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:01 smithi037 ceph-mon[31830]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-03-21T09:10:01.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:01 smithi053 ceph-mon[36169]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-03-21T09:10:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 -- 172.21.15.37:0/2207428399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41073a0 msgr2=0x7f02a410ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 --2- 172.21.15.37:0/2207428399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41073a0 0x7f02a410ef70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f029c009f90 tx=0x7f029c00b330 comp rx=0 tx=0).stop 2024-03-21T09:10:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 -- 172.21.15.37:0/2207428399 shutdown_connections 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 --2- 172.21.15.37:0/2207428399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41073a0 0x7f02a410ef70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 --2- 172.21.15.37:0/2207428399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02a41022c0 0x7f02a41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 --2- 172.21.15.37:0/2207428399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02a4102c70 0x7f02a4106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 -- 172.21.15.37:0/2207428399 >> 172.21.15.37:0/2207428399 conn(0x7f02a40fdbb0 msgr2=0x7f02a40fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.589+0000 7f02a9800700 1 -- 172.21.15.37:0/2207428399 shutdown_connections 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.590+0000 7f02a9800700 1 -- 172.21.15.37:0/2207428399 wait complete. 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 Processor -- start 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 -- start start 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41022c0 0x7f02a41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02a4102c70 0x7f02a41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02a41073a0 0x7f02a419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41123e0 con 0x7f02a4102c70 2024-03-21T09:10:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4112260 con 0x7f02a41022c0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.591+0000 7f02a9800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4112560 con 0x7f02a41073a0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41022c0 0x7f02a41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41022c0 0x7f02a41a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58438/0 (socket says 172.21.15.37:58438) 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 -- 172.21.15.37:0/2919770551 learned_addr learned my addr 172.21.15.37:0/2919770551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f029be00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02a4102c70 0x7f02a41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a8e00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02a41073a0 0x7f02a419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 -- 172.21.15.37:0/2919770551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02a41073a0 msgr2=0x7f02a419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02a41073a0 0x7f02a419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 -- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02a4102c70 msgr2=0x7f02a41a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02a4102c70 0x7f02a41a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.592+0000 7f02a3e00700 1 -- 172.21.15.37:0/2919770551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a419b040 con 0x7f02a41022c0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.593+0000 7f02a3e00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41022c0 0x7f02a41a0720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f029400e9c0 tx=0x7f029400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.593+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294004a30 con 0x7f02a41022c0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.593+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f029400cca0 con 0x7f02a41022c0 2024-03-21T09:10:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.593+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a419b330 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.593+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294005610 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.593+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41a75f0 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.594+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02a40756a0 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.596+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f0294004cf0 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.597+0000 7f02a1600700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f02880a30c0 0x7f02880a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f02940cf990 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f029be00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f02880a30c0 0x7f02880a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f02a1600700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] conn(0x7f02880ae220 0x7f02880b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 --> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f029400ce50 con 0x7f02880ae220 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02940d42a0 con 0x7f02a41022c0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f02a8e00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] conn(0x7f02880ae220 0x7f02880b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:01.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.598+0000 7f029be00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f02880a30c0 0x7f02880a5580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02a40fe280 tx=0x7f028c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.599+0000 7f02a8e00700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] conn(0x7f02880ae220 0x7f02880b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.600+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== osd.6 v2:172.21.15.37:6818/1203908785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f029400ce50 con 0x7f02880ae220 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.624+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 --> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02a41037a0 con 0x7f02880ae220 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.624+0000 7f02a1600700 1 -- 172.21.15.37:0/2919770551 <== osd.6 v2:172.21.15.37:6818/1203908785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f02a41037a0 con 0x7f02880ae220 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] conn(0x7f02880ae220 msgr2=0x7f02880b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] conn(0x7f02880ae220 0x7f02880b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f02880a30c0 msgr2=0x7f02880a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f02880a30c0 0x7f02880a5580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02a40fe280 tx=0x7f028c009260 comp rx=0 tx=0).stop 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41022c0 msgr2=0x7f02a41a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 --2- 172.21.15.37:0/2919770551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02a41022c0 0x7f02a41a0720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f029400e9c0 tx=0x7f029400ecd0 comp rx=0 tx=0).stop 2024-03-21T09:10:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a8e00700 1 -- 172.21.15.37:0/2919770551 reap_dead start 2024-03-21T09:10:01.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 shutdown_connections 2024-03-21T09:10:01.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 >> 172.21.15.37:0/2919770551 conn(0x7f02a40fdbb0 msgr2=0x7f02a410ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:01.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 shutdown_connections 2024-03-21T09:10:01.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:01.625+0000 7f02a9800700 1 -- 172.21.15.37:0/2919770551 wait complete. 2024-03-21T09:10:03.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:03 smithi071 ceph-mon[36597]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-21T09:10:03.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:03 smithi037 ceph-mon[31830]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-21T09:10:03.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:03 smithi053 ceph-mon[36169]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-21T09:10:04.514 INFO:teuthology.orchestra.run.smithi037.stdout:253403070509 2024-03-21T09:10:04.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 -- 172.21.15.37:0/1366529295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5000743c0 msgr2=0x7fd5000747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 --2- 172.21.15.37:0/1366529295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5000743c0 0x7fd5000747a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00099c0 tx=0x7fd4f0009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 -- 172.21.15.37:0/1366529295 shutdown_connections 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 --2- 172.21.15.37:0/1366529295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd500113360 0x7fd500115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 --2- 172.21.15.37:0/1366529295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5000743c0 0x7fd5000747a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 --2- 172.21.15.37:0/1366529295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd500074ce0 0x7fd500112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 -- 172.21.15.37:0/1366529295 >> 172.21.15.37:0/1366529295 conn(0x7fd5000792f0 msgr2=0x7fd500079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 -- 172.21.15.37:0/1366529295 shutdown_connections 2024-03-21T09:10:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.757+0000 7fd507000700 1 -- 172.21.15.37:0/1366529295 wait complete. 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 Processor -- start 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 -- start start 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5000743c0 0x7fd50019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd500074ce0 0x7fd50019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd500113360 0x7fd5001a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500118b90 con 0x7fd500113360 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500118a10 con 0x7fd5000743c0 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.759+0000 7fd507000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500118d10 con 0x7fd500074ce0 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5000743c0 0x7fd50019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5000743c0 0x7fd50019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46940/0 (socket says 172.21.15.37:46940) 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 -- 172.21.15.37:0/3658871206 learned_addr learned my addr 172.21.15.37:0/3658871206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ff400700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd500074ce0 0x7fd50019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd505000700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd500113360 0x7fd5001a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 -- 172.21.15.37:0/3658871206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd500074ce0 msgr2=0x7fd50019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd500074ce0 0x7fd50019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 -- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd500113360 msgr2=0x7fd5001a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd500113360 0x7fd5001a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 -- 172.21.15.37:0/3658871206 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5001a4030 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.760+0000 7fd4ffe00700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5000743c0 0x7fd50019f020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006010 tx=0x7fd4f0003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.761+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00050c0 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.761+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5001a42c0 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.761+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f000f460 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.761+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0017650 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.761+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5001abba0 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.762+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd50010f7e0 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.764+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd4f000f610 con 0x7fd5000743c0 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.765+0000 7fd4fcc00700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4e40a3040 0x7fd4e40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.765+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd4f00d0690 con 0x7fd5000743c0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.765+0000 7fd4ff400700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4e40a3040 0x7fd4e40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.765+0000 7fd4fcc00700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] conn(0x7fd4e40ae1a0 0x7fd4e40b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.766+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 --> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4f004fe50 con 0x7fd4e40ae1a0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.766+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f00d0a40 con 0x7fd5000743c0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.766+0000 7fd505000700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] conn(0x7fd4e40ae1a0 0x7fd4e40b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.766+0000 7fd4ff400700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4e40a3040 0x7fd4e40a5500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd500112810 tx=0x7fd4ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.767+0000 7fd505000700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] conn(0x7fd4e40ae1a0 0x7fd4e40b0640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.768+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== osd.3 v2:172.21.15.37:6810/3211293526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd4f004fe50 con 0x7fd4e40ae1a0 2024-03-21T09:10:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.786+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 --> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd500075af0 con 0x7fd4e40ae1a0 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.787+0000 7fd4fcc00700 1 -- 172.21.15.37:0/3658871206 <== osd.3 v2:172.21.15.37:6810/3211293526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd500075af0 con 0x7fd4e40ae1a0 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] conn(0x7fd4e40ae1a0 msgr2=0x7fd4e40b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] conn(0x7fd4e40ae1a0 0x7fd4e40b0640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4e40a3040 msgr2=0x7fd4e40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd4e40a3040 0x7fd4e40a5500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd500112810 tx=0x7fd4ec009260 comp rx=0 tx=0).stop 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5000743c0 msgr2=0x7fd50019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 --2- 172.21.15.37:0/3658871206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5000743c0 0x7fd50019f020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006010 tx=0x7fd4f0003a90 comp rx=0 tx=0).stop 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd505000700 1 -- 172.21.15.37:0/3658871206 reap_dead start 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 shutdown_connections 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 >> 172.21.15.37:0/3658871206 conn(0x7fd5000792f0 msgr2=0x7fd50010ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 shutdown_connections 2024-03-21T09:10:04.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:04.788+0000 7fd507000700 1 -- 172.21.15.37:0/3658871206 wait complete. 2024-03-21T09:10:05.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:05 smithi037 ceph-mon[31830]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:05.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:05 smithi071 ceph-mon[36597]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:05.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:05 smithi053 ceph-mon[36169]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:06.030 INFO:teuthology.orchestra.run.smithi037.stdout:313532612643 2024-03-21T09:10:06.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-21T09:10:06.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 -- 172.21.15.37:0/3052665860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768100940 msgr2=0x7fb768100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 --2- 172.21.15.37:0/3052665860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768100940 0x7fb768100dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb754009990 tx=0x7fb754009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 -- 172.21.15.37:0/3052665860 shutdown_connections 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 --2- 172.21.15.37:0/3052665860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb768101300 0x7fb76810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 --2- 172.21.15.37:0/3052665860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7681070d0 0x7fb7681074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 --2- 172.21.15.37:0/3052665860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768100940 0x7fb768100dc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 -- 172.21.15.37:0/3052665860 >> 172.21.15.37:0/3052665860 conn(0x7fb7680762d0 msgr2=0x7fb7680766e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 -- 172.21.15.37:0/3052665860 shutdown_connections 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.844+0000 7fb76ee00700 1 -- 172.21.15.37:0/3052665860 wait complete. 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.845+0000 7fb76ee00700 1 Processor -- start 2024-03-21T09:10:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.845+0000 7fb76ee00700 1 -- start start 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76ee00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb768100940 0x7fb7681a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76ee00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768101300 0x7fb7681a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7681070d0 0x7fb76819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768112380 con 0x7fb7681070d0 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76ee00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768112200 con 0x7fb768100940 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76ee00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768112500 con 0x7fb768101300 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb768100940 0x7fb7681a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76da00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb768100940 0x7fb7681a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46954/0 (socket says 172.21.15.37:46954) 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76da00700 1 -- 172.21.15.37:0/2454627438 learned_addr learned my addr 172.21.15.37:0/2454627438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768101300 0x7fb7681a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.846+0000 7fb76e400700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7681070d0 0x7fb76819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76d000700 1 -- 172.21.15.37:0/2454627438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb768100940 msgr2=0x7fb7681a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76d000700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb768100940 0x7fb7681a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76d000700 1 -- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7681070d0 msgr2=0x7fb76819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76d000700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7681070d0 0x7fb76819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76d000700 1 -- 172.21.15.37:0/2454627438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76819b030 con 0x7fb768101300 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76e400700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7681070d0 0x7fb76819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76da00700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb768100940 0x7fb7681a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76d000700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768101300 0x7fb7681a0bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7540059d0 tx=0x7fb75400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754027070 con 0x7fb768101300 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb754003d70 con 0x7fb768101300 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76819b2c0 con 0x7fb768101300 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75400f460 con 0x7fb768101300 2024-03-21T09:10:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.847+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7681a7600 con 0x7fb768101300 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.848+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7680ff1d0 con 0x7fb768101300 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.851+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb75402a070 con 0x7fb768101300 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.852+0000 7fb766000700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7500a30c0 0x7fb7500a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.852+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb7540d1160 con 0x7fb768101300 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.852+0000 7fb76da00700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7500a30c0 0x7fb7500a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.852+0000 7fb766000700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] conn(0x7fb7500ae220 0x7fb7500b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.852+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 --> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7500b0d90 con 0x7fb7500ae220 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.852+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7540d1510 con 0x7fb768101300 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.853+0000 7fb76e400700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] conn(0x7fb7500ae220 0x7fb7500b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.853+0000 7fb76da00700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7500a30c0 0x7fb7500a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005950 tx=0x7fb75c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.853+0000 7fb76e400700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] conn(0x7fb7500ae220 0x7fb7500b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.854+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== osd.9 v2:172.21.15.37:6826/3963720627 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb7500b0d90 con 0x7fb7500ae220 2024-03-21T09:10:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.873+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 --> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7680699d0 con 0x7fb7500ae220 2024-03-21T09:10:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb766000700 1 -- 172.21.15.37:0/2454627438 <== osd.9 v2:172.21.15.37:6826/3963720627 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb7680699d0 con 0x7fb7500ae220 2024-03-21T09:10:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] conn(0x7fb7500ae220 msgr2=0x7fb7500b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb76ee00700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] conn(0x7fb7500ae220 0x7fb7500b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7500a30c0 msgr2=0x7fb7500a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb76ee00700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb7500a30c0 0x7fb7500a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005950 tx=0x7fb75c009260 comp rx=0 tx=0).stop 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768101300 msgr2=0x7fb7681a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.874+0000 7fb76ee00700 1 --2- 172.21.15.37:0/2454627438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb768101300 0x7fb7681a0bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7540059d0 tx=0x7fb75400bab0 comp rx=0 tx=0).stop 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.875+0000 7fb76e400700 1 -- 172.21.15.37:0/2454627438 reap_dead start 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.875+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 shutdown_connections 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.875+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 >> 172.21.15.37:0/2454627438 conn(0x7fb7680762d0 msgr2=0x7fb7680fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.875+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 shutdown_connections 2024-03-21T09:10:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:05.875+0000 7fb76ee00700 1 -- 172.21.15.37:0/2454627438 wait complete. 2024-03-21T09:10:06.525 INFO:teuthology.orchestra.run.smithi037.stdout:141733920835 2024-03-21T09:10:06.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T09:10:07.424 INFO:teuthology.orchestra.run.smithi037.stdout:107374182474 2024-03-21T09:10:07.425 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T09:10:07.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:07 smithi037 ceph-mon[31830]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:07.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:07 smithi071 ceph-mon[36597]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:07.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:07 smithi053 ceph-mon[36169]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:07.764 INFO:teuthology.orchestra.run.smithi037.stdout:197568495673 2024-03-21T09:10:07.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 -- 172.21.15.37:0/886257640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 msgr2=0x7f776c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 --2- 172.21.15.37:0/886257640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c106e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77580099c0 tx=0x7f7758009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 -- 172.21.15.37:0/886257640 shutdown_connections 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 --2- 172.21.15.37:0/886257640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f776c107510 0x7f776c10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 --2- 172.21.15.37:0/886257640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 0x7f776c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 --2- 172.21.15.37:0/886257640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c106e30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 -- 172.21.15.37:0/886257640 >> 172.21.15.37:0/886257640 conn(0x7f776c0792f0 msgr2=0x7f776c079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 -- 172.21.15.37:0/886257640 shutdown_connections 2024-03-21T09:10:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.122+0000 7f7772600700 1 -- 172.21.15.37:0/886257640 wait complete. 2024-03-21T09:10:08.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.123+0000 7f7772600700 1 Processor -- start 2024-03-21T09:10:08.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.123+0000 7f7772600700 1 -- start start 2024-03-21T09:10:08.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f7772600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 0x7f776c076080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f7772600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f7772600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f776c107510 0x7f776c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f7772600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c1125a0 con 0x7f776c102440 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f7772600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c112420 con 0x7f776c107510 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f7772600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c112720 con 0x7f776c102df0 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 0x7f776c076080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58452/0 (socket says 172.21.15.37:58452) 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776aa00700 1 -- 172.21.15.37:0/1856294426 learned_addr learned my addr 172.21.15.37:0/1856294426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776b400700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 0x7f776c076080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56386/0 (socket says 172.21.15.37:56386) 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776be00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f776c107510 0x7f776c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776aa00700 1 -- 172.21.15.37:0/1856294426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f776c107510 msgr2=0x7f776c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776aa00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f776c107510 0x7f776c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.124+0000 7f776aa00700 1 -- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 msgr2=0x7f776c076080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f776aa00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 0x7f776c076080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f776aa00700 1 -- 172.21.15.37:0/1856294426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c1ab4e0 con 0x7f776c102df0 2024-03-21T09:10:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f776b400700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f776c102440 0x7f776c076080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f776aa00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c074490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7758005b40 tx=0x7f7758003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:08.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758009e60 con 0x7f776c102df0 2024-03-21T09:10:08.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7758003680 con 0x7f776c102df0 2024-03-21T09:10:08.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.125+0000 7f7772600700 1 -- 172.21.15.37:0/1856294426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c1ab710 con 0x7f776c102df0 2024-03-21T09:10:08.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.126+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580173f0 con 0x7f776c102df0 2024-03-21T09:10:08.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.126+0000 7f7772600700 1 -- 172.21.15.37:0/1856294426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c1abb60 con 0x7f776c102df0 2024-03-21T09:10:08.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.127+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f776c0628a0 con 0x7f776c102df0 2024-03-21T09:10:08.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.129+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f775800f680 con 0x7f776c102df0 2024-03-21T09:10:08.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.130+0000 7f7763e00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f77500a30c0 0x7f77500a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.130+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f77580d06f0 con 0x7f776c102df0 2024-03-21T09:10:08.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.131+0000 7f776b400700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f77500a30c0 0x7f77500a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.131+0000 7f7763e00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] conn(0x7f77500ae220 0x7f77500b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.131+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 --> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f775800b790 con 0x7f77500ae220 2024-03-21T09:10:08.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.131+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77580d7050 con 0x7f776c102df0 2024-03-21T09:10:08.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.131+0000 7f776be00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] conn(0x7f77500ae220 0x7f77500b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.132+0000 7f776be00700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] conn(0x7f77500ae220 0x7f77500b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:08.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.132+0000 7f776b400700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f77500a30c0 0x7f77500a5580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f775c011490 tx=0x7f775c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:08.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.133+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== osd.5 v2:172.21.15.71:6808/2412287007 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f775800b790 con 0x7f77500ae220 2024-03-21T09:10:08.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.152+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 --> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f776c062ca0 con 0x7f77500ae220 2024-03-21T09:10:08.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.152+0000 7f7763e00700 1 -- 172.21.15.37:0/1856294426 <== osd.5 v2:172.21.15.71:6808/2412287007 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f776c062ca0 con 0x7f77500ae220 2024-03-21T09:10:08.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] conn(0x7f77500ae220 msgr2=0x7f77500b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] conn(0x7f77500ae220 0x7f77500b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f77500a30c0 msgr2=0x7f77500a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f77500a30c0 0x7f77500a5580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f775c011490 tx=0x7f775c009260 comp rx=0 tx=0).stop 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 msgr2=0x7f776c074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 --2- 172.21.15.37:0/1856294426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c102df0 0x7f776c074490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7758005b40 tx=0x7f7758003e00 comp rx=0 tx=0).stop 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f776be00700 1 -- 172.21.15.37:0/1856294426 reap_dead start 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 shutdown_connections 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 >> 172.21.15.37:0/1856294426 conn(0x7f776c0792f0 msgr2=0x7f776c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 shutdown_connections 2024-03-21T09:10:08.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:08.153+0000 7f7761600700 1 -- 172.21.15.37:0/1856294426 wait complete. 2024-03-21T09:10:08.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 -- 172.21.15.37:0/3306138952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6101081e0 msgr2=0x7fb6101085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 --2- 172.21.15.37:0/3306138952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6101081e0 0x7fb6101085c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb600009990 tx=0x7fb600009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 -- 172.21.15.37:0/3306138952 shutdown_connections 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 --2- 172.21.15.37:0/3306138952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb61006a7e0 0x7fb61010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 --2- 172.21.15.37:0/3306138952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6101081e0 0x7fb6101085c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 --2- 172.21.15.37:0/3306138952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb610069e40 0x7fb61006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.439+0000 7fb615400700 1 -- 172.21.15.37:0/3306138952 >> 172.21.15.37:0/3306138952 conn(0x7fb610077fc0 msgr2=0x7fb6100783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.440+0000 7fb615400700 1 -- 172.21.15.37:0/3306138952 shutdown_connections 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.440+0000 7fb615400700 1 -- 172.21.15.37:0/3306138952 wait complete. 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.441+0000 7fb615400700 1 Processor -- start 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.441+0000 7fb615400700 1 -- start start 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb615400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb610069e40 0x7fb610101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb615400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb61006a7e0 0x7fb6101016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb615400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6101081e0 0x7fb610101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb615400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb610112470 con 0x7fb610069e40 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb615400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101122f0 con 0x7fb61006a7e0 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb615400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101125f0 con 0x7fb6101081e0 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb60e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb610069e40 0x7fb610101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb60de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb61006a7e0 0x7fb6101016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb60de00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb61006a7e0 0x7fb6101016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46980/0 (socket says 172.21.15.37:46980) 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.442+0000 7fb60de00700 1 -- 172.21.15.37:0/1151693009 learned_addr learned my addr 172.21.15.37:0/1151693009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60f200700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6101081e0 0x7fb610101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60de00700 1 -- 172.21.15.37:0/1151693009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6101081e0 msgr2=0x7fb610101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60de00700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6101081e0 0x7fb610101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60de00700 1 -- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb610069e40 msgr2=0x7fb610101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60de00700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb610069e40 0x7fb610101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60de00700 1 -- 172.21.15.37:0/1151693009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6101a6e10 con 0x7fb61006a7e0 2024-03-21T09:10:08.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.443+0000 7fb60de00700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb61006a7e0 0x7fb6101016a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00e9a0 tx=0x7fb5fc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:08.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.444+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00cbd0 con 0x7fb61006a7e0 2024-03-21T09:10:08.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.444+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc00cd70 con 0x7fb61006a7e0 2024-03-21T09:10:08.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.444+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6101a7100 con 0x7fb61006a7e0 2024-03-21T09:10:08.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.444+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc018780 con 0x7fb61006a7e0 2024-03-21T09:10:08.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.444+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6101a7690 con 0x7fb61006a7e0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.445+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6100628a0 con 0x7fb61006a7e0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.447+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb5fc010430 con 0x7fb61006a7e0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.449+0000 7fb607400700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5f40a30c0 0x7fb5f40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.449+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb5fc0cfea0 con 0x7fb61006a7e0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.450+0000 7fb607400700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] conn(0x7fb5f40ae220 0x7fb5f40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.450+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 --> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5fc025e50 con 0x7fb5f40ae220 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.450+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5fc0d0250 con 0x7fb61006a7e0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.450+0000 7fb60e800700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5f40a30c0 0x7fb5f40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.450+0000 7fb60f200700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] conn(0x7fb5f40ae220 0x7fb5f40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.451+0000 7fb60e800700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5f40a30c0 0x7fb5f40a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb60000b2e0 tx=0x7fb60001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.451+0000 7fb60f200700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] conn(0x7fb5f40ae220 0x7fb5f40b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.452+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== osd.1 v2:172.21.15.53:6800/916967711 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb5fc025e50 con 0x7fb5f40ae220 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.485+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 --> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb610062ca0 con 0x7fb5f40ae220 2024-03-21T09:10:08.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.485+0000 7fb607400700 1 -- 172.21.15.37:0/1151693009 <== osd.1 v2:172.21.15.53:6800/916967711 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb610062ca0 con 0x7fb5f40ae220 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] conn(0x7fb5f40ae220 msgr2=0x7fb5f40b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] conn(0x7fb5f40ae220 0x7fb5f40b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5f40a30c0 msgr2=0x7fb5f40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5f40a30c0 0x7fb5f40a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb60000b2e0 tx=0x7fb60001a040 comp rx=0 tx=0).stop 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb61006a7e0 msgr2=0x7fb6101016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 --2- 172.21.15.37:0/1151693009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb61006a7e0 0x7fb6101016a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00e9a0 tx=0x7fb5fc00ecb0 comp rx=0 tx=0).stop 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb60f200700 1 -- 172.21.15.37:0/1151693009 reap_dead start 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 shutdown_connections 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 >> 172.21.15.37:0/1151693009 conn(0x7fb610077fc0 msgr2=0x7fb6100ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 shutdown_connections 2024-03-21T09:10:08.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:07.486+0000 7fb615400700 1 -- 172.21.15.37:0/1151693009 wait complete. 2024-03-21T09:10:09.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:09 smithi037 ceph-mon[31830]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:09.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:09 smithi071 ceph-mon[36597]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:09.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:09 smithi053 ceph-mon[36169]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:10.393 INFO:teuthology.orchestra.run.smithi037.stdout:34359738456 2024-03-21T09:10:10.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T09:10:10.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 -- 172.21.15.37:0/2225053024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded0104600 msgr2=0x7fded01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:10.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 --2- 172.21.15.37:0/2225053024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded0104600 0x7fded01049e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdec00099c0 tx=0x7fdec0009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 -- 172.21.15.37:0/2225053024 shutdown_connections 2024-03-21T09:10:10.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 --2- 172.21.15.37:0/2225053024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fded01096e0 0x7fded01112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 --2- 172.21.15.37:0/2225053024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded0104600 0x7fded01049e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 --2- 172.21.15.37:0/2225053024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fded0104fb0 0x7fded0109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.674+0000 7fded6600700 1 -- 172.21.15.37:0/2225053024 >> 172.21.15.37:0/2225053024 conn(0x7fded00ffe90 msgr2=0x7fded01022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:10.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.675+0000 7fded6600700 1 -- 172.21.15.37:0/2225053024 shutdown_connections 2024-03-21T09:10:10.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.675+0000 7fded6600700 1 -- 172.21.15.37:0/2225053024 wait complete. 2024-03-21T09:10:10.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.676+0000 7fded6600700 1 Processor -- start 2024-03-21T09:10:10.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.676+0000 7fded6600700 1 -- start start 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fded6600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fded0104600 0x7fded01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fded6600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fded0104fb0 0x7fded01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fded6600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded01096e0 0x7fded019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fded6600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded0114710 con 0x7fded0104600 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fded6600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded0114590 con 0x7fded0104fb0 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fded6600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded0114890 con 0x7fded01096e0 2024-03-21T09:10:10.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fdecfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded01096e0 0x7fded019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fdecea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fded0104fb0 0x7fded01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fdecfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded01096e0 0x7fded019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58466/0 (socket says 172.21.15.37:58466) 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.677+0000 7fdecfe00700 1 -- 172.21.15.37:0/1169304375 learned_addr learned my addr 172.21.15.37:0/1169304375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecf400700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fded0104600 0x7fded01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecfe00700 1 -- 172.21.15.37:0/1169304375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fded0104fb0 msgr2=0x7fded01a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecfe00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fded0104fb0 0x7fded01a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecfe00700 1 -- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fded0104600 msgr2=0x7fded01a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecfe00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fded0104600 0x7fded01a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecfe00700 1 -- 172.21.15.37:0/1169304375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded019b0b0 con 0x7fded01096e0 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecea00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fded0104fb0 0x7fded01a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecf400700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fded0104600 0x7fded01a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdecfe00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded01096e0 0x7fded019a950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdec8007ee0 tx=0x7fdec800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.678+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec800ddd0 con 0x7fded01096e0 2024-03-21T09:10:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.679+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdec8004020 con 0x7fded01096e0 2024-03-21T09:10:10.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.679+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded019b3a0 con 0x7fded01096e0 2024-03-21T09:10:10.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.679+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8015660 con 0x7fded01096e0 2024-03-21T09:10:10.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.679+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded01a7660 con 0x7fded01096e0 2024-03-21T09:10:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.682+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fded0062920 con 0x7fded01096e0 2024-03-21T09:10:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.683+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fdec80041d0 con 0x7fded01096e0 2024-03-21T09:10:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.685+0000 7fdec7e00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdeb40a30c0 0x7fdeb40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.685+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdec80cf1f0 con 0x7fded01096e0 2024-03-21T09:10:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.685+0000 7fdecf400700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdeb40a30c0 0x7fdeb40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.685+0000 7fdec7e00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] conn(0x7fdeb40ae220 0x7fdeb40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.685+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 --> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdec8021e50 con 0x7fdeb40ae220 2024-03-21T09:10:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.686+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdec80d6050 con 0x7fded01096e0 2024-03-21T09:10:10.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.686+0000 7fdecea00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] conn(0x7fdeb40ae220 0x7fdeb40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:10.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.686+0000 7fdecf400700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdeb40a30c0 0x7fdeb40a5580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b580 tx=0x7fdec0009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:10.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.687+0000 7fdecea00700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] conn(0x7fdeb40ae220 0x7fdeb40b06c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:10.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.687+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== osd.11 v2:172.21.15.71:6824/3952075855 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdec8021e50 con 0x7fdeb40ae220 2024-03-21T09:10:10.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.706+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 --> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fded0062da0 con 0x7fdeb40ae220 2024-03-21T09:10:10.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.706+0000 7fdec7e00700 1 -- 172.21.15.37:0/1169304375 <== osd.11 v2:172.21.15.71:6824/3952075855 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fded0062da0 con 0x7fdeb40ae220 2024-03-21T09:10:10.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] conn(0x7fdeb40ae220 msgr2=0x7fdeb40b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:10.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] conn(0x7fdeb40ae220 0x7fdeb40b06c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdeb40a30c0 msgr2=0x7fdeb40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:10.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdeb40a30c0 0x7fdeb40a5580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b580 tx=0x7fdec0009510 comp rx=0 tx=0).stop 2024-03-21T09:10:10.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded01096e0 msgr2=0x7fded019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:10.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 --2- 172.21.15.37:0/1169304375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fded01096e0 0x7fded019a950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdec8007ee0 tx=0x7fdec800d3b0 comp rx=0 tx=0).stop 2024-03-21T09:10:10.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fdecfe00700 1 -- 172.21.15.37:0/1169304375 reap_dead start 2024-03-21T09:10:10.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 shutdown_connections 2024-03-21T09:10:10.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 >> 172.21.15.37:0/1169304375 conn(0x7fded00ffe90 msgr2=0x7fded0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:10.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 shutdown_connections 2024-03-21T09:10:10.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:10.707+0000 7fded6600700 1 -- 172.21.15.37:0/1169304375 wait complete. 2024-03-21T09:10:11.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:11 smithi037 ceph-mon[31830]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:11.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:11 smithi071 ceph-mon[36597]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:11.617 INFO:teuthology.orchestra.run.smithi037.stdout:347892351008 2024-03-21T09:10:11.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-21T09:10:11.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:11 smithi053 ceph-mon[36169]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:55834574935 2024-03-21T09:10:12.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T09:10:12.213 INFO:teuthology.orchestra.run.smithi037.stdout:287762808874 2024-03-21T09:10:12.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-21T09:10:12.681 INFO:teuthology.orchestra.run.smithi037.stdout:163208757313 2024-03-21T09:10:12.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T09:10:13.231 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:13.242 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:13.255 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:13.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:13 smithi037 ceph-mon[31830]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:13.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:13 smithi071 ceph-mon[36597]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:13.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:13 smithi053 ceph-mon[36169]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:15.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:15 smithi037 ceph-mon[31830]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-21T09:10:15.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 -- 172.21.15.37:0/3073655042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102410 msgr2=0x7f4ccc1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:15.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 --2- 172.21.15.37:0/3073655042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102410 0x7f4ccc1027f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009990 tx=0x7f4cbc009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 -- 172.21.15.37:0/3073655042 shutdown_connections 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 --2- 172.21.15.37:0/3073655042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1074f0 0x7f4ccc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 --2- 172.21.15.37:0/3073655042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102410 0x7f4ccc1027f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 --2- 172.21.15.37:0/3073655042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc102dc0 0x7f4ccc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 -- 172.21.15.37:0/3073655042 >> 172.21.15.37:0/3073655042 conn(0x7f4ccc0fdc60 msgr2=0x7f4ccc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 -- 172.21.15.37:0/3073655042 shutdown_connections 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.598+0000 7f4cd2000700 1 -- 172.21.15.37:0/3073655042 wait complete. 2024-03-21T09:10:15.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.600+0000 7f4cd2000700 1 Processor -- start 2024-03-21T09:10:15.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.600+0000 7f4cd2000700 1 -- start start 2024-03-21T09:10:15.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4cd2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102410 0x7f4ccc1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4cd2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102dc0 0x7f4ccc1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4cd2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc1074f0 0x7f4ccc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4cd2000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc1124d0 con 0x7f4ccc102410 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4cd2000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc112350 con 0x7f4ccc102dc0 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4cd2000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc112650 con 0x7f4ccc1074f0 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4ccb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102410 0x7f4ccc1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4ccbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc1074f0 0x7f4ccc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4ccb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102410 0x7f4ccc1a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39838/0 (socket says 172.21.15.37:39838) 2024-03-21T09:10:15.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4ccb400700 1 -- 172.21.15.37:0/1751461345 learned_addr learned my addr 172.21.15.37:0/1751461345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.601+0000 7f4ccaa00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102dc0 0x7f4ccc1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccbe00700 1 -- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102dc0 msgr2=0x7f4ccc1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccbe00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102dc0 0x7f4ccc1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccbe00700 1 -- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102410 msgr2=0x7f4ccc1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccbe00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102410 0x7f4ccc1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccbe00700 1 -- 172.21.15.37:0/1751461345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc19b210 con 0x7f4ccc1074f0 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccaa00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ccc102dc0 0x7f4ccc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:15.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccb400700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102410 0x7f4ccc1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4ccbe00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc1074f0 0x7f4ccc19aab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400efc0 tx=0x7f4cc400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40092e0 con 0x7f4ccc1074f0 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cc4004bb0 con 0x7f4ccc1074f0 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.602+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc19b4a0 con 0x7f4ccc1074f0 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.603+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc1a76d0 con 0x7f4ccc1074f0 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.603+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4004280 con 0x7f4ccc1074f0 2024-03-21T09:10:15.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.603+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cac001030 con 0x7f4ccc1074f0 2024-03-21T09:10:15.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.606+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f4cc401b030 con 0x7f4ccc1074f0 2024-03-21T09:10:15.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.607+0000 7f4cc3e00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4cb00a30c0 0x7f4cb00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:15.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.607+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4cc40cfbe0 con 0x7f4ccc1074f0 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.607+0000 7f4cc3e00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] conn(0x7f4cb00ae220 0x7f4cb00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.607+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 --> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cc40048a0 con 0x7f4cb00ae220 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.607+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc40cff90 con 0x7f4ccc1074f0 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.607+0000 7f4ccb400700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4cb00a30c0 0x7f4cb00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.608+0000 7f4ccaa00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] conn(0x7f4cb00ae220 0x7f4cb00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.608+0000 7f4ccaa00700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] conn(0x7f4cb00ae220 0x7f4cb00b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:15.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.608+0000 7f4ccb400700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4cb00a30c0 0x7f4cb00a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009750 tx=0x7f4cbc005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:15 smithi071 ceph-mon[36597]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-21T09:10:15.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.609+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== osd.7 v2:172.21.15.53:6816/884972742 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4cc40048a0 con 0x7f4cb00ae220 2024-03-21T09:10:15.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.643+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 --> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4cac002e60 con 0x7f4cb00ae220 2024-03-21T09:10:15.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.644+0000 7f4cc3e00700 1 -- 172.21.15.37:0/1751461345 <== osd.7 v2:172.21.15.53:6816/884972742 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4cac002e60 con 0x7f4cb00ae220 2024-03-21T09:10:15.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] conn(0x7f4cb00ae220 msgr2=0x7f4cb00b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] conn(0x7f4cb00ae220 0x7f4cb00b06c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4cb00a30c0 msgr2=0x7f4cb00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f4cb00a30c0 0x7f4cb00a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009750 tx=0x7f4cbc005e80 comp rx=0 tx=0).stop 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc1074f0 msgr2=0x7f4ccc19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 --2- 172.21.15.37:0/1751461345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc1074f0 0x7f4ccc19aab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400efc0 tx=0x7f4cc400c3a0 comp rx=0 tx=0).stop 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4ccbe00700 1 -- 172.21.15.37:0/1751461345 reap_dead start 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 shutdown_connections 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 >> 172.21.15.37:0/1751461345 conn(0x7f4ccc0fdc60 msgr2=0x7f4ccc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:15.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 shutdown_connections 2024-03-21T09:10:15.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:15.645+0000 7f4cd2000700 1 -- 172.21.15.37:0/1751461345 wait complete. 2024-03-21T09:10:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:15 smithi053 ceph-mon[36169]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-21T09:10:17.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:17 smithi037 ceph-mon[31830]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:17.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:17 smithi071 ceph-mon[36597]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:17.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:17 smithi053 ceph-mon[36169]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 -- 172.21.15.37:0/2825081233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd590074ce0 msgr2=0x7fd590112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 --2- 172.21.15.37:0/2825081233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd590074ce0 0x7fd590112e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009990 tx=0x7fd57c009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 -- 172.21.15.37:0/2825081233 shutdown_connections 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 --2- 172.21.15.37:0/2825081233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 0x7fd590115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 --2- 172.21.15.37:0/2825081233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5900743c0 0x7fd5900747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 --2- 172.21.15.37:0/2825081233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd590074ce0 0x7fd590112e20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 -- 172.21.15.37:0/2825081233 >> 172.21.15.37:0/2825081233 conn(0x7fd5900792f0 msgr2=0x7fd590079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.460+0000 7fd595400700 1 -- 172.21.15.37:0/2825081233 shutdown_connections 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.461+0000 7fd595400700 1 -- 172.21.15.37:0/2825081233 wait complete. 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.462+0000 7fd595400700 1 Processor -- start 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.462+0000 7fd595400700 1 -- start start 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd595400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5900743c0 0x7fd59019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd595400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 0x7fd59019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd595400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 0x7fd5901a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd595400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590118ba0 con 0x7fd590113360 2024-03-21T09:10:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd595400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590118a20 con 0x7fd5900743c0 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd595400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590118d20 con 0x7fd590074ce0 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5900743c0 0x7fd59019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 0x7fd59019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58e800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5900743c0 0x7fd59019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:48758/0 (socket says 172.21.15.37:48758) 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 0x7fd59019f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33062/0 (socket says 172.21.15.37:33062) 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58e800700 1 -- 172.21.15.37:0/3767705952 learned_addr learned my addr 172.21.15.37:0/3767705952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58f200700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 0x7fd5901a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 -- 172.21.15.37:0/3767705952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5900743c0 msgr2=0x7fd59019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5900743c0 0x7fd59019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 -- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 msgr2=0x7fd5901a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 0x7fd5901a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58de00700 1 -- 172.21.15.37:0/3767705952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5901a4020 con 0x7fd590074ce0 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.463+0000 7fd58f200700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 0x7fd5901a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.464+0000 7fd58de00700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 0x7fd59019f550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0098e0 tx=0x7fd57c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.464+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c027070 con 0x7fd590074ce0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.464+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57c00fde0 con 0x7fd590074ce0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.464+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901a42b0 con 0x7fd590074ce0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.464+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0173f0 con 0x7fd590074ce0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.464+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901abb90 con 0x7fd590074ce0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.466+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590068e50 con 0x7fd590074ce0 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stdout:313532612647 2024-03-21T09:10:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.467+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd57c02a070 con 0x7fd590074ce0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.469+0000 7fd587400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a30c0 0x7fd5740a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.469+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd57c0d06a0 con 0x7fd590074ce0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.469+0000 7fd58e800700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a30c0 0x7fd5740a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.470+0000 7fd58e800700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a30c0 0x7fd5740a5580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd580009d20 tx=0x7fd580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.477+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd57c05e740 con 0x7fd590074ce0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.742+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd5901abd60 con 0x7fd590074ce0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.743+0000 7fd587400700 1 -- 172.21.15.37:0/3767705952 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd57c013070 con 0x7fd590074ce0 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a30c0 msgr2=0x7fd5740a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a30c0 0x7fd5740a5580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd580009d20 tx=0x7fd580009260 comp rx=0 tx=0).stop 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 msgr2=0x7fd59019f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 0x7fd59019f550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0098e0 tx=0x7fd57c00b840 comp rx=0 tx=0).stop 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 shutdown_connections 2024-03-21T09:10:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd590113360 0x7fd5901a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5740a30c0 0x7fd5740a5580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5900743c0 0x7fd59019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 --2- 172.21.15.37:0/3767705952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd590074ce0 0x7fd59019f550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 >> 172.21.15.37:0/3767705952 conn(0x7fd5900792f0 msgr2=0x7fd590112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 shutdown_connections 2024-03-21T09:10:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:18.746+0000 7fd595400700 1 -- 172.21.15.37:0/3767705952 wait complete. 2024-03-21T09:10:19.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:19 smithi037 ceph-mon[31830]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:19.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:19 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3767705952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T09:10:19.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:19 smithi071 ceph-mon[36597]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:19.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:19 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3767705952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T09:10:19.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:19 smithi053 ceph-mon[36169]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:19.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:19 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3767705952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T09:10:21.244 INFO:teuthology.orchestra.run.smithi037.stdout:219043332151 2024-03-21T09:10:21.245 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T09:10:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:21 smithi071 ceph-mon[36597]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:21.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:21 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3755459654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T09:10:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:21 smithi037 ceph-mon[31830]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:21 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3755459654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T09:10:21.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:21 smithi053 ceph-mon[36169]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:21.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:21 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3755459654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T09:10:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:22.156 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 -- 172.21.15.37:0/2074294368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b006a930 msgr2=0x7f50b006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 --2- 172.21.15.37:0/2074294368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b006a930 0x7f50b006ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f50a00098f0 tx=0x7f50a0009c00 comp rx=0 tx=0).stop 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 -- 172.21.15.37:0/2074294368 shutdown_connections 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 --2- 172.21.15.37:0/2074294368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b0107510 0x7f50b010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 --2- 172.21.15.37:0/2074294368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b006a930 0x7f50b006ad10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 --2- 172.21.15.37:0/2074294368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 -- 172.21.15.37:0/2074294368 >> 172.21.15.37:0/2074294368 conn(0x7f50b00770e0 msgr2=0x7f50b00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 -- 172.21.15.37:0/2074294368 shutdown_connections 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.922+0000 7f50b5600700 1 -- 172.21.15.37:0/2074294368 wait complete. 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 Processor -- start 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 -- start start 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b006a930 0x7f50b019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b0107510 0x7f50b0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b01124f0 con 0x7f50b006a930 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0112370 con 0x7f50b0107510 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.924+0000 7f50b5600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0112670 con 0x7f50b006b2e0 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50aea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b006a930 0x7f50b019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50af400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b0107510 0x7f50b0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33088/0 (socket says 172.21.15.37:33088) 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50aea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b006a930 0x7f50b019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39868/0 (socket says 172.21.15.37:39868) 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 -- 172.21.15.37:0/3755459654 learned_addr learned my addr 172.21.15.37:0/3755459654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 -- 172.21.15.37:0/3755459654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b0107510 msgr2=0x7f50b0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b0107510 0x7f50b0198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 -- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b006a930 msgr2=0x7f50b019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b006a930 0x7f50b019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 -- 172.21.15.37:0/3755459654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0199070 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.925+0000 7f50ae000700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b019ec00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f509c00e9f0 tx=0x7f509c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.926+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c004a30 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.926+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f509c00cca0 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.926+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b0199360 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.926+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c005510 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.926+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b01a5520 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.927+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b010a6e0 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.929+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f509c004bd0 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.931+0000 7f50a7400700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f50940a30c0 0x7f50940a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.931+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f509c0cfa30 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.931+0000 7f50aea00700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f50940a30c0 0x7f50940a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.932+0000 7f50aea00700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f50940a30c0 0x7f50940a5580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50a00098c0 tx=0x7f50a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:20.934+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f509c09c340 con 0x7f50b006b2e0 2024-03-21T09:10:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.208+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f50b0199d70 con 0x7f50b006b2e0 2024-03-21T09:10:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.208+0000 7f50a7400700 1 -- 172.21.15.37:0/3755459654 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f509c09bc40 con 0x7f50b006b2e0 2024-03-21T09:10:23.408 INFO:teuthology.orchestra.run.smithi037.stdout:287762808876 2024-03-21T09:10:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.213+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f50940a30c0 msgr2=0x7f50940a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.213+0000 7f50b5600700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f50940a30c0 0x7f50940a5580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50a00098c0 tx=0x7f50a001a040 comp rx=0 tx=0).stop 2024-03-21T09:10:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.213+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 msgr2=0x7f50b019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.213+0000 7f50b5600700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b019ec00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f509c00e9f0 tx=0x7f509c00ed00 comp rx=0 tx=0).stop 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 shutdown_connections 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f50940a30c0 0x7f50940a5580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50b0107510 0x7f50b0198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50b006a930 0x7f50b019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 --2- 172.21.15.37:0/3755459654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b006b2e0 0x7f50b019ec00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 >> 172.21.15.37:0/3755459654 conn(0x7f50b00770e0 msgr2=0x7f50b00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 shutdown_connections 2024-03-21T09:10:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:21.214+0000 7f50b5600700 1 -- 172.21.15.37:0/3755459654 wait complete. 2024-03-21T09:10:23.564 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:23 smithi037 ceph-mon[31830]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:23.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:23 smithi071 ceph-mon[36597]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:23.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:23 smithi053 ceph-mon[36169]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.064+0000 7f3b81a00700 1 -- 172.21.15.37:0/3582218189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c069e40 msgr2=0x7f3b7c06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.064+0000 7f3b81a00700 1 --2- 172.21.15.37:0/3582218189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c069e40 0x7f3b7c06a2a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b680099c0 tx=0x7f3b68009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 -- 172.21.15.37:0/3582218189 shutdown_connections 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 --2- 172.21.15.37:0/3582218189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c06a7e0 0x7f3b7c10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 --2- 172.21.15.37:0/3582218189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 0x7f3b7c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 --2- 172.21.15.37:0/3582218189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c069e40 0x7f3b7c06a2a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 -- 172.21.15.37:0/3582218189 >> 172.21.15.37:0/3582218189 conn(0x7f3b7c077fc0 msgr2=0x7f3b7c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 -- 172.21.15.37:0/3582218189 shutdown_connections 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.065+0000 7f3b81a00700 1 -- 172.21.15.37:0/3582218189 wait complete. 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.066+0000 7f3b81a00700 1 Processor -- start 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 -- start start 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 0x7f3b7c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c06a7e0 0x7f3b7c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 0x7f3b7c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c112490 con 0x7f3b7c1081e0 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c112310 con 0x7f3b7c069e40 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b81a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c112610 con 0x7f3b7c06a7e0 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b7aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 0x7f3b7c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.067+0000 7f3b7aa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 0x7f3b7c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:48876/0 (socket says 172.21.15.37:48876) 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 -- 172.21.15.37:0/1797181983 learned_addr learned my addr 172.21.15.37:0/1797181983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c06a7e0 0x7f3b7c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7b400700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 0x7f3b7c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 -- 172.21.15.37:0/1797181983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c06a7e0 msgr2=0x7f3b7c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c06a7e0 0x7f3b7c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 -- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 msgr2=0x7f3b7c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 0x7f3b7c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 -- 172.21.15.37:0/1797181983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c198fd0 con 0x7f3b7c069e40 2024-03-21T09:10:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7b400700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 0x7f3b7c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b7aa00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 0x7f3b7c19e670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e9a0 tx=0x7f3b6c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.068+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00cbd0 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.069+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b6c00cd70 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.069+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c1992c0 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.069+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c018750 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.069+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c1a54d0 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.070+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c103890 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.072+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3b6c010430 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.073+0000 7f3b73400700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600a30c0 0x7f3b600a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.073+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3b6c0cfe40 con 0x7f3b7c069e40 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.074+0000 7f3b7a000700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600a30c0 0x7f3b600a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.074+0000 7f3b7a000700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600a30c0 0x7f3b600a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0786c0 tx=0x7f3b68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.075+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b6c09c750 con 0x7f3b7c069e40 2024-03-21T09:10:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.311+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3b7c02da30 con 0x7f3b7c069e40 2024-03-21T09:10:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.312+0000 7f3b73400700 1 -- 172.21.15.37:0/1797181983 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b6c09c050 con 0x7f3b7c069e40 2024-03-21T09:10:25.307 INFO:teuthology.orchestra.run.smithi037.stdout:107374182478 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600a30c0 msgr2=0x7f3b600a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600a30c0 0x7f3b600a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0786c0 tx=0x7f3b68005fb0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 msgr2=0x7f3b7c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 0x7f3b7c19e670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e9a0 tx=0x7f3b6c00ecb0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 shutdown_connections 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7c1081e0 0x7f3b7c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3b600a30c0 0x7f3b600a5580 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c069e40 0x7f3b7c19e670 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 --2- 172.21.15.37:0/1797181983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b7c06a7e0 0x7f3b7c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 >> 172.21.15.37:0/1797181983 conn(0x7f3b7c077fc0 msgr2=0x7f3b7c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 shutdown_connections 2024-03-21T09:10:25.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:25.315+0000 7f3b81a00700 1 -- 172.21.15.37:0/1797181983 wait complete. 2024-03-21T09:10:25.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:25 smithi071 ceph-mon[36597]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:25.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:25 smithi037 ceph-mon[31830]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:25.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:25 smithi053 ceph-mon[36169]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:25.999 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612643 got 313532612647 for osd.10 2024-03-21T09:10:25.999 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:26.122 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808874 got 287762808876 for osd.9 2024-03-21T09:10:26.122 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 -- 172.21.15.37:0/2123837587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 msgr2=0x7f0520112b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 --2- 172.21.15.37:0/2123837587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f0520112b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f050c009990 tx=0x7f050c009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 -- 172.21.15.37:0/2123837587 shutdown_connections 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 --2- 172.21.15.37:0/2123837587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520113080 0x7f0520115470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 --2- 172.21.15.37:0/2123837587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f0520074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 --2- 172.21.15.37:0/2123837587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f0520112b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.378+0000 7f0526c00700 1 -- 172.21.15.37:0/2123837587 >> 172.21.15.37:0/2123837587 conn(0x7f05200fdb60 msgr2=0x7f05200fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.379+0000 7f0526c00700 1 -- 172.21.15.37:0/2123837587 shutdown_connections 2024-03-21T09:10:26.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.379+0000 7f0526c00700 1 -- 172.21.15.37:0/2123837587 wait complete. 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 Processor -- start 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 -- start start 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f052019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f052019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520113080 0x7f05201a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520118a00 con 0x7f0520113080 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520118880 con 0x7f0520074530 2024-03-21T09:10:26.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.380+0000 7f0526c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520118b80 con 0x7f0520074e50 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0525800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f052019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f052019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0525800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f052019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:48884/0 (socket says 172.21.15.37:48884) 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0525800700 1 -- 172.21.15.37:0/1310921145 learned_addr learned my addr 172.21.15.37:0/1310921145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0526200700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520113080 0x7f05201a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 -- 172.21.15.37:0/1310921145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 msgr2=0x7f052019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f052019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 -- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520113080 msgr2=0x7f05201a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520113080 0x7f05201a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 -- 172.21.15.37:0/1310921145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05201a3ec0 con 0x7f0520074e50 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0525800700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f052019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.381+0000 7f0524e00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f052019f3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c00 tx=0x7f050c00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.382+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c003890 con 0x7f0520074e50 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.382+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f050c005030 con 0x7f0520074e50 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.382+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c004b40 con 0x7f0520074e50 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.382+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05201a4150 con 0x7f0520074e50 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.382+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05201a4580 con 0x7f0520074e50 2024-03-21T09:10:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.383+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520068e50 con 0x7f0520074e50 2024-03-21T09:10:26.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.385+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f050c009d10 con 0x7f0520074e50 2024-03-21T09:10:26.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.386+0000 7f051e000700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a30c0 0x7f05080a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.386+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f050c0d0490 con 0x7f0520074e50 2024-03-21T09:10:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.386+0000 7f0525800700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a30c0 0x7f05080a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:26.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.387+0000 7f0525800700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a30c0 0x7f05080a5580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0514011590 tx=0x7f0514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:26.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.390+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f050c013070 con 0x7f0520074e50 2024-03-21T09:10:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:26 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1797181983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T09:10:26.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:26 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1797181983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T09:10:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.632+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f052002da30 con 0x7f0520074e50 2024-03-21T09:10:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.632+0000 7f051e000700 1 -- 172.21.15.37:0/1310921145 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f050c09ce30 con 0x7f0520074e50 2024-03-21T09:10:26.628 INFO:teuthology.orchestra.run.smithi037.stdout:197568495678 2024-03-21T09:10:26.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a30c0 msgr2=0x7f05080a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:26.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a30c0 0x7f05080a5580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0514011590 tx=0x7f0514009260 comp rx=0 tx=0).stop 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 msgr2=0x7f052019f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f052019f3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c00 tx=0x7f050c00bab0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 shutdown_connections 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0520113080 0x7f05201a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f05080a30c0 0x7f05080a5580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0520074530 0x7f052019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 --2- 172.21.15.37:0/1310921145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0520074e50 0x7f052019f3f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.635+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 >> 172.21.15.37:0/1310921145 conn(0x7f05200fdb60 msgr2=0x7f05200fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.636+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 shutdown_connections 2024-03-21T09:10:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:26.636+0000 7f0526c00700 1 -- 172.21.15.37:0/1310921145 wait complete. 2024-03-21T09:10:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:26 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1797181983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T09:10:27.185 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182474 got 107374182478 for osd.3 2024-03-21T09:10:27.185 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:27.404 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:27 smithi037 ceph-mon[31830]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:27.404 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:27 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1310921145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T09:10:27.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:27 smithi071 ceph-mon[36597]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:27.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:27 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1310921145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T09:10:27.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:27 smithi053 ceph-mon[36169]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:27.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:27 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1310921145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T09:10:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.151+0000 7f0804a00700 1 -- 172.21.15.37:0/2668079815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 msgr2=0x7f08001085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.151+0000 7f0804a00700 1 --2- 172.21.15.37:0/2668079815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 0x7f08001085c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098f0 tx=0x7f07f0009c00 comp rx=0 tx=0).stop 2024-03-21T09:10:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 -- 172.21.15.37:0/2668079815 shutdown_connections 2024-03-21T09:10:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 --2- 172.21.15.37:0/2668079815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 0x7f080010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 --2- 172.21.15.37:0/2668079815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 0x7f08001085c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 --2- 172.21.15.37:0/2668079815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f080006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 -- 172.21.15.37:0/2668079815 >> 172.21.15.37:0/2668079815 conn(0x7f0800077fc0 msgr2=0x7f08000783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 -- 172.21.15.37:0/2668079815 shutdown_connections 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.152+0000 7f0804a00700 1 -- 172.21.15.37:0/2668079815 wait complete. 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.153+0000 7f0804a00700 1 Processor -- start 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.153+0000 7f0804a00700 1 -- start start 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.153+0000 7f0804a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f0800101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.153+0000 7f0804a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 0x7f08001016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f0804a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 0x7f0800101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f0804a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800112470 con 0x7f080006a7e0 2024-03-21T09:10:28.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f0804a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001122f0 con 0x7f08001081e0 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f0804a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001125f0 con 0x7f0800069e40 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f0800101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f0800101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55308/0 (socket says 172.21.15.37:55308) 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 0x7f08001016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 -- 172.21.15.37:0/2665484567 learned_addr learned my addr 172.21.15.37:0/2665484567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fe800700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 0x7f0800101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 -- 172.21.15.37:0/2665484567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 msgr2=0x7f0800101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 0x7f0800101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 -- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 msgr2=0x7f08001016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 0x7f08001016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fde00700 1 -- 172.21.15.37:0/2665484567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08001a6e90 con 0x7f0800069e40 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.154+0000 7f07fd400700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 0x7f08001016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.155+0000 7f07fde00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f0800101160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009fd0 tx=0x7f07f00094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:28.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.155+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0025070 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.155+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07f000ba40 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.155+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08001a7120 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.155+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004c50 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.155+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08001a75d0 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.156+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0005330 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.158+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f07f0028070 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.159+0000 7f07f6a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f07e40a30c0 0x7f07e40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.159+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f07f00cff60 con 0x7f0800069e40 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.160+0000 7f07fd400700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f07e40a30c0 0x7f07e40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.160+0000 7f07fd400700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f07e40a30c0 0x7f07e40a5580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009750 tx=0x7f07ec009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.163+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07f005c740 con 0x7f0800069e40 2024-03-21T09:10:28.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.431+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f07e00058f0 con 0x7f0800069e40 2024-03-21T09:10:28.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.431+0000 7f07f6a00700 1 -- 172.21.15.37:0/2665484567 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f07f0013030 con 0x7f0800069e40 2024-03-21T09:10:28.427 INFO:teuthology.orchestra.run.smithi037.stdout:347892351011 2024-03-21T09:10:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.434+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f07e40a30c0 msgr2=0x7f07e40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.434+0000 7f0804a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f07e40a30c0 0x7f07e40a5580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009750 tx=0x7f07ec009310 comp rx=0 tx=0).stop 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.434+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 msgr2=0x7f0800101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.434+0000 7f0804a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f0800101160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009fd0 tx=0x7f07f00094b0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 shutdown_connections 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f07e40a30c0 0x7f07e40a5580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08001081e0 0x7f0800101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0800069e40 0x7f0800101160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 --2- 172.21.15.37:0/2665484567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080006a7e0 0x7f08001016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 >> 172.21.15.37:0/2665484567 conn(0x7f0800077fc0 msgr2=0x7f08000ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 shutdown_connections 2024-03-21T09:10:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:28.435+0000 7f0804a00700 1 -- 172.21.15.37:0/2665484567 wait complete. 2024-03-21T09:10:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:29.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 -- 172.21.15.37:0/243113979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 msgr2=0x7f062410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 --2- 172.21.15.37:0/243113979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f062410f020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b0a0 tx=0x7f061c00b3b0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 -- 172.21.15.37:0/243113979 shutdown_connections 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 --2- 172.21.15.37:0/243113979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f062410f020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 --2- 172.21.15.37:0/243113979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06241081e0 0x7f06241085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 --2- 172.21.15.37:0/243113979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0624069e40 0x7f062406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 -- 172.21.15.37:0/243113979 >> 172.21.15.37:0/243113979 conn(0x7f0624077fc0 msgr2=0x7f06240783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.454+0000 7f062a400700 1 -- 172.21.15.37:0/243113979 shutdown_connections 2024-03-21T09:10:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.455+0000 7f062a400700 1 -- 172.21.15.37:0/243113979 wait complete. 2024-03-21T09:10:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 Processor -- start 2024-03-21T09:10:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 -- start start 2024-03-21T09:10:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624069e40 0x7f06241a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f06241a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06241081e0 0x7f062419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06241124b0 con 0x7f0624069e40 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0624112330 con 0x7f062406a7e0 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.456+0000 7f062a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0624112630 con 0x7f06241081e0 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0622a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f06241a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06241081e0 0x7f062419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0622a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f06241a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:48916/0 (socket says 172.21.15.37:48916) 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0622a00700 1 -- 172.21.15.37:0/934891735 learned_addr learned my addr 172.21.15.37:0/934891735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:29.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624069e40 0x7f06241a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 -- 172.21.15.37:0/934891735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 msgr2=0x7f06241a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f06241a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 -- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624069e40 msgr2=0x7f06241a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624069e40 0x7f06241a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 -- 172.21.15.37:0/934891735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062419b160 con 0x7f06241081e0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0622a00700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f06241a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624069e40 0x7f06241a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.457+0000 7f0623e00700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06241081e0 0x7f062419aa00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f061c0062a0 tx=0x7f061c004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.458+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c021070 con 0x7f06241081e0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.458+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f061c003c00 con 0x7f06241081e0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.458+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0047b0 con 0x7f06241081e0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.458+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062419b3f0 con 0x7f06241081e0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.458+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06241a76c0 con 0x7f06241081e0 2024-03-21T09:10:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.459+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0624103890 con 0x7f06241081e0 2024-03-21T09:10:29.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.462+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f061c007980 con 0x7f06241081e0 2024-03-21T09:10:29.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.464+0000 7f061be00700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06080a30c0 0x7f06080a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.464+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f061c00f020 con 0x7f06241081e0 2024-03-21T09:10:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.464+0000 7f0623400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06080a30c0 0x7f06080a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.465+0000 7f0623400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06080a30c0 0x7f06080a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0614005e10 tx=0x7f0614005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:29.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.469+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f061c02a020 con 0x7f06241081e0 2024-03-21T09:10:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:29 smithi037 ceph-mon[31830]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:29.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:29 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2665484567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T09:10:29.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:29 smithi071 ceph-mon[36597]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:29.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:29 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2665484567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T09:10:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:29 smithi053 ceph-mon[36169]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:29.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:29 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2665484567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T09:10:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.736+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f06241a7890 con 0x7f06241081e0 2024-03-21T09:10:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.737+0000 7f061be00700 1 -- 172.21.15.37:0/934891735 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f061c09d5c0 con 0x7f06241081e0 2024-03-21T09:10:29.732 INFO:teuthology.orchestra.run.smithi037.stdout:77309411415 2024-03-21T09:10:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06080a30c0 msgr2=0x7f06080a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06080a30c0 0x7f06080a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0614005e10 tx=0x7f0614005d60 comp rx=0 tx=0).stop 2024-03-21T09:10:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06241081e0 msgr2=0x7f062419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06241081e0 0x7f062419aa00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f061c0062a0 tx=0x7f061c004cf0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 shutdown_connections 2024-03-21T09:10:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06241081e0 0x7f062419aa00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624069e40 0x7f06241a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06080a30c0 0x7f06080a5580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 --2- 172.21.15.37:0/934891735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062406a7e0 0x7f06241a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:29.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.740+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 >> 172.21.15.37:0/934891735 conn(0x7f0624077fc0 msgr2=0x7f06240ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:29.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.741+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 shutdown_connections 2024-03-21T09:10:29.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:29.741+0000 7f062a400700 1 -- 172.21.15.37:0/934891735 wait complete. 2024-03-21T09:10:30.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:30 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/934891735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T09:10:30.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:30 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:30.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:30 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:30 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/934891735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T09:10:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:30 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:30 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:30.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:30 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/934891735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T09:10:30.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:30 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:30.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:30 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:31.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:31 smithi071 ceph-mon[36597]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:31.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:31 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:10:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:31 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:10:31.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:31 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:31.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:31 smithi037 ceph-mon[31830]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:31.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:31 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:10:31.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:31 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:10:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:31 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:31 smithi053 ceph-mon[36169]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:31 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:10:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:31 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:10:31.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:31 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:10:31.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351008 got 347892351011 for osd.11 2024-03-21T09:10:31.842 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:32.386 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495673 got 197568495678 for osd.6 2024-03-21T09:10:32.386 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:32.862 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:33.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411415 for osd.2 2024-03-21T09:10:33.033 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:33.538 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:33 smithi037 ceph-mon[31830]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:33.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:33 smithi071 ceph-mon[36597]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:33.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:33 smithi053 ceph-mon[36169]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 -- 172.21.15.37:0/1418987175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 msgr2=0x7f73881027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 --2- 172.21.15.37:0/1418987175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f73881027f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7378009990 tx=0x7f7378009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 -- 172.21.15.37:0/1418987175 shutdown_connections 2024-03-21T09:10:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 --2- 172.21.15.37:0/1418987175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73881074f0 0x7f738810f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 --2- 172.21.15.37:0/1418987175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f73881027f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 --2- 172.21.15.37:0/1418987175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7388102dc0 0x7f7388106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 -- 172.21.15.37:0/1418987175 >> 172.21.15.37:0/1418987175 conn(0x7f73880fdc60 msgr2=0x7f7388100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 -- 172.21.15.37:0/1418987175 shutdown_connections 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.806+0000 7f738d600700 1 -- 172.21.15.37:0/1418987175 wait complete. 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.807+0000 7f738d600700 1 Processor -- start 2024-03-21T09:10:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 -- start start 2024-03-21T09:10:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f7388074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388102dc0 0x7f73880749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73881074f0 0x7f7388074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7388112520 con 0x7f7388102dc0 2024-03-21T09:10:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881123a0 con 0x7f7388102410 2024-03-21T09:10:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f738d600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881126a0 con 0x7f73881074f0 2024-03-21T09:10:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f7386a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f7388074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.808+0000 7f7386a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f7388074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:48934/0 (socket says 172.21.15.37:48934) 2024-03-21T09:10:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 -- 172.21.15.37:0/3314989085 learned_addr learned my addr 172.21.15.37:0/3314989085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386000700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388102dc0 0x7f73880749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 -- 172.21.15.37:0/3314989085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73881074f0 msgr2=0x7f7388074f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73881074f0 0x7f7388074f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 -- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388102dc0 msgr2=0x7f73880749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388102dc0 0x7f73880749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 -- 172.21.15.37:0/3314989085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73881a6f00 con 0x7f7388102410 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.809+0000 7f7386a00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f7388074490 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7378005910 tx=0x7f73780047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.810+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378027070 con 0x7f7388102410 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.810+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7378022470 con 0x7f7388102410 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.810+0000 7f738d600700 1 -- 172.21.15.37:0/3314989085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73881a7190 con 0x7f7388102410 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.810+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378017650 con 0x7f7388102410 2024-03-21T09:10:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.810+0000 7f738d600700 1 -- 172.21.15.37:0/3314989085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73881a7640 con 0x7f7388102410 2024-03-21T09:10:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.812+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7360005330 con 0x7f7388102410 2024-03-21T09:10:33.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.813+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f737802a070 con 0x7f7388102410 2024-03-21T09:10:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.814+0000 7f737f400700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f736c0a30c0 0x7f736c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.814+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f73780d0590 con 0x7f7388102410 2024-03-21T09:10:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.814+0000 7f7386000700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f736c0a30c0 0x7f736c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.815+0000 7f7386000700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f736c0a30c0 0x7f736c0a5580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7374009680 tx=0x7f7374009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:33.818+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7378013070 con 0x7f7388102410 2024-03-21T09:10:34.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.086+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73600058f0 con 0x7f7388102410 2024-03-21T09:10:34.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.086+0000 7f737f400700 1 -- 172.21.15.37:0/3314989085 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f737809cf30 con 0x7f7388102410 2024-03-21T09:10:34.081 INFO:teuthology.orchestra.run.smithi037.stdout:55834574940 2024-03-21T09:10:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f736c0a30c0 msgr2=0x7f736c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f736c0a30c0 0x7f736c0a5580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7374009680 tx=0x7f7374009260 comp rx=0 tx=0).stop 2024-03-21T09:10:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 msgr2=0x7f7388074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f7388074490 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7378005910 tx=0x7f73780047b0 comp rx=0 tx=0).stop 2024-03-21T09:10:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 shutdown_connections 2024-03-21T09:10:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73881074f0 0x7f7388074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:34.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f736c0a30c0 0x7f736c0a5580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:34.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7388102410 0x7f7388074490 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:34.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 --2- 172.21.15.37:0/3314989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388102dc0 0x7f73880749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:34.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.089+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 >> 172.21.15.37:0/3314989085 conn(0x7f73880fdc60 msgr2=0x7f73880fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:34.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.090+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 shutdown_connections 2024-03-21T09:10:34.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:34.090+0000 7f737cc00700 1 -- 172.21.15.37:0/3314989085 wait complete. 2024-03-21T09:10:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:34 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3314989085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T09:10:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:34 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3314989085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T09:10:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:34 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3314989085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T09:10:35.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:35 smithi037 ceph-mon[31830]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:35.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:35 smithi071 ceph-mon[36597]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:35.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:35 smithi053 ceph-mon[36169]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:36.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574935 got 55834574940 for osd.1 2024-03-21T09:10:36.139 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:37.586 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:37 smithi037 ceph-mon[31830]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:37.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:37 smithi071 ceph-mon[36597]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.627+0000 7f2849a00700 1 -- 172.21.15.37:0/2583979363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2844109190 msgr2=0x7f2844109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.627+0000 7f2849a00700 1 --2- 172.21.15.37:0/2583979363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2844109190 0x7f2844109570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2834009990 tx=0x7f2834009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.627+0000 7f2849a00700 1 -- 172.21.15.37:0/2583979363 shutdown_connections 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.627+0000 7f2849a00700 1 --2- 172.21.15.37:0/2583979363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28441152e0 0x7f28441176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.627+0000 7f2849a00700 1 --2- 172.21.15.37:0/2583979363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2844109190 0x7f2844109570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.627+0000 7f2849a00700 1 --2- 172.21.15.37:0/2583979363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109ab0 0x7f2844114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.628+0000 7f2849a00700 1 -- 172.21.15.37:0/2583979363 >> 172.21.15.37:0/2583979363 conn(0x7f284406d2d0 msgr2=0x7f284406d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.628+0000 7f2849a00700 1 -- 172.21.15.37:0/2583979363 shutdown_connections 2024-03-21T09:10:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.628+0000 7f2849a00700 1 -- 172.21.15.37:0/2583979363 wait complete. 2024-03-21T09:10:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.629+0000 7f2849a00700 1 Processor -- start 2024-03-21T09:10:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.629+0000 7f2849a00700 1 -- start start 2024-03-21T09:10:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.629+0000 7f2849a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 0x7f284410ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.629+0000 7f2849a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2844109ab0 0x7f284410d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2849a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28441152e0 0x7f284410d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2849a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284411ad70 con 0x7f2844109ab0 2024-03-21T09:10:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2849a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284411abf0 con 0x7f2844109190 2024-03-21T09:10:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2849a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284411aef0 con 0x7f28441152e0 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 0x7f284410ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 0x7f284410ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47810/0 (socket says 172.21.15.37:47810) 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 -- 172.21.15.37:0/1698341497 learned_addr learned my addr 172.21.15.37:0/1698341497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842000700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2844109ab0 0x7f284410d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 -- 172.21.15.37:0/1698341497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28441152e0 msgr2=0x7f284410d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28441152e0 0x7f284410d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 -- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2844109ab0 msgr2=0x7f284410d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2844109ab0 0x7f284410d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842a00700 1 -- 172.21.15.37:0/1698341497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28441b3ae0 con 0x7f2844109190 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.630+0000 7f2842000700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2844109ab0 0x7f284410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:10:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.631+0000 7f2842a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 0x7f284410ce10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2834005f50 tx=0x7f283400bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.631+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834027070 con 0x7f2844109190 2024-03-21T09:10:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.631+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2834003d70 con 0x7f2844109190 2024-03-21T09:10:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.631+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28441b3d70 con 0x7f2844109190 2024-03-21T09:10:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.631+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400f6c0 con 0x7f2844109190 2024-03-21T09:10:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.632+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28441b41c0 con 0x7f2844109190 2024-03-21T09:10:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.633+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284406e830 con 0x7f2844109190 2024-03-21T09:10:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.635+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f283402a070 con 0x7f2844109190 2024-03-21T09:10:37.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.636+0000 7f283b400700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f28280a30c0 0x7f28280a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:37.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.637+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f28340d0ec0 con 0x7f2844109190 2024-03-21T09:10:37.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.637+0000 7f2842000700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f28280a30c0 0x7f28280a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:37.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.638+0000 7f2842000700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f28280a30c0 0x7f28280a5580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f284410e5d0 tx=0x7f2830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:37.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.639+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2834013070 con 0x7f2844109190 2024-03-21T09:10:37.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:37 smithi053 ceph-mon[36169]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:37.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.874+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f28441b45a0 con 0x7f2844109190 2024-03-21T09:10:37.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.874+0000 7f283b400700 1 -- 172.21.15.37:0/1698341497 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f283409d980 con 0x7f2844109190 2024-03-21T09:10:37.869 INFO:teuthology.orchestra.run.smithi037.stdout:163208757318 2024-03-21T09:10:37.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f28280a30c0 msgr2=0x7f28280a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f28280a30c0 0x7f28280a5580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f284410e5d0 tx=0x7f2830009260 comp rx=0 tx=0).stop 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 msgr2=0x7f284410ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 0x7f284410ce10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2834005f50 tx=0x7f283400bb60 comp rx=0 tx=0).stop 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 shutdown_connections 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28441152e0 0x7f284410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f28280a30c0 0x7f28280a5580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2844109190 0x7f284410ce10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 --2- 172.21.15.37:0/1698341497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2844109ab0 0x7f284410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 >> 172.21.15.37:0/1698341497 conn(0x7f284406d2d0 msgr2=0x7f284410aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 shutdown_connections 2024-03-21T09:10:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:37.878+0000 7f2849a00700 1 -- 172.21.15.37:0/1698341497 wait complete. 2024-03-21T09:10:38.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1698341497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T09:10:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1698341497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T09:10:38.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1698341497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T09:10:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 -- 172.21.15.37:0/1594128197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738107510 msgr2=0x7f473810f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 --2- 172.21.15.37:0/1594128197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738107510 0x7f473810f0e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f473000b0d0 tx=0x7f473000b3e0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 -- 172.21.15.37:0/1594128197 shutdown_connections 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 --2- 172.21.15.37:0/1594128197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738107510 0x7f473810f0e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 --2- 172.21.15.37:0/1594128197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738102440 0x7f4738102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 --2- 172.21.15.37:0/1594128197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f4738106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 -- 172.21.15.37:0/1594128197 >> 172.21.15.37:0/1594128197 conn(0x7f47380792f0 msgr2=0x7f4738079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.206+0000 7f473d400700 1 -- 172.21.15.37:0/1594128197 shutdown_connections 2024-03-21T09:10:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.207+0000 7f473d400700 1 -- 172.21.15.37:0/1594128197 wait complete. 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 Processor -- start 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 -- start start 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 0x7f4738074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f47380749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738107510 0x7f4738074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738112580 con 0x7f4738107510 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738112400 con 0x7f4738102df0 2024-03-21T09:10:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.208+0000 7f473d400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738112700 con 0x7f4738102440 2024-03-21T09:10:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4735e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f47380749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 0x7f4738074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4737200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738107510 0x7f4738074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 0x7f4738074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42308/0 (socket says 172.21.15.37:42308) 2024-03-21T09:10:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4735e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f47380749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47832/0 (socket says 172.21.15.37:47832) 2024-03-21T09:10:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 -- 172.21.15.37:0/1294318154 learned_addr learned my addr 172.21.15.37:0/1294318154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 -- 172.21.15.37:0/1294318154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 msgr2=0x7f47380749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f47380749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 -- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738107510 msgr2=0x7f4738074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738107510 0x7f4738074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 -- 172.21.15.37:0/1294318154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47381a6f40 con 0x7f4738102440 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4735e00700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f47380749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4737200700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738107510 0x7f4738074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.209+0000 7f4736800700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 0x7f4738074490 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9a0 tx=0x7f472800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.210+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800ca10 con 0x7f4738102440 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.210+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f472800cbb0 con 0x7f4738102440 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.210+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728018680 con 0x7f4738102440 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.210+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47381a7230 con 0x7f4738102440 2024-03-21T09:10:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.210+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47381a7720 con 0x7f4738102440 2024-03-21T09:10:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.211+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738103910 con 0x7f4738102440 2024-03-21T09:10:39.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.216+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f4728010430 con 0x7f4738102440 2024-03-21T09:10:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.217+0000 7f472f400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f471c0a30c0 0x7f471c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.218+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f47280d06b0 con 0x7f4738102440 2024-03-21T09:10:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.218+0000 7f4735e00700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f471c0a30c0 0x7f471c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.218+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4728026030 con 0x7f4738102440 2024-03-21T09:10:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.218+0000 7f4735e00700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f471c0a30c0 0x7f471c0a5580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4738108950 tx=0x7f4724009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.462+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f47380759c0 con 0x7f4738102440 2024-03-21T09:10:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.462+0000 7f472f400700 1 -- 172.21.15.37:0/1294318154 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f472809cfc0 con 0x7f4738102440 2024-03-21T09:10:39.456 INFO:teuthology.orchestra.run.smithi037.stdout:253403070517 2024-03-21T09:10:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.465+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f471c0a30c0 msgr2=0x7f471c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.465+0000 7f473d400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f471c0a30c0 0x7f471c0a5580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4738108950 tx=0x7f4724009260 comp rx=0 tx=0).stop 2024-03-21T09:10:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.465+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 msgr2=0x7f4738074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.465+0000 7f473d400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 0x7f4738074490 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9a0 tx=0x7f472800ecb0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 shutdown_connections 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738107510 0x7f4738074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4738102440 0x7f4738074490 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f471c0a30c0 0x7f471c0a5580 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 --2- 172.21.15.37:0/1294318154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738102df0 0x7f47380749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 >> 172.21.15.37:0/1294318154 conn(0x7f47380792f0 msgr2=0x7f4738100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 shutdown_connections 2024-03-21T09:10:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.466+0000 7f473d400700 1 -- 172.21.15.37:0/1294318154 wait complete. 2024-03-21T09:10:39.542 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:39 smithi037 ceph-mon[31830]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:39.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:39 smithi053 ceph-mon[36169]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 -- 172.21.15.37:0/2576339342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 msgr2=0x7f0630108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 --2- 172.21.15.37:0/2576339342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 0x7f0630108640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06240099c0 tx=0x7f0624009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 -- 172.21.15.37:0/2576339342 shutdown_connections 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 --2- 172.21.15.37:0/2576339342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f063010f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 --2- 172.21.15.37:0/2576339342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 0x7f0630108640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 --2- 172.21.15.37:0/2576339342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0630102210 0x7f0630102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 -- 172.21.15.37:0/2576339342 >> 172.21.15.37:0/2576339342 conn(0x7f06300792e0 msgr2=0x7f06300796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 -- 172.21.15.37:0/2576339342 shutdown_connections 2024-03-21T09:10:39.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.708+0000 7f0638200700 1 -- 172.21.15.37:0/2576339342 wait complete. 2024-03-21T09:10:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.709+0000 7f0638200700 1 Processor -- start 2024-03-21T09:10:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.709+0000 7f0638200700 1 -- start start 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0638200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0630102210 0x7f06301a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0638200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f06301a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0638200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 0x7f063019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0638200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06301124d0 con 0x7f0630102210 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0638200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630112350 con 0x7f0630108260 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0638200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630112650 con 0x7f0630102bd0 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f06301a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0635800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0630102210 0x7f06301a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f06301a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42332/0 (socket says 172.21.15.37:42332) 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 -- 172.21.15.37:0/133182583 learned_addr learned my addr 172.21.15.37:0/133182583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0636200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 0x7f063019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 -- 172.21.15.37:0/133182583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 msgr2=0x7f063019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 0x7f063019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 -- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0630102210 msgr2=0x7f06301a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0630102210 0x7f06301a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.710+0000 7f0634e00700 1 -- 172.21.15.37:0/133182583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063019b0d0 con 0x7f0630102bd0 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.711+0000 7f0634e00700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f06301a0cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f061c00d980 tx=0x7f061c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.711+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c016070 con 0x7f0630102bd0 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.711+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f061c004d10 con 0x7f0630102bd0 2024-03-21T09:10:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.711+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063019b3c0 con 0x7f0630102bd0 2024-03-21T09:10:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.711+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c005670 con 0x7f0630102bd0 2024-03-21T09:10:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.711+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06301a7680 con 0x7f0630102bd0 2024-03-21T09:10:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.713+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0630068e50 con 0x7f0630102bd0 2024-03-21T09:10:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.714+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f061c004750 con 0x7f0630102bd0 2024-03-21T09:10:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.715+0000 7f062e000700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06180a3040 0x7f06180a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.716+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f061c0cf6f0 con 0x7f0630102bd0 2024-03-21T09:10:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.716+0000 7f0635800700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06180a3040 0x7f06180a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.717+0000 7f0635800700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06180a3040 0x7f06180a5500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0624000c00 tx=0x7f0624005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:39.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.719+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f061c01f020 con 0x7f0630102bd0 2024-03-21T09:10:39.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:39 smithi071 ceph-mon[36597]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:10:39.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 -- 172.21.15.37:0/3988573794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074e80 msgr2=0x7f563010e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 --2- 172.21.15.37:0/3988573794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074e80 0x7f563010e3d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5620009990 tx=0x7f5620009ca0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 -- 172.21.15.37:0/3988573794 shutdown_connections 2024-03-21T09:10:39.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 --2- 172.21.15.37:0/3988573794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563010e910 0x7f5630110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 --2- 172.21.15.37:0/3988573794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074560 0x7f5630074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 --2- 172.21.15.37:0/3988573794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074e80 0x7f563010e3d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 -- 172.21.15.37:0/3988573794 >> 172.21.15.37:0/3988573794 conn(0x7f56300fdb60 msgr2=0x7f56300fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:39.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 -- 172.21.15.37:0/3988573794 shutdown_connections 2024-03-21T09:10:39.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.922+0000 7f5637200700 1 -- 172.21.15.37:0/3988573794 wait complete. 2024-03-21T09:10:39.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.923+0000 7f5637200700 1 Processor -- start 2024-03-21T09:10:39.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 -- start start 2024-03-21T09:10:39.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074560 0x7f56301a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 0x7f56301a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563010e910 0x7f563019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630114610 con 0x7f563010e910 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630114490 con 0x7f5630074560 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5637200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630114790 con 0x7f5630074e80 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5635400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 0x7f56301a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5635e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074560 0x7f56301a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5635400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 0x7f56301a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42356/0 (socket says 172.21.15.37:42356) 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5635400700 1 -- 172.21.15.37:0/671916681 learned_addr learned my addr 172.21.15.37:0/671916681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.924+0000 7f5636800700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563010e910 0x7f563019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635400700 1 -- 172.21.15.37:0/671916681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074560 msgr2=0x7f56301a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635400700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074560 0x7f56301a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635400700 1 -- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563010e910 msgr2=0x7f563019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635400700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563010e910 0x7f563019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635400700 1 -- 172.21.15.37:0/671916681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563019b050 con 0x7f5630074e80 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635e00700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074560 0x7f56301a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.925+0000 7f5635400700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 0x7f56301a0c70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5620000c00 tx=0x7f562000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.926+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620027070 con 0x7f5630074e80 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.926+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5620003d70 con 0x7f5630074e80 2024-03-21T09:10:39.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.926+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563019b2e0 con 0x7f5630074e80 2024-03-21T09:10:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.926+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562000f460 con 0x7f5630074e80 2024-03-21T09:10:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.926+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56301a7600 con 0x7f5630074e80 2024-03-21T09:10:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.928+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5630068e50 con 0x7f5630074e80 2024-03-21T09:10:39.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.929+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f562002a070 con 0x7f5630074e80 2024-03-21T09:10:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.931+0000 7f5626a00700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f561c0a30c0 0x7f561c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.932+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f56200d0720 con 0x7f5630074e80 2024-03-21T09:10:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.932+0000 7f5635e00700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f561c0a30c0 0x7f561c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.933+0000 7f5635e00700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f561c0a30c0 0x7f561c0a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f563019ba30 tx=0x7f562800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:39.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:39.934+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f562005e740 con 0x7f5630074e80 2024-03-21T09:10:40.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.018+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06301a78b0 con 0x7f0630102bd0 2024-03-21T09:10:40.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.019+0000 7f062e000700 1 -- 172.21.15.37:0/133182583 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f061c022350 con 0x7f0630102bd0 2024-03-21T09:10:40.013 INFO:teuthology.orchestra.run.smithi037.stdout:34359738465 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06180a3040 msgr2=0x7f06180a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06180a3040 0x7f06180a5500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0624000c00 tx=0x7f0624005fb0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 msgr2=0x7f06301a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f06301a0cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f061c00d980 tx=0x7f061c00dc90 comp rx=0 tx=0).stop 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 shutdown_connections 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06180a3040 0x7f06180a5500 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0630108260 0x7f063019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0630102210 0x7f06301a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 --2- 172.21.15.37:0/133182583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0630102bd0 0x7f06301a0cf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 >> 172.21.15.37:0/133182583 conn(0x7f06300792e0 msgr2=0x7f063010d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:40.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 shutdown_connections 2024-03-21T09:10:40.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.022+0000 7f0638200700 1 -- 172.21.15.37:0/133182583 wait complete. 2024-03-21T09:10:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.212+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f563002da30 con 0x7f5630074e80 2024-03-21T09:10:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.212+0000 7f5626a00700 1 -- 172.21.15.37:0/671916681 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5620013070 con 0x7f5630074e80 2024-03-21T09:10:40.207 INFO:teuthology.orchestra.run.smithi037.stdout:141733920845 2024-03-21T09:10:40.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f561c0a30c0 msgr2=0x7f561c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:40.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f561c0a30c0 0x7f561c0a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f563019ba30 tx=0x7f562800b040 comp rx=0 tx=0).stop 2024-03-21T09:10:40.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 msgr2=0x7f56301a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:40.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 0x7f56301a0c70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5620000c00 tx=0x7f562000ba90 comp rx=0 tx=0).stop 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 shutdown_connections 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563010e910 0x7f563019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f561c0a30c0 0x7f561c0a5580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5630074560 0x7f56301a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 --2- 172.21.15.37:0/671916681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5630074e80 0x7f56301a0c70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 >> 172.21.15.37:0/671916681 conn(0x7f56300fdb60 msgr2=0x7f56300fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.215+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 shutdown_connections 2024-03-21T09:10:40.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:40.216+0000 7f5637200700 1 -- 172.21.15.37:0/671916681 wait complete. 2024-03-21T09:10:40.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:40 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1294318154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T09:10:40.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:40 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/133182583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T09:10:40.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:40 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/671916681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T09:10:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:40 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1294318154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T09:10:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:40 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/133182583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T09:10:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:40 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/671916681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T09:10:40.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:40 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1294318154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T09:10:40.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:40 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/133182583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T09:10:40.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:40 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/671916681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T09:10:41.110 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920835 got 141733920845 for osd.4 2024-03-21T09:10:41.110 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:41.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:41 smithi037 ceph-mon[31830]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:41.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:41 smithi053 ceph-mon[36169]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:41.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:41 smithi071 ceph-mon[36597]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:10:41.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:41.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070509 got 253403070517 for osd.8 2024-03-21T09:10:41.952 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:42.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757313 got 163208757318 for osd.5 2024-03-21T09:10:42.591 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:42.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738465 for osd.0 2024-03-21T09:10:42.939 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:43.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:43 smithi037 ceph-mon[31830]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:43.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:43 smithi053 ceph-mon[36169]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:43.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:43 smithi071 ceph-mon[36597]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 -- 172.21.15.37:0/780703255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1778109160 msgr2=0x7f1778115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 --2- 172.21.15.37:0/780703255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1778109160 0x7f1778115670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f177000b0a0 tx=0x7f177000b3b0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 -- 172.21.15.37:0/780703255 shutdown_connections 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 --2- 172.21.15.37:0/780703255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1778109160 0x7f1778115670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 --2- 172.21.15.37:0/780703255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f177810e7a0 0x7f177810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 --2- 172.21.15.37:0/780703255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 0x7f1778108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.304+0000 7f177ec00700 1 -- 172.21.15.37:0/780703255 >> 172.21.15.37:0/780703255 conn(0x7f1778075960 msgr2=0x7f1778077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.305+0000 7f177ec00700 1 -- 172.21.15.37:0/780703255 shutdown_connections 2024-03-21T09:10:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.305+0000 7f177ec00700 1 -- 172.21.15.37:0/780703255 wait complete. 2024-03-21T09:10:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 Processor -- start 2024-03-21T09:10:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 -- start start 2024-03-21T09:10:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 0x7f177819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778109160 0x7f177819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f177810e7a0 0x7f17781a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1778118b40 con 0x7f17781087a0 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17781189c0 con 0x7f177810e7a0 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.306+0000 7f177ec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1778118cc0 con 0x7f1778109160 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 0x7f177819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778109160 0x7f177819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 0x7f177819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40348/0 (socket says 172.21.15.37:40348) 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777e00700 1 -- 172.21.15.37:0/4293580573 learned_addr learned my addr 172.21.15.37:0/4293580573 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f177cc00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f177810e7a0 0x7f17781a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777400700 1 -- 172.21.15.37:0/4293580573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f177810e7a0 msgr2=0x7f17781a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777400700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f177810e7a0 0x7f17781a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777400700 1 -- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 msgr2=0x7f177819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777400700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 0x7f177819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.307+0000 7f1777400700 1 -- 172.21.15.37:0/4293580573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17781a3ec0 con 0x7f1778109160 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.308+0000 7f1777400700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778109160 0x7f177819f3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f176400e9f0 tx=0x7f176400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.308+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400c9c0 con 0x7f1778109160 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.308+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f176400cb60 con 0x7f1778109160 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.308+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764018680 con 0x7f1778109160 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.308+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17781a41b0 con 0x7f1778109160 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.308+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17781aba30 con 0x7f1778109160 2024-03-21T09:10:45.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.309+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1778068e50 con 0x7f1778109160 2024-03-21T09:10:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.312+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f1764010430 con 0x7f1778109160 2024-03-21T09:10:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.314+0000 7f1774c00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f175c0a3040 0x7f175c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.314+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f17640cfbf0 con 0x7f1778109160 2024-03-21T09:10:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.315+0000 7f1777e00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f175c0a3040 0x7f175c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.315+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f176409c500 con 0x7f1778109160 2024-03-21T09:10:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.315+0000 7f1777e00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f175c0a3040 0x7f175c0a5500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1768009ce0 tx=0x7f1768009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:45.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.629+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f17781abc60 con 0x7f1778109160 2024-03-21T09:10:45.624 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:45 smithi037 ceph-mon[31830]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.630+0000 7f1774c00700 1 -- 172.21.15.37:0/4293580573 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f176409be00 con 0x7f1778109160 2024-03-21T09:10:45.624 INFO:teuthology.orchestra.run.smithi037.stdout:219043332156 2024-03-21T09:10:45.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f175c0a3040 msgr2=0x7f175c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:45.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f175c0a3040 0x7f175c0a5500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1768009ce0 tx=0x7f1768009260 comp rx=0 tx=0).stop 2024-03-21T09:10:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778109160 msgr2=0x7f177819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778109160 0x7f177819f3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f176400e9f0 tx=0x7f176400ed00 comp rx=0 tx=0).stop 2024-03-21T09:10:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 shutdown_connections 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f175c0a3040 0x7f175c0a5500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f177810e7a0 0x7f17781a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17781087a0 0x7f177819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 --2- 172.21.15.37:0/4293580573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778109160 0x7f177819f3f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 >> 172.21.15.37:0/4293580573 conn(0x7f1778075960 msgr2=0x7f1778075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 shutdown_connections 2024-03-21T09:10:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:45.635+0000 7f177ec00700 1 -- 172.21.15.37:0/4293580573 wait complete. 2024-03-21T09:10:45.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:45 smithi053 ceph-mon[36169]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:45.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:45 smithi071 ceph-mon[36597]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:10:46.469 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332151 got 219043332156 for osd.7 2024-03-21T09:10:46.470 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:46.470 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T09:10:46.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T09:10:46.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:46 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4293580573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T09:10:46.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-21T09:10:46.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:46 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-21T09:10:46.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:46 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4293580573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T09:10:46.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-21T09:10:46.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:46 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-21T09:10:46.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:46 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4293580573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T09:10:46.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-21T09:10:46.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:46 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-21T09:10:46.926 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:47 smithi037 ceph-mon[31830]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-21T09:10:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:47 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-21T09:10:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:47 smithi037 ceph-mon[31830]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T09:10:47.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:47 smithi037 ceph-mon[31830]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-21T09:10:47.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:47 smithi053 ceph-mon[36169]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:47.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-21T09:10:47.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:47 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-21T09:10:47.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:47 smithi053 ceph-mon[36169]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T09:10:47.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:47 smithi053 ceph-mon[36169]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-21T09:10:47.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:47 smithi071 ceph-mon[36597]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:10:47.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-21T09:10:47.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:47 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-21T09:10:47.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:47 smithi071 ceph-mon[36597]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T09:10:47.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:47 smithi071 ceph-mon[36597]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-21T09:10:48.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:48 smithi037 ceph-mon[31830]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T09:10:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:48 smithi053 ceph-mon[36169]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T09:10:48.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:48 smithi071 ceph-mon[36597]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T09:10:49.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 -- 172.21.15.37:0/386695340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 msgr2=0x7f1864111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 --2- 172.21.15.37:0/386695340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 0x7f1864111290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f185c00b0d0 tx=0x7f185c00b3e0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 -- 172.21.15.37:0/386695340 shutdown_connections 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 --2- 172.21.15.37:0/386695340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 0x7f1864111290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 --2- 172.21.15.37:0/386695340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 --2- 172.21.15.37:0/386695340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864104f90 0x7f1864108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 -- 172.21.15.37:0/386695340 >> 172.21.15.37:0/386695340 conn(0x7f18640ffe70 msgr2=0x7f1864102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 -- 172.21.15.37:0/386695340 shutdown_connections 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.299+0000 7f186a800700 1 -- 172.21.15.37:0/386695340 wait complete. 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.300+0000 7f186a800700 1 Processor -- start 2024-03-21T09:10:49.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.300+0000 7f186a800700 1 -- start start 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.300+0000 7f186a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.300+0000 7f186a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864104f90 0x7f18641a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.300+0000 7f186a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 0x7f186419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.300+0000 7f186a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641146b0 con 0x7f18641096c0 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f186a800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864114530 con 0x7f18641045e0 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f186a800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864114830 con 0x7f1864104f90 2024-03-21T09:10:49.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:51050/0 (socket says 172.21.15.37:51050) 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 -- 172.21.15.37:0/368443823 learned_addr learned my addr 172.21.15.37:0/368443823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1862a00700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864104f90 0x7f18641a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863e00700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 0x7f186419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 -- 172.21.15.37:0/368443823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864104f90 msgr2=0x7f18641a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864104f90 0x7f18641a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 -- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 msgr2=0x7f186419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 0x7f186419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 -- 172.21.15.37:0/368443823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186419aff0 con 0x7f18641045e0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.301+0000 7f1863400700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641a06d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f185400e9a0 tx=0x7f185400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.302+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185400cbd0 con 0x7f18641045e0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.302+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186419b2e0 con 0x7f18641045e0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.302+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f185400cd70 con 0x7f18641045e0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.302+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854018750 con 0x7f18641045e0 2024-03-21T09:10:49.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.302+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641a75a0 con 0x7f18641045e0 2024-03-21T09:10:49.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.303+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1864068e50 con 0x7f18641045e0 2024-03-21T09:10:49.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.305+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1854010430 con 0x7f18641045e0 2024-03-21T09:10:49.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.306+0000 7f185be00700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f18480a2fb0 0x7f18480a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:49.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.306+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18540cfa70 con 0x7f18641045e0 2024-03-21T09:10:49.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.306+0000 7f1862a00700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f18480a2fb0 0x7f18480a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:49.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.307+0000 7f1862a00700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f18480a2fb0 0x7f18480a5470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f186410ab60 tx=0x7f1850009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.310+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f185409c4b0 con 0x7f18641045e0 2024-03-21T09:10:49.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.522+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1864100fc0 con 0x7f18480a2fb0 2024-03-21T09:10:49.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:49 smithi037 ceph-mon[31830]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-03-21T09:10:49.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.552+0000 7f185be00700 1 -- 172.21.15.37:0/368443823 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340757 (secure 0 0 0) 0x7f1864100fc0 con 0x7f18480a2fb0 2024-03-21T09:10:49.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:10:49.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.559+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f18480a2fb0 msgr2=0x7f18480a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:49.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.559+0000 7f186a800700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f18480a2fb0 0x7f18480a5470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f186410ab60 tx=0x7f1850009260 comp rx=0 tx=0).stop 2024-03-21T09:10:49.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.559+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 msgr2=0x7f18641a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641a06d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f185400e9a0 tx=0x7f185400ecb0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 shutdown_connections 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18641096c0 0x7f186419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f18480a2fb0 0x7f18480a5470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641045e0 0x7f18641a06d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 --2- 172.21.15.37:0/368443823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864104f90 0x7f18641a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.560+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 >> 172.21.15.37:0/368443823 conn(0x7f18640ffe70 msgr2=0x7f18641008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.561+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 shutdown_connections 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:49.561+0000 7f186a800700 1 -- 172.21.15.37:0/368443823 wait complete. 2024-03-21T09:10:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:10:49.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:49 smithi053 ceph-mon[36169]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-03-21T09:10:49.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:49 smithi071 ceph-mon[36597]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-03-21T09:10:50.381 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-03-21T09:10:47.821750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":675,"num_read_kb":460,"num_write":347,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":418,"ondisk_log_size":418,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378164,"kb_used_data":12304,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563660,"statfs":{"total":1151940427776,"available":1151553187840,"internally_reserved":0,"allocated":12599296,"data_stored":8045546,"data_compressed":364800,"data_compressed_allocated":495616,"data_compressed_original":2490368,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":26,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003503"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197358+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:09:47.197358+0000","last_peered":"2024-03-21T09:09:47.197358+0000","last_clean":"2024-03-21T09:09:47.197358+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:09:47.197358+0000","last_undegraded":"2024-03-21T09:09:47.197358+0000","last_fullsized":"2024-03-21T09:09:47.197358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.430032+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:09:47.430032+0000","last_peered":"2024-03-21T09:09:47.430032+0000","last_clean":"2024-03-21T09:09:47.430032+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:09:47.430032+0000","last_undegraded":"2024-03-21T09:09:47.430032+0000","last_fullsized":"2024-03-21T09:09:47.430032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.429639+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:09:47.429639+0000","last_peered":"2024-03-21T09:09:47.429639+0000","last_clean":"2024-03-21T09:09:47.429639+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:09:47.429639+0000","last_undegraded":"2024-03-21T09:09:47.429639+0000","last_fullsized":"2024-03-21T09:09:47.429639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.204585+0000","last_change":"2024-03-21T09:09:47.203174+0000","last_active":"2024-03-21T09:09:47.204585+0000","last_peered":"2024-03-21T09:09:47.204585+0000","last_clean":"2024-03-21T09:09:47.204585+0000","last_became_active":"2024-03-21T09:09:47.202519+0000","last_became_peered":"2024-03-21T09:09:47.202519+0000","last_unstale":"2024-03-21T09:09:47.204585+0000","last_undegraded":"2024-03-21T09:09:47.204585+0000","last_fullsized":"2024-03-21T09:09:47.204585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:48.445311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194015+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:09:47.194015+0000","last_peered":"2024-03-21T09:09:47.194015+0000","last_clean":"2024-03-21T09:09:47.194015+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:09:47.194015+0000","last_undegraded":"2024-03-21T09:09:47.194015+0000","last_fullsized":"2024-03-21T09:09:47.194015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.417919+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:10:46.417919+0000","last_peered":"2024-03-21T09:10:46.417919+0000","last_clean":"2024-03-21T09:10:46.417919+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:10:46.417919+0000","last_undegraded":"2024-03-21T09:10:46.417919+0000","last_fullsized":"2024-03-21T09:10:46.417919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197019+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:09:47.197019+0000","last_peered":"2024-03-21T09:09:47.197019+0000","last_clean":"2024-03-21T09:09:47.197019+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:09:47.197019+0000","last_undegraded":"2024-03-21T09:09:47.197019+0000","last_fullsized":"2024-03-21T09:09:47.197019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.207152+0000","last_change":"2024-03-21T09:09:47.204487+0000","last_active":"2024-03-21T09:09:47.207152+0000","last_peered":"2024-03-21T09:09:47.207152+0000","last_clean":"2024-03-21T09:09:47.207152+0000","last_became_active":"2024-03-21T09:09:47.202961+0000","last_became_peered":"2024-03-21T09:09:47.202961+0000","last_unstale":"2024-03-21T09:09:47.207152+0000","last_undegraded":"2024-03-21T09:09:47.207152+0000","last_fullsized":"2024-03-21T09:09:47.207152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:50.524550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:45.797169+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:10:45.797169+0000","last_peered":"2024-03-21T09:10:45.797169+0000","last_clean":"2024-03-21T09:10:45.797169+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:10:45.797169+0000","last_undegraded":"2024-03-21T09:10:45.797169+0000","last_fullsized":"2024-03-21T09:10:45.797169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418323+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:10:46.418323+0000","last_peered":"2024-03-21T09:10:46.418323+0000","last_clean":"2024-03-21T09:10:46.418323+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:10:46.418323+0000","last_undegraded":"2024-03-21T09:10:46.418323+0000","last_fullsized":"2024-03-21T09:10:46.418323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197406+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:09:47.197406+0000","last_peered":"2024-03-21T09:09:47.197406+0000","last_clean":"2024-03-21T09:09:47.197406+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:09:47.197406+0000","last_undegraded":"2024-03-21T09:09:47.197406+0000","last_fullsized":"2024-03-21T09:09:47.197406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194848+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:09:47.194848+0000","last_peered":"2024-03-21T09:09:47.194848+0000","last_clean":"2024-03-21T09:09:47.194848+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:09:47.194848+0000","last_undegraded":"2024-03-21T09:09:47.194848+0000","last_fullsized":"2024-03-21T09:09:47.194848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.431987+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:09:47.431987+0000","last_peered":"2024-03-21T09:09:47.431987+0000","last_clean":"2024-03-21T09:09:47.431987+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:09:47.431987+0000","last_undegraded":"2024-03-21T09:09:47.431987+0000","last_fullsized":"2024-03-21T09:09:47.431987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.429120+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:09:47.429120+0000","last_peered":"2024-03-21T09:09:47.429120+0000","last_clean":"2024-03-21T09:09:47.429120+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:09:47.429120+0000","last_undegraded":"2024-03-21T09:09:47.429120+0000","last_fullsized":"2024-03-21T09:09:47.429120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.430849+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:09:47.430849+0000","last_peered":"2024-03-21T09:09:47.430849+0000","last_clean":"2024-03-21T09:09:47.430849+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:09:47.430849+0000","last_undegraded":"2024-03-21T09:09:47.430849+0000","last_fullsized":"2024-03-21T09:09:47.430849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:45.797545+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:10:45.797545+0000","last_peered":"2024-03-21T09:10:45.797545+0000","last_clean":"2024-03-21T09:10:45.797545+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:10:45.797545+0000","last_undegraded":"2024-03-21T09:10:45.797545+0000","last_fullsized":"2024-03-21T09:10:45.797545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197120+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:09:47.197120+0000","last_peered":"2024-03-21T09:09:47.197120+0000","last_clean":"2024-03-21T09:09:47.197120+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:09:47.197120+0000","last_undegraded":"2024-03-21T09:09:47.197120+0000","last_fullsized":"2024-03-21T09:09:47.197120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197686+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:09:47.197686+0000","last_peered":"2024-03-21T09:09:47.197686+0000","last_clean":"2024-03-21T09:09:47.197686+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:09:47.197686+0000","last_undegraded":"2024-03-21T09:09:47.197686+0000","last_fullsized":"2024-03-21T09:09:47.197686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199258+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:09:47.199258+0000","last_peered":"2024-03-21T09:09:47.199258+0000","last_clean":"2024-03-21T09:09:47.199258+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:09:47.199258+0000","last_undegraded":"2024-03-21T09:09:47.199258+0000","last_fullsized":"2024-03-21T09:09:47.199258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199625+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:09:47.199625+0000","last_peered":"2024-03-21T09:09:47.199625+0000","last_clean":"2024-03-21T09:09:47.199625+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:09:47.199625+0000","last_undegraded":"2024-03-21T09:09:47.199625+0000","last_fullsized":"2024-03-21T09:09:47.199625+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.200921+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:09:47.200921+0000","last_peered":"2024-03-21T09:09:47.200921+0000","last_clean":"2024-03-21T09:09:47.200921+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:09:47.200921+0000","last_undegraded":"2024-03-21T09:09:47.200921+0000","last_fullsized":"2024-03-21T09:09:47.200921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.407124+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:10:46.407124+0000","last_peered":"2024-03-21T09:10:46.407124+0000","last_clean":"2024-03-21T09:10:46.407124+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:10:46.407124+0000","last_undegraded":"2024-03-21T09:10:46.407124+0000","last_fullsized":"2024-03-21T09:10:46.407124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194305+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:09:47.194305+0000","last_peered":"2024-03-21T09:09:47.194305+0000","last_clean":"2024-03-21T09:09:47.194305+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:09:47.194305+0000","last_undegraded":"2024-03-21T09:09:47.194305+0000","last_fullsized":"2024-03-21T09:09:47.194305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.407421+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:10:46.407421+0000","last_peered":"2024-03-21T09:10:46.407421+0000","last_clean":"2024-03-21T09:10:46.407421+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:10:46.407421+0000","last_undegraded":"2024-03-21T09:10:46.407421+0000","last_fullsized":"2024-03-21T09:10:46.407421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.431086+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:09:47.431086+0000","last_peered":"2024-03-21T09:09:47.431086+0000","last_clean":"2024-03-21T09:09:47.431086+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:09:47.431086+0000","last_undegraded":"2024-03-21T09:09:47.431086+0000","last_fullsized":"2024-03-21T09:09:47.431086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194518+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:09:47.194518+0000","last_peered":"2024-03-21T09:09:47.194518+0000","last_clean":"2024-03-21T09:09:47.194518+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:09:47.194518+0000","last_undegraded":"2024-03-21T09:09:47.194518+0000","last_fullsized":"2024-03-21T09:09:47.194518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194304+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:09:47.194304+0000","last_peered":"2024-03-21T09:09:47.194304+0000","last_clean":"2024-03-21T09:09:47.194304+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:09:47.194304+0000","last_undegraded":"2024-03-21T09:09:47.194304+0000","last_fullsized":"2024-03-21T09:09:47.194304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:45.808654+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:10:45.808654+0000","last_peered":"2024-03-21T09:10:45.808654+0000","last_clean":"2024-03-21T09:10:45.808654+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:10:45.808654+0000","last_undegraded":"2024-03-21T09:10:45.808654+0000","last_fullsized":"2024-03-21T09:10:45.808654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194372+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:09:47.194372+0000","last_peered":"2024-03-21T09:09:47.194372+0000","last_clean":"2024-03-21T09:09:47.194372+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:09:47.194372+0000","last_undegraded":"2024-03-21T09:09:47.194372+0000","last_fullsized":"2024-03-21T09:09:47.194372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194136+0000","last_change":"2024-03-21T09:08:45.818384+0000","last_active":"2024-03-21T09:09:47.194136+0000","last_peered":"2024-03-21T09:09:47.194136+0000","last_clean":"2024-03-21T09:09:47.194136+0000","last_became_active":"2024-03-21T09:08:45.818028+0000","last_became_peered":"2024-03-21T09:08:45.818028+0000","last_unstale":"2024-03-21T09:09:47.194136+0000","last_undegraded":"2024-03-21T09:09:47.194136+0000","last_fullsized":"2024-03-21T09:09:47.194136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:04.433350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.433837+0000","last_change":"2024-03-21T09:09:47.433837+0000","last_active":"2024-03-21T09:09:47.433837+0000","last_peered":"2024-03-21T09:09:47.433837+0000","last_clean":"2024-03-21T09:09:47.433837+0000","last_became_active":"2024-03-21T09:09:47.433538+0000","last_became_peered":"2024-03-21T09:09:47.433538+0000","last_unstale":"2024-03-21T09:09:47.433837+0000","last_undegraded":"2024-03-21T09:09:47.433837+0000","last_fullsized":"2024-03-21T09:09:47.433837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:41.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199024+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:09:47.199024+0000","last_peered":"2024-03-21T09:09:47.199024+0000","last_clean":"2024-03-21T09:09:47.199024+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:09:47.199024+0000","last_undegraded":"2024-03-21T09:09:47.199024+0000","last_fullsized":"2024-03-21T09:09:47.199024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":117,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:41.890611+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:10:41.890611+0000","last_peered":"2024-03-21T09:10:41.890611+0000","last_clean":"2024-03-21T09:10:41.890611+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:10:41.890611+0000","last_undegraded":"2024-03-21T09:10:41.890611+0000","last_fullsized":"2024-03-21T09:10:41.890611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194020+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:09:47.194020+0000","last_peered":"2024-03-21T09:09:47.194020+0000","last_clean":"2024-03-21T09:09:47.194020+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:09:47.194020+0000","last_undegraded":"2024-03-21T09:09:47.194020+0000","last_fullsized":"2024-03-21T09:09:47.194020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.198967+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:09:47.198967+0000","last_peered":"2024-03-21T09:09:47.198967+0000","last_clean":"2024-03-21T09:09:47.198967+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:09:47.198967+0000","last_undegraded":"2024-03-21T09:09:47.198967+0000","last_fullsized":"2024-03-21T09:09:47.198967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.417893+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:10:46.417893+0000","last_peered":"2024-03-21T09:10:46.417893+0000","last_clean":"2024-03-21T09:10:46.417893+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:10:46.417893+0000","last_undegraded":"2024-03-21T09:10:46.417893+0000","last_fullsized":"2024-03-21T09:10:46.417893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.198155+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:09:47.198155+0000","last_peered":"2024-03-21T09:09:47.198155+0000","last_clean":"2024-03-21T09:09:47.198155+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:09:47.198155+0000","last_undegraded":"2024-03-21T09:09:47.198155+0000","last_fullsized":"2024-03-21T09:09:47.198155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.432250+0000","last_change":"2024-03-21T09:09:47.432250+0000","last_active":"2024-03-21T09:09:47.432250+0000","last_peered":"2024-03-21T09:09:47.432250+0000","last_clean":"2024-03-21T09:09:47.432250+0000","last_became_active":"2024-03-21T09:09:47.431784+0000","last_became_peered":"2024-03-21T09:09:47.431784+0000","last_unstale":"2024-03-21T09:09:47.432250+0000","last_undegraded":"2024-03-21T09:09:47.432250+0000","last_fullsized":"2024-03-21T09:09:47.432250+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:58.438472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.431514+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:09:47.431514+0000","last_peered":"2024-03-21T09:09:47.431514+0000","last_clean":"2024-03-21T09:09:47.431514+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:09:47.431514+0000","last_undegraded":"2024-03-21T09:09:47.431514+0000","last_fullsized":"2024-03-21T09:09:47.431514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.404452+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:10:46.404452+0000","last_peered":"2024-03-21T09:10:46.404452+0000","last_clean":"2024-03-21T09:10:46.404452+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:10:46.404452+0000","last_undegraded":"2024-03-21T09:10:46.404452+0000","last_fullsized":"2024-03-21T09:10:46.404452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.429920+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:09:47.429920+0000","last_peered":"2024-03-21T09:09:47.429920+0000","last_clean":"2024-03-21T09:09:47.429920+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:09:47.429920+0000","last_undegraded":"2024-03-21T09:09:47.429920+0000","last_fullsized":"2024-03-21T09:09:47.429920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197854+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:09:47.197854+0000","last_peered":"2024-03-21T09:09:47.197854+0000","last_clean":"2024-03-21T09:09:47.197854+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:09:47.197854+0000","last_undegraded":"2024-03-21T09:09:47.197854+0000","last_fullsized":"2024-03-21T09:09:47.197854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.205219+0000","last_change":"2024-03-21T09:09:47.205219+0000","last_active":"2024-03-21T09:09:47.205219+0000","last_peered":"2024-03-21T09:09:47.205219+0000","last_clean":"2024-03-21T09:09:47.205219+0000","last_became_active":"2024-03-21T09:09:47.204655+0000","last_became_peered":"2024-03-21T09:09:47.204655+0000","last_unstale":"2024-03-21T09:09:47.205219+0000","last_undegraded":"2024-03-21T09:09:47.205219+0000","last_fullsized":"2024-03-21T09:09:47.205219+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:44.407879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.405354+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:10:46.405354+0000","last_peered":"2024-03-21T09:10:46.405354+0000","last_clean":"2024-03-21T09:10:46.405354+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:10:46.405354+0000","last_undegraded":"2024-03-21T09:10:46.405354+0000","last_fullsized":"2024-03-21T09:10:46.405354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418300+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:10:46.418300+0000","last_peered":"2024-03-21T09:10:46.418300+0000","last_clean":"2024-03-21T09:10:46.418300+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:10:46.418300+0000","last_undegraded":"2024-03-21T09:10:46.418300+0000","last_fullsized":"2024-03-21T09:10:46.418300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:45.797038+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:10:45.797038+0000","last_peered":"2024-03-21T09:10:45.797038+0000","last_clean":"2024-03-21T09:10:45.797038+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:10:45.797038+0000","last_undegraded":"2024-03-21T09:10:45.797038+0000","last_fullsized":"2024-03-21T09:10:45.797038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197452+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:09:47.197452+0000","last_peered":"2024-03-21T09:09:47.197452+0000","last_clean":"2024-03-21T09:09:47.197452+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:09:47.197452+0000","last_undegraded":"2024-03-21T09:09:47.197452+0000","last_fullsized":"2024-03-21T09:09:47.197452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197396+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:09:47.197396+0000","last_peered":"2024-03-21T09:09:47.197396+0000","last_clean":"2024-03-21T09:09:47.197396+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:09:47.197396+0000","last_undegraded":"2024-03-21T09:09:47.197396+0000","last_fullsized":"2024-03-21T09:09:47.197396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.431283+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:09:47.431283+0000","last_peered":"2024-03-21T09:09:47.431283+0000","last_clean":"2024-03-21T09:09:47.431283+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:09:47.431283+0000","last_undegraded":"2024-03-21T09:09:47.431283+0000","last_fullsized":"2024-03-21T09:09:47.431283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.283764+0000","last_change":"2024-03-21T09:09:47.283764+0000","last_active":"2024-03-21T09:09:47.283764+0000","last_peered":"2024-03-21T09:09:47.283764+0000","last_clean":"2024-03-21T09:09:47.283764+0000","last_became_active":"2024-03-21T09:09:47.283203+0000","last_became_peered":"2024-03-21T09:09:47.283203+0000","last_unstale":"2024-03-21T09:09:47.283764+0000","last_undegraded":"2024-03-21T09:09:47.283764+0000","last_fullsized":"2024-03-21T09:09:47.283764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:55.502176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197044+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:09:47.197044+0000","last_peered":"2024-03-21T09:09:47.197044+0000","last_clean":"2024-03-21T09:09:47.197044+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:09:47.197044+0000","last_undegraded":"2024-03-21T09:09:47.197044+0000","last_fullsized":"2024-03-21T09:09:47.197044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:45.798026+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:10:45.798026+0000","last_peered":"2024-03-21T09:10:45.798026+0000","last_clean":"2024-03-21T09:10:45.798026+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:10:45.798026+0000","last_undegraded":"2024-03-21T09:10:45.798026+0000","last_fullsized":"2024-03-21T09:10:45.798026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197559+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:09:47.197559+0000","last_peered":"2024-03-21T09:09:47.197559+0000","last_clean":"2024-03-21T09:09:47.197559+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:09:47.197559+0000","last_undegraded":"2024-03-21T09:09:47.197559+0000","last_fullsized":"2024-03-21T09:09:47.197559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":245,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.895799+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:10:46.895799+0000","last_peered":"2024-03-21T09:10:46.895799+0000","last_clean":"2024-03-21T09:10:46.895799+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:10:46.895799+0000","last_undegraded":"2024-03-21T09:10:46.895799+0000","last_fullsized":"2024-03-21T09:10:46.895799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":183,"num_read_kb":152,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194139+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:09:47.194139+0000","last_peered":"2024-03-21T09:09:47.194139+0000","last_clean":"2024-03-21T09:09:47.194139+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:09:47.194139+0000","last_undegraded":"2024-03-21T09:09:47.194139+0000","last_fullsized":"2024-03-21T09:09:47.194139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197414+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:09:47.197414+0000","last_peered":"2024-03-21T09:09:47.197414+0000","last_clean":"2024-03-21T09:09:47.197414+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:09:47.197414+0000","last_undegraded":"2024-03-21T09:09:47.197414+0000","last_fullsized":"2024-03-21T09:09:47.197414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199271+0000","last_change":"2024-03-21T09:08:45.826707+0000","last_active":"2024-03-21T09:09:47.199271+0000","last_peered":"2024-03-21T09:09:47.199271+0000","last_clean":"2024-03-21T09:09:47.199271+0000","last_became_active":"2024-03-21T09:08:45.826364+0000","last_became_peered":"2024-03-21T09:08:45.826364+0000","last_unstale":"2024-03-21T09:09:47.199271+0000","last_undegraded":"2024-03-21T09:09:47.199271+0000","last_fullsized":"2024-03-21T09:09:47.199271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:38:48.129131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.431746+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:09:47.431746+0000","last_peered":"2024-03-21T09:09:47.431746+0000","last_clean":"2024-03-21T09:09:47.431746+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:09:47.431746+0000","last_undegraded":"2024-03-21T09:09:47.431746+0000","last_fullsized":"2024-03-21T09:09:47.431746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196835+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:09:47.196835+0000","last_peered":"2024-03-21T09:09:47.196835+0000","last_clean":"2024-03-21T09:09:47.196835+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:09:47.196835+0000","last_undegraded":"2024-03-21T09:09:47.196835+0000","last_fullsized":"2024-03-21T09:09:47.196835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197496+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:09:47.197496+0000","last_peered":"2024-03-21T09:09:47.197496+0000","last_clean":"2024-03-21T09:09:47.197496+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:09:47.197496+0000","last_undegraded":"2024-03-21T09:09:47.197496+0000","last_fullsized":"2024-03-21T09:09:47.197496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197294+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:09:47.197294+0000","last_peered":"2024-03-21T09:09:47.197294+0000","last_clean":"2024-03-21T09:09:47.197294+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:09:47.197294+0000","last_undegraded":"2024-03-21T09:09:47.197294+0000","last_fullsized":"2024-03-21T09:09:47.197294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.198208+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:09:47.198208+0000","last_peered":"2024-03-21T09:09:47.198208+0000","last_clean":"2024-03-21T09:09:47.198208+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:09:47.198208+0000","last_undegraded":"2024-03-21T09:09:47.198208+0000","last_fullsized":"2024-03-21T09:09:47.198208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":117,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:42.922925+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:10:42.922925+0000","last_peered":"2024-03-21T09:10:42.922925+0000","last_clean":"2024-03-21T09:10:42.922925+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:10:42.922925+0000","last_undegraded":"2024-03-21T09:10:42.922925+0000","last_fullsized":"2024-03-21T09:10:42.922925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196956+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:09:47.196956+0000","last_peered":"2024-03-21T09:09:47.196956+0000","last_clean":"2024-03-21T09:09:47.196956+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:09:47.196956+0000","last_undegraded":"2024-03-21T09:09:47.196956+0000","last_fullsized":"2024-03-21T09:09:47.196956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.432855+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:09:47.432855+0000","last_peered":"2024-03-21T09:09:47.432855+0000","last_clean":"2024-03-21T09:09:47.432855+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:09:47.432855+0000","last_undegraded":"2024-03-21T09:09:47.432855+0000","last_fullsized":"2024-03-21T09:09:47.432855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.193880+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:09:47.193880+0000","last_peered":"2024-03-21T09:09:47.193880+0000","last_clean":"2024-03-21T09:09:47.193880+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:09:47.193880+0000","last_undegraded":"2024-03-21T09:09:47.193880+0000","last_fullsized":"2024-03-21T09:09:47.193880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196766+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:09:47.196766+0000","last_peered":"2024-03-21T09:09:47.196766+0000","last_clean":"2024-03-21T09:09:47.196766+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:09:47.196766+0000","last_undegraded":"2024-03-21T09:09:47.196766+0000","last_fullsized":"2024-03-21T09:09:47.196766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196668+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:09:47.196668+0000","last_peered":"2024-03-21T09:09:47.196668+0000","last_clean":"2024-03-21T09:09:47.196668+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:09:47.196668+0000","last_undegraded":"2024-03-21T09:09:47.196668+0000","last_fullsized":"2024-03-21T09:09:47.196668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194205+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:09:47.194205+0000","last_peered":"2024-03-21T09:09:47.194205+0000","last_clean":"2024-03-21T09:09:47.194205+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:09:47.194205+0000","last_undegraded":"2024-03-21T09:09:47.194205+0000","last_fullsized":"2024-03-21T09:09:47.194205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197315+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:09:47.197315+0000","last_peered":"2024-03-21T09:09:47.197315+0000","last_clean":"2024-03-21T09:09:47.197315+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:09:47.197315+0000","last_undegraded":"2024-03-21T09:09:47.197315+0000","last_fullsized":"2024-03-21T09:09:47.197315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.193983+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:09:47.193983+0000","last_peered":"2024-03-21T09:09:47.193983+0000","last_clean":"2024-03-21T09:09:47.193983+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:09:47.193983+0000","last_undegraded":"2024-03-21T09:09:47.193983+0000","last_fullsized":"2024-03-21T09:09:47.193983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197578+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:09:47.197578+0000","last_peered":"2024-03-21T09:09:47.197578+0000","last_clean":"2024-03-21T09:09:47.197578+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:09:47.197578+0000","last_undegraded":"2024-03-21T09:09:47.197578+0000","last_fullsized":"2024-03-21T09:09:47.197578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418195+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:10:46.418195+0000","last_peered":"2024-03-21T09:10:46.418195+0000","last_clean":"2024-03-21T09:10:46.418195+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:10:46.418195+0000","last_undegraded":"2024-03-21T09:10:46.418195+0000","last_fullsized":"2024-03-21T09:10:46.418195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.432298+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:09:47.432298+0000","last_peered":"2024-03-21T09:09:47.432298+0000","last_clean":"2024-03-21T09:09:47.432298+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:09:47.432298+0000","last_undegraded":"2024-03-21T09:09:47.432298+0000","last_fullsized":"2024-03-21T09:09:47.432298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197084+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:09:47.197084+0000","last_peered":"2024-03-21T09:09:47.197084+0000","last_clean":"2024-03-21T09:09:47.197084+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:09:47.197084+0000","last_undegraded":"2024-03-21T09:09:47.197084+0000","last_fullsized":"2024-03-21T09:09:47.197084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418323+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:10:46.418323+0000","last_peered":"2024-03-21T09:10:46.418323+0000","last_clean":"2024-03-21T09:10:46.418323+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:10:46.418323+0000","last_undegraded":"2024-03-21T09:10:46.418323+0000","last_fullsized":"2024-03-21T09:10:46.418323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.428487+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:09:47.428487+0000","last_peered":"2024-03-21T09:09:47.428487+0000","last_clean":"2024-03-21T09:09:47.428487+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:09:47.428487+0000","last_undegraded":"2024-03-21T09:09:47.428487+0000","last_fullsized":"2024-03-21T09:09:47.428487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.207057+0000","last_change":"2024-03-21T09:09:47.205699+0000","last_active":"2024-03-21T09:09:47.207057+0000","last_peered":"2024-03-21T09:09:47.207057+0000","last_clean":"2024-03-21T09:09:47.207057+0000","last_became_active":"2024-03-21T09:09:47.203657+0000","last_became_peered":"2024-03-21T09:09:47.203657+0000","last_unstale":"2024-03-21T09:09:47.207057+0000","last_undegraded":"2024-03-21T09:09:47.207057+0000","last_fullsized":"2024-03-21T09:09:47.207057+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:42.930153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":114,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:41.920963+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:10:41.920963+0000","last_peered":"2024-03-21T09:10:41.920963+0000","last_clean":"2024-03-21T09:10:41.920963+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:10:41.920963+0000","last_undegraded":"2024-03-21T09:10:41.920963+0000","last_fullsized":"2024-03-21T09:10:41.920963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197324+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:09:47.197324+0000","last_peered":"2024-03-21T09:09:47.197324+0000","last_clean":"2024-03-21T09:09:47.197324+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:09:47.197324+0000","last_undegraded":"2024-03-21T09:09:47.197324+0000","last_fullsized":"2024-03-21T09:09:47.197324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.406616+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:10:46.406616+0000","last_peered":"2024-03-21T09:10:46.406616+0000","last_clean":"2024-03-21T09:10:46.406616+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:10:46.406616+0000","last_undegraded":"2024-03-21T09:10:46.406616+0000","last_fullsized":"2024-03-21T09:10:46.406616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194667+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:09:47.194667+0000","last_peered":"2024-03-21T09:09:47.194667+0000","last_clean":"2024-03-21T09:09:47.194667+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:09:47.194667+0000","last_undegraded":"2024-03-21T09:09:47.194667+0000","last_fullsized":"2024-03-21T09:09:47.194667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196865+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:09:47.196865+0000","last_peered":"2024-03-21T09:09:47.196865+0000","last_clean":"2024-03-21T09:09:47.196865+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:09:47.196865+0000","last_undegraded":"2024-03-21T09:09:47.196865+0000","last_fullsized":"2024-03-21T09:09:47.196865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197206+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:09:47.197206+0000","last_peered":"2024-03-21T09:09:47.197206+0000","last_clean":"2024-03-21T09:09:47.197206+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:09:47.197206+0000","last_undegraded":"2024-03-21T09:09:47.197206+0000","last_fullsized":"2024-03-21T09:09:47.197206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.430446+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:09:47.430446+0000","last_peered":"2024-03-21T09:09:47.430446+0000","last_clean":"2024-03-21T09:09:47.430446+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:09:47.430446+0000","last_undegraded":"2024-03-21T09:09:47.430446+0000","last_fullsized":"2024-03-21T09:09:47.430446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197005+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:09:47.197005+0000","last_peered":"2024-03-21T09:09:47.197005+0000","last_clean":"2024-03-21T09:09:47.197005+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:09:47.197005+0000","last_undegraded":"2024-03-21T09:09:47.197005+0000","last_fullsized":"2024-03-21T09:09:47.197005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196718+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:09:47.196718+0000","last_peered":"2024-03-21T09:09:47.196718+0000","last_clean":"2024-03-21T09:09:47.196718+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:09:47.196718+0000","last_undegraded":"2024-03-21T09:09:47.196718+0000","last_fullsized":"2024-03-21T09:09:47.196718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194183+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:09:47.194183+0000","last_peered":"2024-03-21T09:09:47.194183+0000","last_clean":"2024-03-21T09:09:47.194183+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:09:47.194183+0000","last_undegraded":"2024-03-21T09:09:47.194183+0000","last_fullsized":"2024-03-21T09:09:47.194183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.432477+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:09:47.432477+0000","last_peered":"2024-03-21T09:09:47.432477+0000","last_clean":"2024-03-21T09:09:47.432477+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:09:47.432477+0000","last_undegraded":"2024-03-21T09:09:47.432477+0000","last_fullsized":"2024-03-21T09:09:47.432477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.404302+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:10:46.404302+0000","last_peered":"2024-03-21T09:10:46.404302+0000","last_clean":"2024-03-21T09:10:46.404302+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:10:46.404302+0000","last_undegraded":"2024-03-21T09:10:46.404302+0000","last_fullsized":"2024-03-21T09:10:46.404302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.198147+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:09:47.198147+0000","last_peered":"2024-03-21T09:09:47.198147+0000","last_clean":"2024-03-21T09:09:47.198147+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:09:47.198147+0000","last_undegraded":"2024-03-21T09:09:47.198147+0000","last_fullsized":"2024-03-21T09:09:47.198147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.405533+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:10:46.405533+0000","last_peered":"2024-03-21T09:10:46.405533+0000","last_clean":"2024-03-21T09:10:46.405533+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:10:46.405533+0000","last_undegraded":"2024-03-21T09:10:46.405533+0000","last_fullsized":"2024-03-21T09:10:46.405533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199581+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:09:47.199581+0000","last_peered":"2024-03-21T09:09:47.199581+0000","last_clean":"2024-03-21T09:09:47.199581+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:09:47.199581+0000","last_undegraded":"2024-03-21T09:09:47.199581+0000","last_fullsized":"2024-03-21T09:09:47.199581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194735+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:09:47.194735+0000","last_peered":"2024-03-21T09:09:47.194735+0000","last_clean":"2024-03-21T09:09:47.194735+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:09:47.194735+0000","last_undegraded":"2024-03-21T09:09:47.194735+0000","last_fullsized":"2024-03-21T09:09:47.194735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.430134+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:09:47.430134+0000","last_peered":"2024-03-21T09:09:47.430134+0000","last_clean":"2024-03-21T09:09:47.430134+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:09:47.430134+0000","last_undegraded":"2024-03-21T09:09:47.430134+0000","last_fullsized":"2024-03-21T09:09:47.430134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.203560+0000","last_change":"2024-03-21T09:09:47.203560+0000","last_active":"2024-03-21T09:09:47.203560+0000","last_peered":"2024-03-21T09:09:47.203560+0000","last_clean":"2024-03-21T09:09:47.203560+0000","last_became_active":"2024-03-21T09:09:47.203072+0000","last_became_peered":"2024-03-21T09:09:47.203072+0000","last_unstale":"2024-03-21T09:09:47.203560+0000","last_undegraded":"2024-03-21T09:09:47.203560+0000","last_fullsized":"2024-03-21T09:09:47.203560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:03:39.037042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.282187+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:09:47.282187+0000","last_peered":"2024-03-21T09:09:47.282187+0000","last_clean":"2024-03-21T09:09:47.282187+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:09:47.282187+0000","last_undegraded":"2024-03-21T09:09:47.282187+0000","last_fullsized":"2024-03-21T09:09:47.282187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.193881+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:09:47.193881+0000","last_peered":"2024-03-21T09:09:47.193881+0000","last_clean":"2024-03-21T09:09:47.193881+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:09:47.193881+0000","last_undegraded":"2024-03-21T09:09:47.193881+0000","last_fullsized":"2024-03-21T09:09:47.193881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199142+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:09:47.199142+0000","last_peered":"2024-03-21T09:09:47.199142+0000","last_clean":"2024-03-21T09:09:47.199142+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:09:47.199142+0000","last_undegraded":"2024-03-21T09:09:47.199142+0000","last_fullsized":"2024-03-21T09:09:47.199142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.404714+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:10:46.404714+0000","last_peered":"2024-03-21T09:10:46.404714+0000","last_clean":"2024-03-21T09:10:46.404714+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:10:46.404714+0000","last_undegraded":"2024-03-21T09:10:46.404714+0000","last_fullsized":"2024-03-21T09:10:46.404714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196331+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:09:47.196331+0000","last_peered":"2024-03-21T09:09:47.196331+0000","last_clean":"2024-03-21T09:09:47.196331+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:09:47.196331+0000","last_undegraded":"2024-03-21T09:09:47.196331+0000","last_fullsized":"2024-03-21T09:09:47.196331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.206281+0000","last_change":"2024-03-21T09:09:47.206281+0000","last_active":"2024-03-21T09:09:47.206281+0000","last_peered":"2024-03-21T09:09:47.206281+0000","last_clean":"2024-03-21T09:09:47.206281+0000","last_became_active":"2024-03-21T09:09:47.198427+0000","last_became_peered":"2024-03-21T09:09:47.198427+0000","last_unstale":"2024-03-21T09:09:47.206281+0000","last_undegraded":"2024-03-21T09:09:47.206281+0000","last_fullsized":"2024-03-21T09:09:47.206281+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:32:46.182225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199319+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:09:47.199319+0000","last_peered":"2024-03-21T09:09:47.199319+0000","last_clean":"2024-03-21T09:09:47.199319+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:09:47.199319+0000","last_undegraded":"2024-03-21T09:09:47.199319+0000","last_fullsized":"2024-03-21T09:09:47.199319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.407435+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:10:46.407435+0000","last_peered":"2024-03-21T09:10:46.407435+0000","last_clean":"2024-03-21T09:10:46.407435+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:10:46.407435+0000","last_undegraded":"2024-03-21T09:10:46.407435+0000","last_fullsized":"2024-03-21T09:10:46.407435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.194532+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:09:47.194532+0000","last_peered":"2024-03-21T09:09:47.194532+0000","last_clean":"2024-03-21T09:09:47.194532+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:09:47.194532+0000","last_undegraded":"2024-03-21T09:09:47.194532+0000","last_fullsized":"2024-03-21T09:09:47.194532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.193993+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:09:47.193993+0000","last_peered":"2024-03-21T09:09:47.193993+0000","last_clean":"2024-03-21T09:09:47.193993+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:09:47.193993+0000","last_undegraded":"2024-03-21T09:09:47.193993+0000","last_fullsized":"2024-03-21T09:09:47.193993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197146+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:09:47.197146+0000","last_peered":"2024-03-21T09:09:47.197146+0000","last_clean":"2024-03-21T09:09:47.197146+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:09:47.197146+0000","last_undegraded":"2024-03-21T09:09:47.197146+0000","last_fullsized":"2024-03-21T09:09:47.197146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.282254+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:09:47.282254+0000","last_peered":"2024-03-21T09:09:47.282254+0000","last_clean":"2024-03-21T09:09:47.282254+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:09:47.282254+0000","last_undegraded":"2024-03-21T09:09:47.282254+0000","last_fullsized":"2024-03-21T09:09:47.282254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199175+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:09:47.199175+0000","last_peered":"2024-03-21T09:09:47.199175+0000","last_clean":"2024-03-21T09:09:47.199175+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:09:47.199175+0000","last_undegraded":"2024-03-21T09:09:47.199175+0000","last_fullsized":"2024-03-21T09:09:47.199175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197510+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:09:47.197510+0000","last_peered":"2024-03-21T09:09:47.197510+0000","last_clean":"2024-03-21T09:09:47.197510+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:09:47.197510+0000","last_undegraded":"2024-03-21T09:09:47.197510+0000","last_fullsized":"2024-03-21T09:09:47.197510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.429023+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:09:47.429023+0000","last_peered":"2024-03-21T09:09:47.429023+0000","last_clean":"2024-03-21T09:09:47.429023+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:09:47.429023+0000","last_undegraded":"2024-03-21T09:09:47.429023+0000","last_fullsized":"2024-03-21T09:09:47.429023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197174+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:09:47.197174+0000","last_peered":"2024-03-21T09:09:47.197174+0000","last_clean":"2024-03-21T09:09:47.197174+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:09:47.197174+0000","last_undegraded":"2024-03-21T09:09:47.197174+0000","last_fullsized":"2024-03-21T09:09:47.197174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197198+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:09:47.197198+0000","last_peered":"2024-03-21T09:09:47.197198+0000","last_clean":"2024-03-21T09:09:47.197198+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:09:47.197198+0000","last_undegraded":"2024-03-21T09:09:47.197198+0000","last_fullsized":"2024-03-21T09:09:47.197198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197642+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:09:47.197642+0000","last_peered":"2024-03-21T09:09:47.197642+0000","last_clean":"2024-03-21T09:09:47.197642+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:09:47.197642+0000","last_undegraded":"2024-03-21T09:09:47.197642+0000","last_fullsized":"2024-03-21T09:09:47.197642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.432732+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:09:47.432732+0000","last_peered":"2024-03-21T09:09:47.432732+0000","last_clean":"2024-03-21T09:09:47.432732+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:09:47.432732+0000","last_undegraded":"2024-03-21T09:09:47.432732+0000","last_fullsized":"2024-03-21T09:09:47.432732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197573+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:09:47.197573+0000","last_peered":"2024-03-21T09:09:47.197573+0000","last_clean":"2024-03-21T09:09:47.197573+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:09:47.197573+0000","last_undegraded":"2024-03-21T09:09:47.197573+0000","last_fullsized":"2024-03-21T09:09:47.197573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.196436+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:09:47.196436+0000","last_peered":"2024-03-21T09:09:47.196436+0000","last_clean":"2024-03-21T09:09:47.196436+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:09:47.196436+0000","last_undegraded":"2024-03-21T09:09:47.196436+0000","last_fullsized":"2024-03-21T09:09:47.196436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.200898+0000","last_change":"2024-03-21T09:09:47.200898+0000","last_active":"2024-03-21T09:09:47.200898+0000","last_peered":"2024-03-21T09:09:47.200898+0000","last_clean":"2024-03-21T09:09:47.200898+0000","last_became_active":"2024-03-21T09:09:47.200548+0000","last_became_peered":"2024-03-21T09:09:47.200548+0000","last_unstale":"2024-03-21T09:09:47.200898+0000","last_undegraded":"2024-03-21T09:09:47.200898+0000","last_fullsized":"2024-03-21T09:09:47.200898+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:29.793107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199237+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:09:47.199237+0000","last_peered":"2024-03-21T09:09:47.199237+0000","last_clean":"2024-03-21T09:09:47.199237+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:09:47.199237+0000","last_undegraded":"2024-03-21T09:09:47.199237+0000","last_fullsized":"2024-03-21T09:09:47.199237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.429306+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:09:47.429306+0000","last_peered":"2024-03-21T09:09:47.429306+0000","last_clean":"2024-03-21T09:09:47.429306+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:09:47.429306+0000","last_undegraded":"2024-03-21T09:09:47.429306+0000","last_fullsized":"2024-03-21T09:09:47.429306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.432578+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:09:47.432578+0000","last_peered":"2024-03-21T09:09:47.432578+0000","last_clean":"2024-03-21T09:09:47.432578+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:09:47.432578+0000","last_undegraded":"2024-03-21T09:09:47.432578+0000","last_fullsized":"2024-03-21T09:09:47.432578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197145+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:09:47.197145+0000","last_peered":"2024-03-21T09:09:47.197145+0000","last_clean":"2024-03-21T09:09:47.197145+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:09:47.197145+0000","last_undegraded":"2024-03-21T09:09:47.197145+0000","last_fullsized":"2024-03-21T09:09:47.197145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.199001+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:09:47.199001+0000","last_peered":"2024-03-21T09:09:47.199001+0000","last_clean":"2024-03-21T09:09:47.199001+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:09:47.199001+0000","last_undegraded":"2024-03-21T09:09:47.199001+0000","last_fullsized":"2024-03-21T09:09:47.199001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.408604+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:10:46.408604+0000","last_peered":"2024-03-21T09:10:46.408604+0000","last_clean":"2024-03-21T09:10:46.408604+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:10:46.408604+0000","last_undegraded":"2024-03-21T09:10:46.408604+0000","last_fullsized":"2024-03-21T09:10:46.408604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197871+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:09:47.197871+0000","last_peered":"2024-03-21T09:09:47.197871+0000","last_clean":"2024-03-21T09:09:47.197871+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:09:47.197871+0000","last_undegraded":"2024-03-21T09:09:47.197871+0000","last_fullsized":"2024-03-21T09:09:47.197871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.408147+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:10:46.408147+0000","last_peered":"2024-03-21T09:10:46.408147+0000","last_clean":"2024-03-21T09:10:46.408147+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:10:46.408147+0000","last_undegraded":"2024-03-21T09:10:46.408147+0000","last_fullsized":"2024-03-21T09:10:46.408147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:10:45.812891+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:10:45.812891+0000","last_peered":"2024-03-21T09:10:45.812891+0000","last_clean":"2024-03-21T09:10:45.812891+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:10:45.812891+0000","last_undegraded":"2024-03-21T09:10:45.812891+0000","last_fullsized":"2024-03-21T09:10:45.812891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.431895+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:09:47.431895+0000","last_peered":"2024-03-21T09:09:47.431895+0000","last_clean":"2024-03-21T09:09:47.431895+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:09:47.431895+0000","last_undegraded":"2024-03-21T09:09:47.431895+0000","last_fullsized":"2024-03-21T09:09:47.431895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T09:09:47.197028+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:09:47.197028+0000","last_peered":"2024-03-21T09:09:47.197028+0000","last_clean":"2024-03-21T09:09:47.197028+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:09:47.197028+0000","last_undegraded":"2024-03-21T09:09:47.197028+0000","last_fullsized":"2024-03-21T09:09:47.197028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":539,"num_read_kb":341,"num_write":164,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":198858,"data_compressed":5499,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":507462,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":1171087,"data_compressed":61728,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":507698,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070519,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":507618,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":1103527,"data_compressed":56913,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":502874,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":565105,"data_compressed":22453,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":504262,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":566206,"data_compressed":22453,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":503068,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":507915,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":1098724,"data_compressed":56913,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62321,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63422,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64431,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:10:50.387 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T09:10:50.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:50 smithi037 ceph-mon[31830]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:10:50.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:50 smithi053 ceph-mon[36169]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:10:50.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:50 smithi071 ceph-mon[36597]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:10:50.889 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:51.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:51 smithi037 ceph-mon[31830]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:51.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:51 smithi053 ceph-mon[36169]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:51.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:51 smithi071 ceph-mon[36597]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:53.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 -- 172.21.15.37:0/3983414548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 msgr2=0x7f6ef41085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:53.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 --2- 172.21.15.37:0/3983414548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef41085a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80099c0 tx=0x7f6ee8009cd0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 -- 172.21.15.37:0/3983414548 shutdown_connections 2024-03-21T09:10:53.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 --2- 172.21.15.37:0/3983414548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 0x7f6ef410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 --2- 172.21.15.37:0/3983414548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef41085a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 --2- 172.21.15.37:0/3983414548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 -- 172.21.15.37:0/3983414548 >> 172.21.15.37:0/3983414548 conn(0x7f6ef40792c0 msgr2=0x7f6ef40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:53.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 -- 172.21.15.37:0/3983414548 shutdown_connections 2024-03-21T09:10:53.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.232+0000 7f6efb600700 1 -- 172.21.15.37:0/3983414548 wait complete. 2024-03-21T09:10:53.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.233+0000 7f6efb600700 1 Processor -- start 2024-03-21T09:10:53.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 -- start start 2024-03-21T09:10:53.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:53.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 0x7f6ef40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef4074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41124c0 con 0x7f6ef41081c0 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4112340 con 0x7f6ef41021e0 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6efb600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4112640 con 0x7f6ef4102ba0 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.234+0000 7f6ef8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:51074/0 (socket says 172.21.15.37:51074) 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 -- 172.21.15.37:0/3530125584 learned_addr learned my addr 172.21.15.37:0/3530125584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 0x7f6ef40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef9600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef4074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 -- 172.21.15.37:0/3530125584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 msgr2=0x7f6ef40749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 0x7f6ef40749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 -- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 msgr2=0x7f6ef4074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef4074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 -- 172.21.15.37:0/3530125584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef41a6ea0 con 0x7f6ef41021e0 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef3e00700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 0x7f6ef40749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef9600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef4074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.235+0000 7f6ef8c00700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4074490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006010 tx=0x7f6ee8003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.236+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80050c0 con 0x7f6ef41021e0 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.236+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ee800f460 con 0x7f6ef41021e0 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.236+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8017650 con 0x7f6ef41021e0 2024-03-21T09:10:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.236+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef41a7130 con 0x7f6ef41021e0 2024-03-21T09:10:53.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.236+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef41a7530 con 0x7f6ef41021e0 2024-03-21T09:10:53.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.237+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef4068e50 con 0x7f6ef41021e0 2024-03-21T09:10:53.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.239+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ee800f610 con 0x7f6ef41021e0 2024-03-21T09:10:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.241+0000 7f6ef1600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6edc0a3030 0x7f6edc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.241+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ee80d0580 con 0x7f6ef41021e0 2024-03-21T09:10:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.241+0000 7f6ef3e00700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6edc0a3030 0x7f6edc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.242+0000 7f6ef3e00700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6edc0a3030 0x7f6edc0a54f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009680 tx=0x7f6ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.243+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ee8013070 con 0x7f6ef41021e0 2024-03-21T09:10:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.468+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ef402dbb0 con 0x7f6edc0a3030 2024-03-21T09:10:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.493+0000 7f6ef1600700 1 -- 172.21.15.37:0/3530125584 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340917 (secure 0 0 0) 0x7f6ef402dbb0 con 0x7f6edc0a3030 2024-03-21T09:10:53.488 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.501+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6edc0a3030 msgr2=0x7f6edc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.501+0000 7f6efb600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6edc0a3030 0x7f6edc0a54f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009680 tx=0x7f6ee0009260 comp rx=0 tx=0).stop 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 msgr2=0x7f6ef4074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4074490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006010 tx=0x7f6ee8003a90 comp rx=0 tx=0).stop 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 shutdown_connections 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41081c0 0x7f6ef4074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6edc0a3030 0x7f6edc0a54f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ef41021e0 0x7f6ef4074490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 --2- 172.21.15.37:0/3530125584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ef4102ba0 0x7f6ef40749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:53.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 >> 172.21.15.37:0/3530125584 conn(0x7f6ef40792c0 msgr2=0x7f6ef4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:53.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 shutdown_connections 2024-03-21T09:10:53.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:53.502+0000 7f6efb600700 1 -- 172.21.15.37:0/3530125584 wait complete. 2024-03-21T09:10:53.499 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:10:53.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:53 smithi053 ceph-mon[36169]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:53.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:53 smithi071 ceph-mon[36597]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:53.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:53 smithi037 ceph-mon[31830]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:54.363 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-03-21T09:10:51.823498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657065,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":691,"num_read_kb":466,"num_write":351,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382572,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559252,"statfs":{"total":1151940427776,"available":1151548674048,"internally_reserved":0,"allocated":12849152,"data_stored":8233737,"data_compressed":424490,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":17,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003460"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404881+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:10:47.404881+0000","last_peered":"2024-03-21T09:10:47.404881+0000","last_clean":"2024-03-21T09:10:47.404881+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:10:47.404881+0000","last_undegraded":"2024-03-21T09:10:47.404881+0000","last_fullsized":"2024-03-21T09:10:47.404881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438353+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:10:47.438353+0000","last_peered":"2024-03-21T09:10:47.438353+0000","last_clean":"2024-03-21T09:10:47.438353+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:10:47.438353+0000","last_undegraded":"2024-03-21T09:10:47.438353+0000","last_fullsized":"2024-03-21T09:10:47.438353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437885+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:10:47.437885+0000","last_peered":"2024-03-21T09:10:47.437885+0000","last_clean":"2024-03-21T09:10:47.437885+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:10:47.437885+0000","last_undegraded":"2024-03-21T09:10:47.437885+0000","last_fullsized":"2024-03-21T09:10:47.437885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408278+0000","last_change":"2024-03-21T09:09:47.203174+0000","last_active":"2024-03-21T09:10:47.408278+0000","last_peered":"2024-03-21T09:10:47.408278+0000","last_clean":"2024-03-21T09:10:47.408278+0000","last_became_active":"2024-03-21T09:09:47.202519+0000","last_became_peered":"2024-03-21T09:09:47.202519+0000","last_unstale":"2024-03-21T09:10:47.408278+0000","last_undegraded":"2024-03-21T09:10:47.408278+0000","last_fullsized":"2024-03-21T09:10:47.408278+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:48.445311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408452+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:10:47.408452+0000","last_peered":"2024-03-21T09:10:47.408452+0000","last_clean":"2024-03-21T09:10:47.408452+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:10:47.408452+0000","last_undegraded":"2024-03-21T09:10:47.408452+0000","last_fullsized":"2024-03-21T09:10:47.408452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.417919+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:10:46.417919+0000","last_peered":"2024-03-21T09:10:46.417919+0000","last_clean":"2024-03-21T09:10:46.417919+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:10:46.417919+0000","last_undegraded":"2024-03-21T09:10:46.417919+0000","last_fullsized":"2024-03-21T09:10:46.417919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436073+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:10:47.436073+0000","last_peered":"2024-03-21T09:10:47.436073+0000","last_clean":"2024-03-21T09:10:47.436073+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:10:47.436073+0000","last_undegraded":"2024-03-21T09:10:47.436073+0000","last_fullsized":"2024-03-21T09:10:47.436073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406994+0000","last_change":"2024-03-21T09:09:47.204487+0000","last_active":"2024-03-21T09:10:47.406994+0000","last_peered":"2024-03-21T09:10:47.406994+0000","last_clean":"2024-03-21T09:10:47.406994+0000","last_became_active":"2024-03-21T09:09:47.202961+0000","last_became_peered":"2024-03-21T09:09:47.202961+0000","last_unstale":"2024-03-21T09:10:47.406994+0000","last_undegraded":"2024-03-21T09:10:47.406994+0000","last_fullsized":"2024-03-21T09:10:47.406994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:50.524550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406860+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:10:47.406860+0000","last_peered":"2024-03-21T09:10:47.406860+0000","last_clean":"2024-03-21T09:10:47.406860+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:10:47.406860+0000","last_undegraded":"2024-03-21T09:10:47.406860+0000","last_fullsized":"2024-03-21T09:10:47.406860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418323+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:10:46.418323+0000","last_peered":"2024-03-21T09:10:46.418323+0000","last_clean":"2024-03-21T09:10:46.418323+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:10:46.418323+0000","last_undegraded":"2024-03-21T09:10:46.418323+0000","last_fullsized":"2024-03-21T09:10:46.418323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436613+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:10:47.436613+0000","last_peered":"2024-03-21T09:10:47.436613+0000","last_clean":"2024-03-21T09:10:47.436613+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:10:47.436613+0000","last_undegraded":"2024-03-21T09:10:47.436613+0000","last_fullsized":"2024-03-21T09:10:47.436613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436916+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:10:47.436916+0000","last_peered":"2024-03-21T09:10:47.436916+0000","last_clean":"2024-03-21T09:10:47.436916+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:10:47.436916+0000","last_undegraded":"2024-03-21T09:10:47.436916+0000","last_fullsized":"2024-03-21T09:10:47.436916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439086+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:10:47.439086+0000","last_peered":"2024-03-21T09:10:47.439086+0000","last_clean":"2024-03-21T09:10:47.439086+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:10:47.439086+0000","last_undegraded":"2024-03-21T09:10:47.439086+0000","last_fullsized":"2024-03-21T09:10:47.439086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436953+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:10:47.436953+0000","last_peered":"2024-03-21T09:10:47.436953+0000","last_clean":"2024-03-21T09:10:47.436953+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:10:47.436953+0000","last_undegraded":"2024-03-21T09:10:47.436953+0000","last_fullsized":"2024-03-21T09:10:47.436953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437152+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:10:47.437152+0000","last_peered":"2024-03-21T09:10:47.437152+0000","last_clean":"2024-03-21T09:10:47.437152+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:10:47.437152+0000","last_undegraded":"2024-03-21T09:10:47.437152+0000","last_fullsized":"2024-03-21T09:10:47.437152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406104+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:10:47.406104+0000","last_peered":"2024-03-21T09:10:47.406104+0000","last_clean":"2024-03-21T09:10:47.406104+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:10:47.406104+0000","last_undegraded":"2024-03-21T09:10:47.406104+0000","last_fullsized":"2024-03-21T09:10:47.406104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404636+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:10:47.404636+0000","last_peered":"2024-03-21T09:10:47.404636+0000","last_clean":"2024-03-21T09:10:47.404636+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:10:47.404636+0000","last_undegraded":"2024-03-21T09:10:47.404636+0000","last_fullsized":"2024-03-21T09:10:47.404636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408721+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:10:47.408721+0000","last_peered":"2024-03-21T09:10:47.408721+0000","last_clean":"2024-03-21T09:10:47.408721+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:10:47.408721+0000","last_undegraded":"2024-03-21T09:10:47.408721+0000","last_fullsized":"2024-03-21T09:10:47.408721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406697+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:10:47.406697+0000","last_peered":"2024-03-21T09:10:47.406697+0000","last_clean":"2024-03-21T09:10:47.406697+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:10:47.406697+0000","last_undegraded":"2024-03-21T09:10:47.406697+0000","last_fullsized":"2024-03-21T09:10:47.406697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406059+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:10:47.406059+0000","last_peered":"2024-03-21T09:10:47.406059+0000","last_clean":"2024-03-21T09:10:47.406059+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:10:47.406059+0000","last_undegraded":"2024-03-21T09:10:47.406059+0000","last_fullsized":"2024-03-21T09:10:47.406059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408966+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:10:47.408966+0000","last_peered":"2024-03-21T09:10:47.408966+0000","last_clean":"2024-03-21T09:10:47.408966+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:10:47.408966+0000","last_undegraded":"2024-03-21T09:10:47.408966+0000","last_fullsized":"2024-03-21T09:10:47.408966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.407124+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:10:46.407124+0000","last_peered":"2024-03-21T09:10:46.407124+0000","last_clean":"2024-03-21T09:10:46.407124+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:10:46.407124+0000","last_undegraded":"2024-03-21T09:10:46.407124+0000","last_fullsized":"2024-03-21T09:10:46.407124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435790+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:10:47.435790+0000","last_peered":"2024-03-21T09:10:47.435790+0000","last_clean":"2024-03-21T09:10:47.435790+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:10:47.435790+0000","last_undegraded":"2024-03-21T09:10:47.435790+0000","last_fullsized":"2024-03-21T09:10:47.435790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.407421+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:10:46.407421+0000","last_peered":"2024-03-21T09:10:46.407421+0000","last_clean":"2024-03-21T09:10:46.407421+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:10:46.407421+0000","last_undegraded":"2024-03-21T09:10:46.407421+0000","last_fullsized":"2024-03-21T09:10:46.407421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439100+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:10:47.439100+0000","last_peered":"2024-03-21T09:10:47.439100+0000","last_clean":"2024-03-21T09:10:47.439100+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:10:47.439100+0000","last_undegraded":"2024-03-21T09:10:47.439100+0000","last_fullsized":"2024-03-21T09:10:47.439100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436222+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:10:47.436222+0000","last_peered":"2024-03-21T09:10:47.436222+0000","last_clean":"2024-03-21T09:10:47.436222+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:10:47.436222+0000","last_undegraded":"2024-03-21T09:10:47.436222+0000","last_fullsized":"2024-03-21T09:10:47.436222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408988+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:10:47.408988+0000","last_peered":"2024-03-21T09:10:47.408988+0000","last_clean":"2024-03-21T09:10:47.408988+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:10:47.408988+0000","last_undegraded":"2024-03-21T09:10:47.408988+0000","last_fullsized":"2024-03-21T09:10:47.408988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406500+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:10:47.406500+0000","last_peered":"2024-03-21T09:10:47.406500+0000","last_clean":"2024-03-21T09:10:47.406500+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:10:47.406500+0000","last_undegraded":"2024-03-21T09:10:47.406500+0000","last_fullsized":"2024-03-21T09:10:47.406500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436179+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:10:47.436179+0000","last_peered":"2024-03-21T09:10:47.436179+0000","last_clean":"2024-03-21T09:10:47.436179+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:10:47.436179+0000","last_undegraded":"2024-03-21T09:10:47.436179+0000","last_fullsized":"2024-03-21T09:10:47.436179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411729+0000","last_change":"2024-03-21T09:10:47.411335+0000","last_active":"2024-03-21T09:10:47.411729+0000","last_peered":"2024-03-21T09:10:47.411729+0000","last_clean":"2024-03-21T09:10:47.411729+0000","last_became_active":"2024-03-21T09:10:47.408103+0000","last_became_peered":"2024-03-21T09:10:47.408103+0000","last_unstale":"2024-03-21T09:10:47.411729+0000","last_undegraded":"2024-03-21T09:10:47.411729+0000","last_fullsized":"2024-03-21T09:10:47.411729+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:51.600076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407535+0000","last_change":"2024-03-21T09:09:47.433837+0000","last_active":"2024-03-21T09:10:47.407535+0000","last_peered":"2024-03-21T09:10:47.407535+0000","last_clean":"2024-03-21T09:10:47.407535+0000","last_became_active":"2024-03-21T09:09:47.433538+0000","last_became_peered":"2024-03-21T09:09:47.433538+0000","last_unstale":"2024-03-21T09:10:47.407535+0000","last_undegraded":"2024-03-21T09:10:47.407535+0000","last_fullsized":"2024-03-21T09:10:47.407535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:41.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406851+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:10:47.406851+0000","last_peered":"2024-03-21T09:10:47.406851+0000","last_clean":"2024-03-21T09:10:47.406851+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:10:47.406851+0000","last_undegraded":"2024-03-21T09:10:47.406851+0000","last_fullsized":"2024-03-21T09:10:47.406851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":126,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407344+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:10:47.407344+0000","last_peered":"2024-03-21T09:10:47.407344+0000","last_clean":"2024-03-21T09:10:47.407344+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:10:47.407344+0000","last_undegraded":"2024-03-21T09:10:47.407344+0000","last_fullsized":"2024-03-21T09:10:47.407344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408226+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:10:47.408226+0000","last_peered":"2024-03-21T09:10:47.408226+0000","last_clean":"2024-03-21T09:10:47.408226+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:10:47.408226+0000","last_undegraded":"2024-03-21T09:10:47.408226+0000","last_fullsized":"2024-03-21T09:10:47.408226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406390+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:10:47.406390+0000","last_peered":"2024-03-21T09:10:47.406390+0000","last_clean":"2024-03-21T09:10:47.406390+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:10:47.406390+0000","last_undegraded":"2024-03-21T09:10:47.406390+0000","last_fullsized":"2024-03-21T09:10:47.406390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.417893+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:10:46.417893+0000","last_peered":"2024-03-21T09:10:46.417893+0000","last_clean":"2024-03-21T09:10:46.417893+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:10:46.417893+0000","last_undegraded":"2024-03-21T09:10:46.417893+0000","last_fullsized":"2024-03-21T09:10:46.417893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407391+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:10:47.407391+0000","last_peered":"2024-03-21T09:10:47.407391+0000","last_clean":"2024-03-21T09:10:47.407391+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:10:47.407391+0000","last_undegraded":"2024-03-21T09:10:47.407391+0000","last_fullsized":"2024-03-21T09:10:47.407391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404606+0000","last_change":"2024-03-21T09:09:47.432250+0000","last_active":"2024-03-21T09:10:47.404606+0000","last_peered":"2024-03-21T09:10:47.404606+0000","last_clean":"2024-03-21T09:10:47.404606+0000","last_became_active":"2024-03-21T09:09:47.431784+0000","last_became_peered":"2024-03-21T09:09:47.431784+0000","last_unstale":"2024-03-21T09:10:47.404606+0000","last_undegraded":"2024-03-21T09:10:47.404606+0000","last_fullsized":"2024-03-21T09:10:47.404606+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:58.438472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438318+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:10:47.438318+0000","last_peered":"2024-03-21T09:10:47.438318+0000","last_clean":"2024-03-21T09:10:47.438318+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:10:47.438318+0000","last_undegraded":"2024-03-21T09:10:47.438318+0000","last_fullsized":"2024-03-21T09:10:47.438318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.404452+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:10:46.404452+0000","last_peered":"2024-03-21T09:10:46.404452+0000","last_clean":"2024-03-21T09:10:46.404452+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:10:46.404452+0000","last_undegraded":"2024-03-21T09:10:46.404452+0000","last_fullsized":"2024-03-21T09:10:46.404452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438181+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:10:47.438181+0000","last_peered":"2024-03-21T09:10:47.438181+0000","last_clean":"2024-03-21T09:10:47.438181+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:10:47.438181+0000","last_undegraded":"2024-03-21T09:10:47.438181+0000","last_fullsized":"2024-03-21T09:10:47.438181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406734+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:10:47.406734+0000","last_peered":"2024-03-21T09:10:47.406734+0000","last_clean":"2024-03-21T09:10:47.406734+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:10:47.406734+0000","last_undegraded":"2024-03-21T09:10:47.406734+0000","last_fullsized":"2024-03-21T09:10:47.406734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407583+0000","last_change":"2024-03-21T09:09:47.205219+0000","last_active":"2024-03-21T09:10:47.407583+0000","last_peered":"2024-03-21T09:10:47.407583+0000","last_clean":"2024-03-21T09:10:47.407583+0000","last_became_active":"2024-03-21T09:09:47.204655+0000","last_became_peered":"2024-03-21T09:09:47.204655+0000","last_unstale":"2024-03-21T09:10:47.407583+0000","last_undegraded":"2024-03-21T09:10:47.407583+0000","last_fullsized":"2024-03-21T09:10:47.407583+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:44.407879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.405354+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:10:46.405354+0000","last_peered":"2024-03-21T09:10:46.405354+0000","last_clean":"2024-03-21T09:10:46.405354+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:10:46.405354+0000","last_undegraded":"2024-03-21T09:10:46.405354+0000","last_fullsized":"2024-03-21T09:10:46.405354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418300+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:10:46.418300+0000","last_peered":"2024-03-21T09:10:46.418300+0000","last_clean":"2024-03-21T09:10:46.418300+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:10:46.418300+0000","last_undegraded":"2024-03-21T09:10:46.418300+0000","last_fullsized":"2024-03-21T09:10:46.418300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406786+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:10:47.406786+0000","last_peered":"2024-03-21T09:10:47.406786+0000","last_clean":"2024-03-21T09:10:47.406786+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:10:47.406786+0000","last_undegraded":"2024-03-21T09:10:47.406786+0000","last_fullsized":"2024-03-21T09:10:47.406786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407424+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:10:47.407424+0000","last_peered":"2024-03-21T09:10:47.407424+0000","last_clean":"2024-03-21T09:10:47.407424+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:10:47.407424+0000","last_undegraded":"2024-03-21T09:10:47.407424+0000","last_fullsized":"2024-03-21T09:10:47.407424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436890+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:10:47.436890+0000","last_peered":"2024-03-21T09:10:47.436890+0000","last_clean":"2024-03-21T09:10:47.436890+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:10:47.436890+0000","last_undegraded":"2024-03-21T09:10:47.436890+0000","last_fullsized":"2024-03-21T09:10:47.436890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439041+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:10:47.439041+0000","last_peered":"2024-03-21T09:10:47.439041+0000","last_clean":"2024-03-21T09:10:47.439041+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:10:47.439041+0000","last_undegraded":"2024-03-21T09:10:47.439041+0000","last_fullsized":"2024-03-21T09:10:47.439041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436367+0000","last_change":"2024-03-21T09:09:47.283764+0000","last_active":"2024-03-21T09:10:47.436367+0000","last_peered":"2024-03-21T09:10:47.436367+0000","last_clean":"2024-03-21T09:10:47.436367+0000","last_became_active":"2024-03-21T09:09:47.283203+0000","last_became_peered":"2024-03-21T09:09:47.283203+0000","last_unstale":"2024-03-21T09:10:47.436367+0000","last_undegraded":"2024-03-21T09:10:47.436367+0000","last_fullsized":"2024-03-21T09:10:47.436367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:55.502176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407203+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:10:47.407203+0000","last_peered":"2024-03-21T09:10:47.407203+0000","last_clean":"2024-03-21T09:10:47.407203+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:10:47.407203+0000","last_undegraded":"2024-03-21T09:10:47.407203+0000","last_fullsized":"2024-03-21T09:10:47.407203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406678+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:10:47.406678+0000","last_peered":"2024-03-21T09:10:47.406678+0000","last_clean":"2024-03-21T09:10:47.406678+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:10:47.406678+0000","last_undegraded":"2024-03-21T09:10:47.406678+0000","last_fullsized":"2024-03-21T09:10:47.406678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407466+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:10:47.407466+0000","last_peered":"2024-03-21T09:10:47.407466+0000","last_clean":"2024-03-21T09:10:47.407466+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:10:47.407466+0000","last_undegraded":"2024-03-21T09:10:47.407466+0000","last_fullsized":"2024-03-21T09:10:47.407466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":245,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.895799+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:10:46.895799+0000","last_peered":"2024-03-21T09:10:46.895799+0000","last_clean":"2024-03-21T09:10:46.895799+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:10:46.895799+0000","last_undegraded":"2024-03-21T09:10:46.895799+0000","last_fullsized":"2024-03-21T09:10:46.895799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":183,"num_read_kb":152,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408807+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:10:47.408807+0000","last_peered":"2024-03-21T09:10:47.408807+0000","last_clean":"2024-03-21T09:10:47.408807+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:10:47.408807+0000","last_undegraded":"2024-03-21T09:10:47.408807+0000","last_fullsized":"2024-03-21T09:10:47.408807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407705+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:10:47.407705+0000","last_peered":"2024-03-21T09:10:47.407705+0000","last_clean":"2024-03-21T09:10:47.407705+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:10:47.407705+0000","last_undegraded":"2024-03-21T09:10:47.407705+0000","last_fullsized":"2024-03-21T09:10:47.407705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.420395+0000","last_change":"2024-03-21T09:10:47.420051+0000","last_active":"2024-03-21T09:10:47.420395+0000","last_peered":"2024-03-21T09:10:47.420395+0000","last_clean":"2024-03-21T09:10:47.420395+0000","last_became_active":"2024-03-21T09:10:47.412598+0000","last_became_peered":"2024-03-21T09:10:47.412598+0000","last_unstale":"2024-03-21T09:10:47.420395+0000","last_undegraded":"2024-03-21T09:10:47.420395+0000","last_fullsized":"2024-03-21T09:10:47.420395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:43.675518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438722+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:10:47.438722+0000","last_peered":"2024-03-21T09:10:47.438722+0000","last_clean":"2024-03-21T09:10:47.438722+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:10:47.438722+0000","last_undegraded":"2024-03-21T09:10:47.438722+0000","last_fullsized":"2024-03-21T09:10:47.438722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404510+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:10:47.404510+0000","last_peered":"2024-03-21T09:10:47.404510+0000","last_clean":"2024-03-21T09:10:47.404510+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:10:47.404510+0000","last_undegraded":"2024-03-21T09:10:47.404510+0000","last_fullsized":"2024-03-21T09:10:47.404510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407155+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:10:47.407155+0000","last_peered":"2024-03-21T09:10:47.407155+0000","last_clean":"2024-03-21T09:10:47.407155+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:10:47.407155+0000","last_undegraded":"2024-03-21T09:10:47.407155+0000","last_fullsized":"2024-03-21T09:10:47.407155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436310+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:10:47.436310+0000","last_peered":"2024-03-21T09:10:47.436310+0000","last_clean":"2024-03-21T09:10:47.436310+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:10:47.436310+0000","last_undegraded":"2024-03-21T09:10:47.436310+0000","last_fullsized":"2024-03-21T09:10:47.436310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408325+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:10:47.408325+0000","last_peered":"2024-03-21T09:10:47.408325+0000","last_clean":"2024-03-21T09:10:47.408325+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:10:47.408325+0000","last_undegraded":"2024-03-21T09:10:47.408325+0000","last_fullsized":"2024-03-21T09:10:47.408325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":128,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.932178+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:10:47.932178+0000","last_peered":"2024-03-21T09:10:47.932178+0000","last_clean":"2024-03-21T09:10:47.932178+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:10:47.932178+0000","last_undegraded":"2024-03-21T09:10:47.932178+0000","last_fullsized":"2024-03-21T09:10:47.932178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436136+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:10:47.436136+0000","last_peered":"2024-03-21T09:10:47.436136+0000","last_clean":"2024-03-21T09:10:47.436136+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:10:47.436136+0000","last_undegraded":"2024-03-21T09:10:47.436136+0000","last_fullsized":"2024-03-21T09:10:47.436136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439584+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:10:47.439584+0000","last_peered":"2024-03-21T09:10:47.439584+0000","last_clean":"2024-03-21T09:10:47.439584+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:10:47.439584+0000","last_undegraded":"2024-03-21T09:10:47.439584+0000","last_fullsized":"2024-03-21T09:10:47.439584+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407976+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:10:47.407976+0000","last_peered":"2024-03-21T09:10:47.407976+0000","last_clean":"2024-03-21T09:10:47.407976+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:10:47.407976+0000","last_undegraded":"2024-03-21T09:10:47.407976+0000","last_fullsized":"2024-03-21T09:10:47.407976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406824+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:10:47.406824+0000","last_peered":"2024-03-21T09:10:47.406824+0000","last_clean":"2024-03-21T09:10:47.406824+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:10:47.406824+0000","last_undegraded":"2024-03-21T09:10:47.406824+0000","last_fullsized":"2024-03-21T09:10:47.406824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406751+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:10:47.406751+0000","last_peered":"2024-03-21T09:10:47.406751+0000","last_clean":"2024-03-21T09:10:47.406751+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:10:47.406751+0000","last_undegraded":"2024-03-21T09:10:47.406751+0000","last_fullsized":"2024-03-21T09:10:47.406751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409098+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:10:47.409098+0000","last_peered":"2024-03-21T09:10:47.409098+0000","last_clean":"2024-03-21T09:10:47.409098+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:10:47.409098+0000","last_undegraded":"2024-03-21T09:10:47.409098+0000","last_fullsized":"2024-03-21T09:10:47.409098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404345+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:10:47.404345+0000","last_peered":"2024-03-21T09:10:47.404345+0000","last_clean":"2024-03-21T09:10:47.404345+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:10:47.404345+0000","last_undegraded":"2024-03-21T09:10:47.404345+0000","last_fullsized":"2024-03-21T09:10:47.404345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408166+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:10:47.408166+0000","last_peered":"2024-03-21T09:10:47.408166+0000","last_clean":"2024-03-21T09:10:47.408166+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:10:47.408166+0000","last_undegraded":"2024-03-21T09:10:47.408166+0000","last_fullsized":"2024-03-21T09:10:47.408166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408295+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:10:47.408295+0000","last_peered":"2024-03-21T09:10:47.408295+0000","last_clean":"2024-03-21T09:10:47.408295+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:10:47.408295+0000","last_undegraded":"2024-03-21T09:10:47.408295+0000","last_fullsized":"2024-03-21T09:10:47.408295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418195+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:10:46.418195+0000","last_peered":"2024-03-21T09:10:46.418195+0000","last_clean":"2024-03-21T09:10:46.418195+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:10:46.418195+0000","last_undegraded":"2024-03-21T09:10:46.418195+0000","last_fullsized":"2024-03-21T09:10:46.418195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439256+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:10:47.439256+0000","last_peered":"2024-03-21T09:10:47.439256+0000","last_clean":"2024-03-21T09:10:47.439256+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:10:47.439256+0000","last_undegraded":"2024-03-21T09:10:47.439256+0000","last_fullsized":"2024-03-21T09:10:47.439256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436162+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:10:47.436162+0000","last_peered":"2024-03-21T09:10:47.436162+0000","last_clean":"2024-03-21T09:10:47.436162+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:10:47.436162+0000","last_undegraded":"2024-03-21T09:10:47.436162+0000","last_fullsized":"2024-03-21T09:10:47.436162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.418323+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:10:46.418323+0000","last_peered":"2024-03-21T09:10:46.418323+0000","last_clean":"2024-03-21T09:10:46.418323+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:10:46.418323+0000","last_undegraded":"2024-03-21T09:10:46.418323+0000","last_fullsized":"2024-03-21T09:10:46.418323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439669+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:10:47.439669+0000","last_peered":"2024-03-21T09:10:47.439669+0000","last_clean":"2024-03-21T09:10:47.439669+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:10:47.439669+0000","last_undegraded":"2024-03-21T09:10:47.439669+0000","last_fullsized":"2024-03-21T09:10:47.439669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406998+0000","last_change":"2024-03-21T09:09:47.205699+0000","last_active":"2024-03-21T09:10:47.406998+0000","last_peered":"2024-03-21T09:10:47.406998+0000","last_clean":"2024-03-21T09:10:47.406998+0000","last_became_active":"2024-03-21T09:09:47.203657+0000","last_became_peered":"2024-03-21T09:09:47.203657+0000","last_unstale":"2024-03-21T09:10:47.406998+0000","last_undegraded":"2024-03-21T09:10:47.406998+0000","last_fullsized":"2024-03-21T09:10:47.406998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:42.930153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":124,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438214+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:10:47.438214+0000","last_peered":"2024-03-21T09:10:47.438214+0000","last_clean":"2024-03-21T09:10:47.438214+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:10:47.438214+0000","last_undegraded":"2024-03-21T09:10:47.438214+0000","last_fullsized":"2024-03-21T09:10:47.438214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436654+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:10:47.436654+0000","last_peered":"2024-03-21T09:10:47.436654+0000","last_clean":"2024-03-21T09:10:47.436654+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:10:47.436654+0000","last_undegraded":"2024-03-21T09:10:47.436654+0000","last_fullsized":"2024-03-21T09:10:47.436654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.406616+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:10:46.406616+0000","last_peered":"2024-03-21T09:10:46.406616+0000","last_clean":"2024-03-21T09:10:46.406616+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:10:46.406616+0000","last_undegraded":"2024-03-21T09:10:46.406616+0000","last_fullsized":"2024-03-21T09:10:46.406616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436273+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:10:47.436273+0000","last_peered":"2024-03-21T09:10:47.436273+0000","last_clean":"2024-03-21T09:10:47.436273+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:10:47.436273+0000","last_undegraded":"2024-03-21T09:10:47.436273+0000","last_fullsized":"2024-03-21T09:10:47.436273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435910+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:10:47.435910+0000","last_peered":"2024-03-21T09:10:47.435910+0000","last_clean":"2024-03-21T09:10:47.435910+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:10:47.435910+0000","last_undegraded":"2024-03-21T09:10:47.435910+0000","last_fullsized":"2024-03-21T09:10:47.435910+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407011+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:10:47.407011+0000","last_peered":"2024-03-21T09:10:47.407011+0000","last_clean":"2024-03-21T09:10:47.407011+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:10:47.407011+0000","last_undegraded":"2024-03-21T09:10:47.407011+0000","last_fullsized":"2024-03-21T09:10:47.407011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438197+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:10:47.438197+0000","last_peered":"2024-03-21T09:10:47.438197+0000","last_clean":"2024-03-21T09:10:47.438197+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:10:47.438197+0000","last_undegraded":"2024-03-21T09:10:47.438197+0000","last_fullsized":"2024-03-21T09:10:47.438197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436241+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:10:47.436241+0000","last_peered":"2024-03-21T09:10:47.436241+0000","last_clean":"2024-03-21T09:10:47.436241+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:10:47.436241+0000","last_undegraded":"2024-03-21T09:10:47.436241+0000","last_fullsized":"2024-03-21T09:10:47.436241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404810+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:10:47.404810+0000","last_peered":"2024-03-21T09:10:47.404810+0000","last_clean":"2024-03-21T09:10:47.404810+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:10:47.404810+0000","last_undegraded":"2024-03-21T09:10:47.404810+0000","last_fullsized":"2024-03-21T09:10:47.404810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409034+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:10:47.409034+0000","last_peered":"2024-03-21T09:10:47.409034+0000","last_clean":"2024-03-21T09:10:47.409034+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:10:47.409034+0000","last_undegraded":"2024-03-21T09:10:47.409034+0000","last_fullsized":"2024-03-21T09:10:47.409034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439437+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:10:47.439437+0000","last_peered":"2024-03-21T09:10:47.439437+0000","last_clean":"2024-03-21T09:10:47.439437+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:10:47.439437+0000","last_undegraded":"2024-03-21T09:10:47.439437+0000","last_fullsized":"2024-03-21T09:10:47.439437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.404302+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:10:46.404302+0000","last_peered":"2024-03-21T09:10:46.404302+0000","last_clean":"2024-03-21T09:10:46.404302+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:10:46.404302+0000","last_undegraded":"2024-03-21T09:10:46.404302+0000","last_fullsized":"2024-03-21T09:10:46.404302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408709+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:10:47.408709+0000","last_peered":"2024-03-21T09:10:47.408709+0000","last_clean":"2024-03-21T09:10:47.408709+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:10:47.408709+0000","last_undegraded":"2024-03-21T09:10:47.408709+0000","last_fullsized":"2024-03-21T09:10:47.408709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.405533+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:10:46.405533+0000","last_peered":"2024-03-21T09:10:46.405533+0000","last_clean":"2024-03-21T09:10:46.405533+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:10:46.405533+0000","last_undegraded":"2024-03-21T09:10:46.405533+0000","last_fullsized":"2024-03-21T09:10:46.405533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405862+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:10:47.405862+0000","last_peered":"2024-03-21T09:10:47.405862+0000","last_clean":"2024-03-21T09:10:47.405862+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:10:47.405862+0000","last_undegraded":"2024-03-21T09:10:47.405862+0000","last_fullsized":"2024-03-21T09:10:47.405862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436847+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:10:47.436847+0000","last_peered":"2024-03-21T09:10:47.436847+0000","last_clean":"2024-03-21T09:10:47.436847+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:10:47.436847+0000","last_undegraded":"2024-03-21T09:10:47.436847+0000","last_fullsized":"2024-03-21T09:10:47.436847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438318+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:10:47.438318+0000","last_peered":"2024-03-21T09:10:47.438318+0000","last_clean":"2024-03-21T09:10:47.438318+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:10:47.438318+0000","last_undegraded":"2024-03-21T09:10:47.438318+0000","last_fullsized":"2024-03-21T09:10:47.438318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436484+0000","last_change":"2024-03-21T09:09:47.203560+0000","last_active":"2024-03-21T09:10:47.436484+0000","last_peered":"2024-03-21T09:10:47.436484+0000","last_clean":"2024-03-21T09:10:47.436484+0000","last_became_active":"2024-03-21T09:09:47.203072+0000","last_became_peered":"2024-03-21T09:09:47.203072+0000","last_unstale":"2024-03-21T09:10:47.436484+0000","last_undegraded":"2024-03-21T09:10:47.436484+0000","last_fullsized":"2024-03-21T09:10:47.436484+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:03:39.037042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406138+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:10:47.406138+0000","last_peered":"2024-03-21T09:10:47.406138+0000","last_clean":"2024-03-21T09:10:47.406138+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:10:47.406138+0000","last_undegraded":"2024-03-21T09:10:47.406138+0000","last_fullsized":"2024-03-21T09:10:47.406138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408044+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:10:47.408044+0000","last_peered":"2024-03-21T09:10:47.408044+0000","last_clean":"2024-03-21T09:10:47.408044+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:10:47.408044+0000","last_undegraded":"2024-03-21T09:10:47.408044+0000","last_fullsized":"2024-03-21T09:10:47.408044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408524+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:10:47.408524+0000","last_peered":"2024-03-21T09:10:47.408524+0000","last_clean":"2024-03-21T09:10:47.408524+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:10:47.408524+0000","last_undegraded":"2024-03-21T09:10:47.408524+0000","last_fullsized":"2024-03-21T09:10:47.408524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.404714+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:10:46.404714+0000","last_peered":"2024-03-21T09:10:46.404714+0000","last_clean":"2024-03-21T09:10:46.404714+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:10:46.404714+0000","last_undegraded":"2024-03-21T09:10:46.404714+0000","last_fullsized":"2024-03-21T09:10:46.404714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404032+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:10:47.404032+0000","last_peered":"2024-03-21T09:10:47.404032+0000","last_clean":"2024-03-21T09:10:47.404032+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:10:47.404032+0000","last_undegraded":"2024-03-21T09:10:47.404032+0000","last_fullsized":"2024-03-21T09:10:47.404032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436838+0000","last_change":"2024-03-21T09:09:47.206281+0000","last_active":"2024-03-21T09:10:47.436838+0000","last_peered":"2024-03-21T09:10:47.436838+0000","last_clean":"2024-03-21T09:10:47.436838+0000","last_became_active":"2024-03-21T09:09:47.198427+0000","last_became_peered":"2024-03-21T09:09:47.198427+0000","last_unstale":"2024-03-21T09:10:47.436838+0000","last_undegraded":"2024-03-21T09:10:47.436838+0000","last_fullsized":"2024-03-21T09:10:47.436838+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:32:46.182225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406047+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:10:47.406047+0000","last_peered":"2024-03-21T09:10:47.406047+0000","last_clean":"2024-03-21T09:10:47.406047+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:10:47.406047+0000","last_undegraded":"2024-03-21T09:10:47.406047+0000","last_fullsized":"2024-03-21T09:10:47.406047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.407435+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:10:46.407435+0000","last_peered":"2024-03-21T09:10:46.407435+0000","last_clean":"2024-03-21T09:10:46.407435+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:10:46.407435+0000","last_undegraded":"2024-03-21T09:10:46.407435+0000","last_fullsized":"2024-03-21T09:10:46.407435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436453+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:10:47.436453+0000","last_peered":"2024-03-21T09:10:47.436453+0000","last_clean":"2024-03-21T09:10:47.436453+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:10:47.436453+0000","last_undegraded":"2024-03-21T09:10:47.436453+0000","last_fullsized":"2024-03-21T09:10:47.436453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408381+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:10:47.408381+0000","last_peered":"2024-03-21T09:10:47.408381+0000","last_clean":"2024-03-21T09:10:47.408381+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:10:47.408381+0000","last_undegraded":"2024-03-21T09:10:47.408381+0000","last_fullsized":"2024-03-21T09:10:47.408381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436146+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:10:47.436146+0000","last_peered":"2024-03-21T09:10:47.436146+0000","last_clean":"2024-03-21T09:10:47.436146+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:10:47.436146+0000","last_undegraded":"2024-03-21T09:10:47.436146+0000","last_fullsized":"2024-03-21T09:10:47.436146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406212+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:10:47.406212+0000","last_peered":"2024-03-21T09:10:47.406212+0000","last_clean":"2024-03-21T09:10:47.406212+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:10:47.406212+0000","last_undegraded":"2024-03-21T09:10:47.406212+0000","last_fullsized":"2024-03-21T09:10:47.406212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406618+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:10:47.406618+0000","last_peered":"2024-03-21T09:10:47.406618+0000","last_clean":"2024-03-21T09:10:47.406618+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:10:47.406618+0000","last_undegraded":"2024-03-21T09:10:47.406618+0000","last_fullsized":"2024-03-21T09:10:47.406618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436748+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:10:47.436748+0000","last_peered":"2024-03-21T09:10:47.436748+0000","last_clean":"2024-03-21T09:10:47.436748+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:10:47.436748+0000","last_undegraded":"2024-03-21T09:10:47.436748+0000","last_fullsized":"2024-03-21T09:10:47.436748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437619+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:10:47.437619+0000","last_peered":"2024-03-21T09:10:47.437619+0000","last_clean":"2024-03-21T09:10:47.437619+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:10:47.437619+0000","last_undegraded":"2024-03-21T09:10:47.437619+0000","last_fullsized":"2024-03-21T09:10:47.437619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407573+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:10:47.407573+0000","last_peered":"2024-03-21T09:10:47.407573+0000","last_clean":"2024-03-21T09:10:47.407573+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:10:47.407573+0000","last_undegraded":"2024-03-21T09:10:47.407573+0000","last_fullsized":"2024-03-21T09:10:47.407573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436628+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:10:47.436628+0000","last_peered":"2024-03-21T09:10:47.436628+0000","last_clean":"2024-03-21T09:10:47.436628+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:10:47.436628+0000","last_undegraded":"2024-03-21T09:10:47.436628+0000","last_fullsized":"2024-03-21T09:10:47.436628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406364+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:10:47.406364+0000","last_peered":"2024-03-21T09:10:47.406364+0000","last_clean":"2024-03-21T09:10:47.406364+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:10:47.406364+0000","last_undegraded":"2024-03-21T09:10:47.406364+0000","last_fullsized":"2024-03-21T09:10:47.406364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439483+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:10:47.439483+0000","last_peered":"2024-03-21T09:10:47.439483+0000","last_clean":"2024-03-21T09:10:47.439483+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:10:47.439483+0000","last_undegraded":"2024-03-21T09:10:47.439483+0000","last_fullsized":"2024-03-21T09:10:47.439483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436741+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:10:47.436741+0000","last_peered":"2024-03-21T09:10:47.436741+0000","last_clean":"2024-03-21T09:10:47.436741+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:10:47.436741+0000","last_undegraded":"2024-03-21T09:10:47.436741+0000","last_fullsized":"2024-03-21T09:10:47.436741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404166+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:10:47.404166+0000","last_peered":"2024-03-21T09:10:47.404166+0000","last_clean":"2024-03-21T09:10:47.404166+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:10:47.404166+0000","last_undegraded":"2024-03-21T09:10:47.404166+0000","last_fullsized":"2024-03-21T09:10:47.404166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407311+0000","last_change":"2024-03-21T09:09:47.200898+0000","last_active":"2024-03-21T09:10:47.407311+0000","last_peered":"2024-03-21T09:10:47.407311+0000","last_clean":"2024-03-21T09:10:47.407311+0000","last_became_active":"2024-03-21T09:09:47.200548+0000","last_became_peered":"2024-03-21T09:09:47.200548+0000","last_unstale":"2024-03-21T09:10:47.407311+0000","last_undegraded":"2024-03-21T09:10:47.407311+0000","last_fullsized":"2024-03-21T09:10:47.407311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:29.793107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405900+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:10:47.405900+0000","last_peered":"2024-03-21T09:10:47.405900+0000","last_clean":"2024-03-21T09:10:47.405900+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:10:47.405900+0000","last_undegraded":"2024-03-21T09:10:47.405900+0000","last_fullsized":"2024-03-21T09:10:47.405900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437896+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:10:47.437896+0000","last_peered":"2024-03-21T09:10:47.437896+0000","last_clean":"2024-03-21T09:10:47.437896+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:10:47.437896+0000","last_undegraded":"2024-03-21T09:10:47.437896+0000","last_fullsized":"2024-03-21T09:10:47.437896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439560+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:10:47.439560+0000","last_peered":"2024-03-21T09:10:47.439560+0000","last_clean":"2024-03-21T09:10:47.439560+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:10:47.439560+0000","last_undegraded":"2024-03-21T09:10:47.439560+0000","last_fullsized":"2024-03-21T09:10:47.439560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404514+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:10:47.404514+0000","last_peered":"2024-03-21T09:10:47.404514+0000","last_clean":"2024-03-21T09:10:47.404514+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:10:47.404514+0000","last_undegraded":"2024-03-21T09:10:47.404514+0000","last_fullsized":"2024-03-21T09:10:47.404514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407214+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:10:47.407214+0000","last_peered":"2024-03-21T09:10:47.407214+0000","last_clean":"2024-03-21T09:10:47.407214+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:10:47.407214+0000","last_undegraded":"2024-03-21T09:10:47.407214+0000","last_fullsized":"2024-03-21T09:10:47.407214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.408604+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:10:46.408604+0000","last_peered":"2024-03-21T09:10:46.408604+0000","last_clean":"2024-03-21T09:10:46.408604+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:10:46.408604+0000","last_undegraded":"2024-03-21T09:10:46.408604+0000","last_fullsized":"2024-03-21T09:10:46.408604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406497+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:10:47.406497+0000","last_peered":"2024-03-21T09:10:47.406497+0000","last_clean":"2024-03-21T09:10:47.406497+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:10:47.406497+0000","last_undegraded":"2024-03-21T09:10:47.406497+0000","last_fullsized":"2024-03-21T09:10:47.406497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T09:10:46.408147+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:10:46.408147+0000","last_peered":"2024-03-21T09:10:46.408147+0000","last_clean":"2024-03-21T09:10:46.408147+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:10:46.408147+0000","last_undegraded":"2024-03-21T09:10:46.408147+0000","last_fullsized":"2024-03-21T09:10:46.408147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406443+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:10:47.406443+0000","last_peered":"2024-03-21T09:10:47.406443+0000","last_clean":"2024-03-21T09:10:47.406443+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:10:47.406443+0000","last_undegraded":"2024-03-21T09:10:47.406443+0000","last_fullsized":"2024-03-21T09:10:47.406443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438973+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:10:47.438973+0000","last_peered":"2024-03-21T09:10:47.438973+0000","last_clean":"2024-03-21T09:10:47.438973+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:10:47.438973+0000","last_undegraded":"2024-03-21T09:10:47.438973+0000","last_fullsized":"2024-03-21T09:10:47.438973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404699+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:10:47.404699+0000","last_peered":"2024-03-21T09:10:47.404699+0000","last_clean":"2024-03-21T09:10:47.404699+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:10:47.404699+0000","last_undegraded":"2024-03-21T09:10:47.404699+0000","last_fullsized":"2024-03-21T09:10:47.404699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66463,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":555,"num_read_kb":347,"num_write":168,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":199389,"data_compressed":5499,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":526262,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":1171087,"data_compressed":61728,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":526464,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":526384,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":1122434,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":521640,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":583871,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":523028,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":585187,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":503068,"data_compressed":20620,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":526822,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1117490,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62321,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63637,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64431,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:10:54.368 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T09:10:54.368 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T09:10:54.368 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T09:10:54.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T09:10:54.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:54 smithi037 ceph-mon[31830]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:10:54.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:54 smithi071 ceph-mon[36597]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:10:54.883 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:10:54.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:54 smithi053 ceph-mon[36169]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:10:55.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:55 smithi037 ceph-mon[31830]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:55.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:55 smithi071 ceph-mon[36597]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:55.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:55 smithi053 ceph-mon[36169]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 -- 172.21.15.37:0/1678922644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 msgr2=0x7f61a0111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 --2- 172.21.15.37:0/1678922644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a0111290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f619800b0a0 tx=0x7f619800b3b0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 -- 172.21.15.37:0/1678922644 shutdown_connections 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 --2- 172.21.15.37:0/1678922644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a0111290 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 --2- 172.21.15.37:0/1678922644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 --2- 172.21.15.37:0/1678922644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 0x7f61a0108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 -- 172.21.15.37:0/1678922644 >> 172.21.15.37:0/1678922644 conn(0x7f61a00ffe70 msgr2=0x7f61a0102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.257+0000 7f61a8200700 1 -- 172.21.15.37:0/1678922644 shutdown_connections 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.258+0000 7f61a8200700 1 -- 172.21.15.37:0/1678922644 wait complete. 2024-03-21T09:10:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.259+0000 7f61a8200700 1 Processor -- start 2024-03-21T09:10:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.259+0000 7f61a8200700 1 -- start start 2024-03-21T09:10:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a8200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 0x7f61a01a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a8200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a8200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0114700 con 0x7f61a0104f90 2024-03-21T09:10:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a8200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0114580 con 0x7f61a01096c0 2024-03-21T09:10:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0114880 con 0x7f61a01045e0 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48600/0 (socket says 172.21.15.37:48600) 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a5800700 1 -- 172.21.15.37:0/1201966300 learned_addr learned my addr 172.21.15.37:0/1201966300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.260+0000 7f61a4e00700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 0x7f61a01a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a5800700 1 -- 172.21.15.37:0/1201966300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 msgr2=0x7f61a019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a6200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a5800700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a5800700 1 -- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 msgr2=0x7f61a01a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a5800700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 0x7f61a01a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a5800700 1 -- 172.21.15.37:0/1201966300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a019b080 con 0x7f61a01045e0 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a4e00700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 0x7f61a01a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a6200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.261+0000 7f61a5800700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01a0760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f619400e9a0 tx=0x7f619400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.262+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619400cbd0 con 0x7f61a01045e0 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.262+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f619400cd70 con 0x7f61a01045e0 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.262+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a019b370 con 0x7f61a01045e0 2024-03-21T09:10:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.262+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940187c0 con 0x7f61a01045e0 2024-03-21T09:10:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.262+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a01a7630 con 0x7f61a01045e0 2024-03-21T09:10:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.263+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a0068e50 con 0x7f61a01045e0 2024-03-21T09:10:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.267+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6194010430 con 0x7f61a01045e0 2024-03-21T09:10:57.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.268+0000 7f619e000700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f61880a3030 0x7f61880a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.268+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f61940cf9e0 con 0x7f61a01045e0 2024-03-21T09:10:57.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.268+0000 7f61a4e00700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f61880a3030 0x7f61880a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.269+0000 7f61a4e00700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f61880a3030 0x7f61880a54f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f61a0109020 tx=0x7f618c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.276+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f619409c480 con 0x7f61a01045e0 2024-03-21T09:10:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.550+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f61a01a78b0 con 0x7f61a01045e0 2024-03-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.551+0000 7f619e000700 1 -- 172.21.15.37:0/1201966300 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f619409bd80 con 0x7f61a01045e0 2024-03-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:10:57.545 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T09:10:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f61880a3030 msgr2=0x7f61880a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f61880a3030 0x7f61880a54f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f61a0109020 tx=0x7f618c009260 comp rx=0 tx=0).stop 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 msgr2=0x7f61a01a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01a0760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f619400e9a0 tx=0x7f619400ed60 comp rx=0 tx=0).stop 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 shutdown_connections 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f61880a3030 0x7f61880a54f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61a01096c0 0x7f61a019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61a01045e0 0x7f61a01a0760 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 --2- 172.21.15.37:0/1201966300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61a0104f90 0x7f61a01a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 >> 172.21.15.37:0/1201966300 conn(0x7f61a00ffe70 msgr2=0x7f61a01007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 shutdown_connections 2024-03-21T09:10:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:57.554+0000 7f61a8200700 1 -- 172.21.15.37:0/1201966300 wait complete. 2024-03-21T09:10:57.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:57 smithi071 ceph-mon[36597]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 543 B/s wr, 7 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:57.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:57 smithi037 ceph-mon[31830]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 543 B/s wr, 7 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:57.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:57 smithi053 ceph-mon[36169]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 543 B/s wr, 7 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-21T09:10:58.407 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T09:10:58.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:10:58.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:58 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1201966300' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 -- 172.21.15.37:0/2531835645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241016a0 msgr2=0x7f9024101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 --2- 172.21.15.37:0/2531835645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241016a0 0x7f9024101b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f900c009900 tx=0x7f900c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 -- 172.21.15.37:0/2531835645 shutdown_connections 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 --2- 172.21.15.37:0/2531835645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9024102060 0x7f90241088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 --2- 172.21.15.37:0/2531835645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241016a0 0x7f9024101b20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 --2- 172.21.15.37:0/2531835645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90241004b0 0x7f90241008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 -- 172.21.15.37:0/2531835645 >> 172.21.15.37:0/2531835645 conn(0x7f90240fba20 msgr2=0x7f90240fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.695+0000 7f9023e00640 1 -- 172.21.15.37:0/2531835645 shutdown_connections 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.696+0000 7f9023e00640 1 -- 172.21.15.37:0/2531835645 wait complete. 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.697+0000 7f9023e00640 1 Processor -- start 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.697+0000 7f9023e00640 1 -- start start 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 0x7f902410d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90241016a0 0x7f902410bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9024102060 0x7f902410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024069090 con 0x7f90241016a0 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024068f10 con 0x7f90241004b0 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024069210 con 0x7f9024102060 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 0x7f902410d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 0x7f902410d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46542/0 (socket says 172.21.15.37:46542) 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 -- 172.21.15.37:0/4234202463 learned_addr learned my addr 172.21.15.37:0/4234202463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023400640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9024102060 0x7f902410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f901be00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90241016a0 0x7f902410bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 -- 172.21.15.37:0/4234202463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9024102060 msgr2=0x7f902410c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9024102060 0x7f902410c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 -- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90241016a0 msgr2=0x7f902410bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90241016a0 0x7f902410bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9022a00640 1 -- 172.21.15.37:0/4234202463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902410ca00 con 0x7f90241004b0 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.698+0000 7f9023400640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9024102060 0x7f902410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.699+0000 7f9022a00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 0x7f902410d5f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f901400e9c0 tx=0x7f901400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.699+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400cd80 con 0x7f90241004b0 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.699+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90140035f0 con 0x7f90241004b0 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.699+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014010690 con 0x7f90241004b0 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.699+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90241a19b0 con 0x7f90241004b0 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.699+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90241a1e80 con 0x7f90241004b0 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.701+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902404f470 con 0x7f90241004b0 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.702+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f90140037a0 con 0x7f90241004b0 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.704+0000 7f901b400640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90080a2f90 0x7f90080a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.704+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f90140cfae0 con 0x7f90241004b0 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.704+0000 7f901be00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90080a2f90 0x7f90080a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.705+0000 7f901be00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90080a2f90 0x7f90080a5450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f900c02c240 tx=0x7f900c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.708+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f901409c520 con 0x7f90241004b0 2024-03-21T09:10:58.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:58 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1201966300' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.915+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90240622a0 con 0x7f90241004b0 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.916+0000 7f901b400640 1 -- 172.21.15.37:0/4234202463 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7f901409be20 con 0x7f90241004b0 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90080a2f90 msgr2=0x7f90080a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90080a2f90 0x7f90080a5450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f900c02c240 tx=0x7f900c02df90 comp rx=0 tx=0).stop 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 msgr2=0x7f902410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 0x7f902410d5f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f901400e9c0 tx=0x7f901400ee80 comp rx=0 tx=0).stop 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 shutdown_connections 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f90080a2f90 0x7f90080a5450 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9024102060 0x7f902410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90241016a0 0x7f902410bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 --2- 172.21.15.37:0/4234202463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90241004b0 0x7f902410d5f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 >> 172.21.15.37:0/4234202463 conn(0x7f90240fba20 msgr2=0x7f9024104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 shutdown_connections 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:58.919+0000 7f9023e00640 1 -- 172.21.15.37:0/4234202463 wait complete. 2024-03-21T09:10:58.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:10:58.933 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:10:58.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:58 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1201966300' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 -- 172.21.15.37:0/3229779407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 msgr2=0x7f99481009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 --2- 172.21.15.37:0/3229779407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f99481009e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9938009900 tx=0x7f993802bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 -- 172.21.15.37:0/3229779407 shutdown_connections 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 --2- 172.21.15.37:0/3229779407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9948102170 0x7f9948108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 --2- 172.21.15.37:0/3229779407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 0x7f9948101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 --2- 172.21.15.37:0/3229779407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f99481009e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 -- 172.21.15.37:0/3229779407 >> 172.21.15.37:0/3229779407 conn(0x7f99480fbb30 msgr2=0x7f99480fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.163+0000 7f994da00640 1 -- 172.21.15.37:0/3229779407 shutdown_connections 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.164+0000 7f994da00640 1 -- 172.21.15.37:0/3229779407 wait complete. 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 Processor -- start 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 -- start start 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f994810d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 0x7f994810bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9948102170 0x7f994810c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948068f80 con 0x7f9948102170 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948068e00 con 0x7f99481005c0 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f994da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948069100 con 0x7f99481017b0 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.165+0000 7f9946a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f994810d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f994810d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46558/0 (socket says 172.21.15.37:46558) 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946a00640 1 -- 172.21.15.37:0/4104885848 learned_addr learned my addr 172.21.15.37:0/4104885848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 0x7f994810bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9947400640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9948102170 0x7f994810c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 -- 172.21.15.37:0/4104885848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 msgr2=0x7f994810d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f994810d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 -- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9948102170 msgr2=0x7f994810c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9948102170 0x7f994810c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 -- 172.21.15.37:0/4104885848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994810cb90 con 0x7f99481017b0 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946a00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f994810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f9946000640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 0x7f994810bdc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9f0 tx=0x7f993400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.166+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993400ccc0 con 0x7f99481017b0 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.167+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9934010910 con 0x7f99481017b0 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.167+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934018960 con 0x7f99481017b0 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.167+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99481a1b10 con 0x7f99481017b0 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.167+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99481a2060 con 0x7f99481017b0 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.168+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9948107880 con 0x7f99481017b0 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.171+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9934010ac0 con 0x7f99481017b0 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.173+0000 7f993f400640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f992c0a2f20 0x7f992c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.173+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f99340cfbf0 con 0x7f99481017b0 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.173+0000 7f9946a00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f992c0a2f20 0x7f992c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.174+0000 7f9946a00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f992c0a2f20 0x7f992c0a53e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f993802c8e0 tx=0x7f9938005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.176+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f993409c630 con 0x7f99481017b0 2024-03-21T09:10:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.389+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99480622a0 con 0x7f99481017b0 2024-03-21T09:10:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.390+0000 7f993f400640 1 -- 172.21.15.37:0/4104885848 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f993409bf30 con 0x7f99481017b0 2024-03-21T09:10:59.384 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:10:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f992c0a2f20 msgr2=0x7f992c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f992c0a2f20 0x7f992c0a53e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f993802c8e0 tx=0x7f9938005b50 comp rx=0 tx=0).stop 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 msgr2=0x7f994810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 0x7f994810bdc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9f0 tx=0x7f993400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 shutdown_connections 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f992c0a2f20 0x7f992c0a53e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9948102170 0x7f994810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99481017b0 0x7f994810bdc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 --2- 172.21.15.37:0/4104885848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99481005c0 0x7f994810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 >> 172.21.15.37:0/4104885848 conn(0x7f99480fbb30 msgr2=0x7f9948104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 shutdown_connections 2024-03-21T09:10:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.393+0000 7f994da00640 1 -- 172.21.15.37:0/4104885848 wait complete. 2024-03-21T09:10:59.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:10:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 -- 172.21.15.37:0/49101551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3184102160 msgr2=0x7f3184104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 --2- 172.21.15.37:0/49101551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3184102160 0x7f3184104550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3174009900 tx=0x7f317402bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 -- 172.21.15.37:0/49101551 shutdown_connections 2024-03-21T09:10:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 --2- 172.21.15.37:0/49101551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31841073c0 0x7f3184109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 --2- 172.21.15.37:0/49101551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 0x7f3184106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 --2- 172.21.15.37:0/49101551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3184102160 0x7f3184104550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 -- 172.21.15.37:0/49101551 >> 172.21.15.37:0/49101551 conn(0x7f31840fbb70 msgr2=0x7f31840fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 -- 172.21.15.37:0/49101551 shutdown_connections 2024-03-21T09:10:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.646+0000 7f318b000640 1 -- 172.21.15.37:0/49101551 wait complete. 2024-03-21T09:10:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.647+0000 7f318b000640 1 Processor -- start 2024-03-21T09:10:59.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 -- start start 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 0x7f3184199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 0x7f318419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31841073c0 0x7f31841a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410d3d0 con 0x7f3184104a90 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410d250 con 0x7f31841073c0 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f318b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410d550 con 0x7f3184102160 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f3183e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 0x7f3184199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f3183e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 0x7f3184199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48626/0 (socket says 172.21.15.37:48626) 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f3183e00640 1 -- 172.21.15.37:0/3142161486 learned_addr learned my addr 172.21.15.37:0/3142161486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.648+0000 7f3189000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31841073c0 0x7f31841a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f317be00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 0x7f318419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f3183e00640 1 -- 172.21.15.37:0/3142161486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31841073c0 msgr2=0x7f31841a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f3183e00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31841073c0 0x7f31841a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f3183e00640 1 -- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 msgr2=0x7f318419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f3183e00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 0x7f318419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f3183e00640 1 -- 172.21.15.37:0/3142161486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31841a1850 con 0x7f3184102160 2024-03-21T09:10:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f317be00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 0x7f318419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.649+0000 7f3183e00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 0x7f3184199c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3174005ab0 tx=0x7f317402c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.650+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174043020 con 0x7f3184102160 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.650+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31740054e0 con 0x7f3184102160 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.650+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174003ce0 con 0x7f3184102160 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.650+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31841a1ae0 con 0x7f3184102160 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.650+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31841a2080 con 0x7f3184102160 2024-03-21T09:10:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.651+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3184067c90 con 0x7f3184102160 2024-03-21T09:10:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.653+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3174046020 con 0x7f3184102160 2024-03-21T09:10:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.654+0000 7f3181600640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f31680a2fa0 0x7f31680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.654+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31740f19a0 con 0x7f3184102160 2024-03-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.654+0000 7f317be00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f31680a2fa0 0x7f31680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:59.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.655+0000 7f317be00640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f31680a2fa0 0x7f31680a5460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f316c0058c0 tx=0x7f316c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.658+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f317404d020 con 0x7f3184102160 2024-03-21T09:10:59.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:59 smithi037 ceph-mon[31830]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 196 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T09:10:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:59 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4234202463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:10:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:10:59 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4104885848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:59 smithi071 ceph-mon[36597]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 196 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T09:10:59.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:59 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4234202463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:10:59.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:10:59 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4104885848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.874+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31840622a0 con 0x7f3184102160 2024-03-21T09:10:59.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.875+0000 7f3181600640 1 -- 172.21.15.37:0/3142161486 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31740be3e0 con 0x7f3184102160 2024-03-21T09:10:59.869 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:10:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f31680a2fa0 msgr2=0x7f31680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f31680a2fa0 0x7f31680a5460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f316c0058c0 tx=0x7f316c005850 comp rx=0 tx=0).stop 2024-03-21T09:10:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 msgr2=0x7f3184199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 0x7f3184199c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3174005ab0 tx=0x7f317402c6c0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 shutdown_connections 2024-03-21T09:10:59.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f31680a2fa0 0x7f31680a5460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31841073c0 0x7f31841a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184104a90 0x7f318419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 --2- 172.21.15.37:0/3142161486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3184102160 0x7f3184199c40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.878+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 >> 172.21.15.37:0/3142161486 conn(0x7f31840fbb70 msgr2=0x7f31841018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.879+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 shutdown_connections 2024-03-21T09:10:59.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:10:59.879+0000 7f318b000640 1 -- 172.21.15.37:0/3142161486 wait complete. 2024-03-21T09:10:59.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:10:59.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:59 smithi053 ceph-mon[36169]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 196 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T09:10:59.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:59 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4234202463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:10:59.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:10:59 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4104885848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:00.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 -- 172.21.15.37:0/3611947665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8107ce0 msgr2=0x7f0cb8108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 --2- 172.21.15.37:0/3611947665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8107ce0 0x7f0cb8108100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009ca0 tx=0x7f0ca802c100 comp rx=0 tx=0).stop 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 -- 172.21.15.37:0/3611947665 shutdown_connections 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 --2- 172.21.15.37:0/3611947665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb8110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 --2- 172.21.15.37:0/3611947665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb8108ed0 0x7f0cb8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 --2- 172.21.15.37:0/3611947665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8107ce0 0x7f0cb8108100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.108+0000 7f0cbe400640 1 -- 172.21.15.37:0/3611947665 >> 172.21.15.37:0/3611947665 conn(0x7f0cb8076c90 msgr2=0x7f0cb80790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.109+0000 7f0cbe400640 1 -- 172.21.15.37:0/3611947665 shutdown_connections 2024-03-21T09:11:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.109+0000 7f0cbe400640 1 -- 172.21.15.37:0/3611947665 wait complete. 2024-03-21T09:11:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.109+0000 7f0cbe400640 1 Processor -- start 2024-03-21T09:11:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 -- start start 2024-03-21T09:11:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb8107ce0 0x7f0cb819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8108ed0 0x7f0cb819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb81a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb8113930 con 0x7f0cb8107ce0 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb81137b0 con 0x7f0cb8109890 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cbe400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb8113ab0 con 0x7f0cb8108ed0 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cb7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb81a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb8107ce0 0x7f0cb819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cb6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8108ed0 0x7f0cb819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cb7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb81a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46608/0 (socket says 172.21.15.37:46608) 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.110+0000 7f0cb7e00640 1 -- 172.21.15.37:0/207573788 learned_addr learned my addr 172.21.15.37:0/207573788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cb7e00640 1 -- 172.21.15.37:0/207573788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8108ed0 msgr2=0x7f0cb819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cb7e00640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8108ed0 0x7f0cb819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cb7e00640 1 -- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb8107ce0 msgr2=0x7f0cb819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cb7e00640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb8107ce0 0x7f0cb819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cb7e00640 1 -- 172.21.15.37:0/207573788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb81a5c30 con 0x7f0cb8109890 2024-03-21T09:11:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cb7e00640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb81a55c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000e9b0 tx=0x7f0cb000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb000cd80 con 0x7f0cb8109890 2024-03-21T09:11:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb81a5f20 con 0x7f0cb8109890 2024-03-21T09:11:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cb00035f0 con 0x7f0cb8109890 2024-03-21T09:11:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.111+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0010670 con 0x7f0cb8109890 2024-03-21T09:11:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.112+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb81a63e0 con 0x7f0cb8109890 2024-03-21T09:11:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.113+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb810f080 con 0x7f0cb8109890 2024-03-21T09:11:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.114+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0cb00037a0 con 0x7f0cb8109890 2024-03-21T09:11:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.117+0000 7f0cafe00640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0c980a2fa0 0x7f0c980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.117+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0cb00cfa50 con 0x7f0cb8109890 2024-03-21T09:11:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.117+0000 7f0cb7400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0c980a2fa0 0x7f0c980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.118+0000 7f0cb7400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0c980a2fa0 0x7f0c980a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0ca802c640 tx=0x7f0ca8009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.120+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0cb009c490 con 0x7f0cb8109890 2024-03-21T09:11:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.381+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0cb8062340 con 0x7f0cb8109890 2024-03-21T09:11:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.382+0000 7f0cafe00640 1 -- 172.21.15.37:0/207573788 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0cb009bd90 con 0x7f0cb8109890 2024-03-21T09:11:00.376 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.384+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0c980a2fa0 msgr2=0x7f0c980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0c980a2fa0 0x7f0c980a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0ca802c640 tx=0x7f0ca8009360 comp rx=0 tx=0).stop 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 msgr2=0x7f0cb81a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb81a55c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000e9b0 tx=0x7f0cb000ee70 comp rx=0 tx=0).stop 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 shutdown_connections 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0c980a2fa0 0x7f0c980a5460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb8109890 0x7f0cb81a55c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cb8108ed0 0x7f0cb819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 --2- 172.21.15.37:0/207573788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb8107ce0 0x7f0cb819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 >> 172.21.15.37:0/207573788 conn(0x7f0cb8076c90 msgr2=0x7f0cb810c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 shutdown_connections 2024-03-21T09:11:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.385+0000 7f0cbe400640 1 -- 172.21.15.37:0/207573788 wait complete. 2024-03-21T09:11:00.397 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 -- 172.21.15.37:0/2941480007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc1003f0 msgr2=0x7f6ecc100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 --2- 172.21.15.37:0/2941480007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc100810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009900 tx=0x7f6ebc02bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 -- 172.21.15.37:0/2941480007 shutdown_connections 2024-03-21T09:11:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 --2- 172.21.15.37:0/2941480007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc101fa0 0x7f6ecc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 --2- 172.21.15.37:0/2941480007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 0x7f6ecc101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 --2- 172.21.15.37:0/2941480007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc100810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 -- 172.21.15.37:0/2941480007 >> 172.21.15.37:0/2941480007 conn(0x7f6ecc0fb980 msgr2=0x7f6ecc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 -- 172.21.15.37:0/2941480007 shutdown_connections 2024-03-21T09:11:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.616+0000 7f6ed1200640 1 -- 172.21.15.37:0/2941480007 wait complete. 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.617+0000 7f6ed1200640 1 Processor -- start 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 -- start start 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 0x7f6ecc1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc101fa0 0x7f6ecc102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc10c080 con 0x7f6ecc101fa0 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc10bf00 con 0x7f6ecc1003f0 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ed1200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc10c200 con 0x7f6ecc1015e0 2024-03-21T09:11:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ecaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 0x7f6ecc1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc104330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46626/0 (socket says 172.21.15.37:46626) 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ecb400640 1 -- 172.21.15.37:0/358594780 learned_addr learned my addr 172.21.15.37:0/358594780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ecbe00640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc101fa0 0x7f6ecc102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.618+0000 7f6ecb400640 1 -- 172.21.15.37:0/358594780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 msgr2=0x7f6ecc1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecb400640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 0x7f6ecc1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecb400640 1 -- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc101fa0 msgr2=0x7f6ecc102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecb400640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc101fa0 0x7f6ecc102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecb400640 1 -- 172.21.15.37:0/358594780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc1037c0 con 0x7f6ecc1003f0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecbe00640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc101fa0 0x7f6ecc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecaa00640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 0x7f6ecc1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ecb400640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc104330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc000c00 tx=0x7f6ebc02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc041070 con 0x7f6ecc1003f0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc004450 con 0x7f6ecc1003f0 2024-03-21T09:11:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc034440 con 0x7f6ecc1003f0 2024-03-21T09:11:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc1a18f0 con 0x7f6ecc1003f0 2024-03-21T09:11:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.619+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc1a1e00 con 0x7f6ecc1003f0 2024-03-21T09:11:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.621+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc1a2030 con 0x7f6ecc1003f0 2024-03-21T09:11:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.622+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ebc044070 con 0x7f6ecc1003f0 2024-03-21T09:11:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.623+0000 7f6ec3e00640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6eb00a2fa0 0x7f6eb00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.623+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ebc0f12b0 con 0x7f6ecc1003f0 2024-03-21T09:11:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.624+0000 7f6ecaa00640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6eb00a2fa0 0x7f6eb00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.624+0000 7f6ecaa00640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6eb00a2fa0 0x7f6eb00a5460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc1040a0 tx=0x7f6eb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.627+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ebc0bdd80 con 0x7f6ecc1003f0 2024-03-21T09:11:00.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3142161486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:00.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/207573788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:00.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.829+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6ecc1a2200 con 0x7f6ecc1003f0 2024-03-21T09:11:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.830+0000 7f6ec3e00640 1 -- 172.21.15.37:0/358594780 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6ebc0bd680 con 0x7f6ecc1003f0 2024-03-21T09:11:00.824 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:00.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.832+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6eb00a2fa0 msgr2=0x7f6eb00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.832+0000 7f6ed1200640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6eb00a2fa0 0x7f6eb00a5460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc1040a0 tx=0x7f6eb8005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.832+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 msgr2=0x7f6ecc104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.832+0000 7f6ed1200640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc104330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc000c00 tx=0x7f6ebc02c650 comp rx=0 tx=0).stop 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 shutdown_connections 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6eb00a2fa0 0x7f6eb00a5460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc101fa0 0x7f6ecc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc1015e0 0x7f6ecc1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 --2- 172.21.15.37:0/358594780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ecc1003f0 0x7f6ecc104330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 >> 172.21.15.37:0/358594780 conn(0x7f6ecc0fb980 msgr2=0x7f6ecc0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 shutdown_connections 2024-03-21T09:11:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:00.833+0000 7f6ed1200640 1 -- 172.21.15.37:0/358594780 wait complete. 2024-03-21T09:11:00.848 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:00.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3142161486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:00.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/207573788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:00.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3142161486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:00.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/207573788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.075+0000 7f869da00640 1 -- 172.21.15.37:0/3749345733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698104b50 msgr2=0x7f8698104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.075+0000 7f869da00640 1 --2- 172.21.15.37:0/3749345733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698104b50 0x7f8698104fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8684009900 tx=0x7f868402bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 -- 172.21.15.37:0/3749345733 shutdown_connections 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 --2- 172.21.15.37:0/3749345733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698105510 0x7f869810bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 --2- 172.21.15.37:0/3749345733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698104b50 0x7f8698104fd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 --2- 172.21.15.37:0/3749345733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8698103960 0x7f8698103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 -- 172.21.15.37:0/3749345733 >> 172.21.15.37:0/3749345733 conn(0x7f86980feeb0 msgr2=0x7f8698101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 -- 172.21.15.37:0/3749345733 shutdown_connections 2024-03-21T09:11:01.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.076+0000 7f869da00640 1 -- 172.21.15.37:0/3749345733 wait complete. 2024-03-21T09:11:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.078+0000 7f869da00640 1 Processor -- start 2024-03-21T09:11:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.078+0000 7f869da00640 1 -- start start 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.078+0000 7f869da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8698103960 0x7f8698199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f869da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698104b50 0x7f869819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f869da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698105510 0x7f86981a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f869da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869810f570 con 0x7f8698103960 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f869da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869810f3f0 con 0x7f8698105510 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f869da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869810f6f0 con 0x7f8698104b50 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8696000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698104b50 0x7f869819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8697400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698105510 0x7f86981a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8696000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698104b50 0x7f869819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48664/0 (socket says 172.21.15.37:48664) 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8696000640 1 -- 172.21.15.37:0/676029516 learned_addr learned my addr 172.21.15.37:0/676029516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8696a00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8698103960 0x7f8698199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8697400640 1 -- 172.21.15.37:0/676029516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698104b50 msgr2=0x7f869819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8697400640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698104b50 0x7f869819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8697400640 1 -- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8698103960 msgr2=0x7f8698199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8697400640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8698103960 0x7f8698199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.079+0000 7f8697400640 1 -- 172.21.15.37:0/676029516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86981a17a0 con 0x7f8698105510 2024-03-21T09:11:01.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.080+0000 7f8697400640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698105510 0x7f86981a1130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8690009fd0 tx=0x7f869000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:01.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.080+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000ce20 con 0x7f8698105510 2024-03-21T09:11:01.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.080+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8690005020 con 0x7f8698105510 2024-03-21T09:11:01.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.080+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690010720 con 0x7f8698105510 2024-03-21T09:11:01.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.080+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86981a1a90 con 0x7f8698105510 2024-03-21T09:11:01.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.080+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86981a1f60 con 0x7f8698105510 2024-03-21T09:11:01.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.081+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869810ac80 con 0x7f8698105510 2024-03-21T09:11:01.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.083+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8690005540 con 0x7f8698105510 2024-03-21T09:11:01.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.085+0000 7f868f400640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f867c0a2fa0 0x7f867c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.085+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f86900cfde0 con 0x7f8698105510 2024-03-21T09:11:01.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.085+0000 7f8696a00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f867c0a2fa0 0x7f867c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.086+0000 7f8696a00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f867c0a2fa0 0x7f867c0a5460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8688005fd0 tx=0x7f8688005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:01.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.089+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f869009c820 con 0x7f8698105510 2024-03-21T09:11:01.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.333+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f869802d150 con 0x7f8698105510 2024-03-21T09:11:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.334+0000 7f868f400640 1 -- 172.21.15.37:0/676029516 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f869009c120 con 0x7f8698105510 2024-03-21T09:11:01.328 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:01.328 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:09:20.795215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:11:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f867c0a2fa0 msgr2=0x7f867c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f867c0a2fa0 0x7f867c0a5460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8688005fd0 tx=0x7f8688005850 comp rx=0 tx=0).stop 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698105510 msgr2=0x7f86981a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698105510 0x7f86981a1130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8690009fd0 tx=0x7f869000ef10 comp rx=0 tx=0).stop 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 shutdown_connections 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f867c0a2fa0 0x7f867c0a5460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8698105510 0x7f86981a1130 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8698104b50 0x7f869819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 --2- 172.21.15.37:0/676029516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8698103960 0x7f8698199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 >> 172.21.15.37:0/676029516 conn(0x7f86980feeb0 msgr2=0x7f8698107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:01.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 shutdown_connections 2024-03-21T09:11:01.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.337+0000 7f869da00640 1 -- 172.21.15.37:0/676029516 wait complete. 2024-03-21T09:11:01.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:01.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 -- 172.21.15.37:0/1176630970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 msgr2=0x7fd978073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 --2- 172.21.15.37:0/1176630970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd978073ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd968009be0 tx=0x7fd96802bf80 comp rx=0 tx=0).stop 2024-03-21T09:11:01.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 -- 172.21.15.37:0/1176630970 shutdown_connections 2024-03-21T09:11:01.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 --2- 172.21.15.37:0/1176630970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd978109450 0x7fd97810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 --2- 172.21.15.37:0/1176630970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9780740a0 0x7fd978074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 --2- 172.21.15.37:0/1176630970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd978073ad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 -- 172.21.15.37:0/1176630970 >> 172.21.15.37:0/1176630970 conn(0x7fd9780fcb30 msgr2=0x7fd9780fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:01.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 -- 172.21.15.37:0/1176630970 shutdown_connections 2024-03-21T09:11:01.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.577+0000 7fd977200640 1 -- 172.21.15.37:0/1176630970 wait complete. 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.578+0000 7fd977200640 1 Processor -- start 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.578+0000 7fd977200640 1 -- start start 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd977200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9780740a0 0x7fd978199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd977200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd97819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd977200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd978109450 0x7fd9781a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd977200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97810f480 con 0x7fd9780740a0 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd977200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97810f300 con 0x7fd978109450 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd977200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97810f600 con 0x7fd978075680 2024-03-21T09:11:01.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd97819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd97819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48674/0 (socket says 172.21.15.37:48674) 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 -- 172.21.15.37:0/1889454756 learned_addr learned my addr 172.21.15.37:0/1889454756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975e00640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9780740a0 0x7fd978199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd976800640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd978109450 0x7fd9781a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 -- 172.21.15.37:0/1889454756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd978109450 msgr2=0x7fd9781a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd978109450 0x7fd9781a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 -- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9780740a0 msgr2=0x7fd978199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9780740a0 0x7fd978199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd975400640 1 -- 172.21.15.37:0/1889454756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9781a16e0 con 0x7fd978075680 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.579+0000 7fd976800640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd978109450 0x7fd9781a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.580+0000 7fd975400640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd97819a000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd96000e9f0 tx=0x7fd96000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.580+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96000cd80 con 0x7fd978075680 2024-03-21T09:11:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.580+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd960005500 con 0x7fd978075680 2024-03-21T09:11:01.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.580+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960010690 con 0x7fd978075680 2024-03-21T09:11:01.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.580+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9781a19d0 con 0x7fd978075680 2024-03-21T09:11:01.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.580+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9781a1ed0 con 0x7fd978075680 2024-03-21T09:11:01.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.582+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97804f470 con 0x7fd978075680 2024-03-21T09:11:01.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.583+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd9600046c0 con 0x7fd978075680 2024-03-21T09:11:01.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.584+0000 7fd96ea00640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd95c0a2f90 0x7fd95c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.584+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd9600cfbf0 con 0x7fd978075680 2024-03-21T09:11:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.584+0000 7fd975e00640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd95c0a2f90 0x7fd95c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.585+0000 7fd975e00640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd95c0a2f90 0x7fd95c0a5450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd968000f80 tx=0x7fd968005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.588+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd96009c630 con 0x7fd978075680 2024-03-21T09:11:01.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:01 smithi037 ceph-mon[31830]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T09:11:01.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:01 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/358594780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:01.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:01 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/676029516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:01.753 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:01 smithi071 ceph-mon[36597]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T09:11:01.754 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:01 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/358594780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:01.754 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:01 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/676029516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:01.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.795+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9780622a0 con 0x7fd978075680 2024-03-21T09:11:01.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.797+0000 7fd96ea00640 1 -- 172.21.15.37:0/1889454756 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7fd96009bf30 con 0x7fd978075680 2024-03-21T09:11:01.791 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:01.791 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:01.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd95c0a2f90 msgr2=0x7fd95c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd95c0a2f90 0x7fd95c0a5450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd968000f80 tx=0x7fd968005fd0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 msgr2=0x7fd97819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:01.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd97819a000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd96000e9f0 tx=0x7fd96000edf0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 shutdown_connections 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd95c0a2f90 0x7fd95c0a5450 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd978109450 0x7fd9781a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd978075680 0x7fd97819a000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 --2- 172.21.15.37:0/1889454756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9780740a0 0x7fd978199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.799+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 >> 172.21.15.37:0/1889454756 conn(0x7fd9780fcb30 msgr2=0x7fd9781053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.800+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 shutdown_connections 2024-03-21T09:11:01.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:01.800+0000 7fd977200640 1 -- 172.21.15.37:0/1889454756 wait complete. 2024-03-21T09:11:01.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:11:01.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:01 smithi053 ceph-mon[36169]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T09:11:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:01 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/358594780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:01.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:01 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/676029516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:02.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 -- 172.21.15.37:0/23307860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc1038d0 msgr2=0x7f35dc103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 --2- 172.21.15.37:0/23307860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc1038d0 0x7f35dc103cf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009900 tx=0x7f35cc02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:02.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 -- 172.21.15.37:0/23307860 shutdown_connections 2024-03-21T09:11:02.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 --2- 172.21.15.37:0/23307860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc105480 0x7f35dc10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 --2- 172.21.15.37:0/23307860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35dc104ac0 0x7f35dc104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 --2- 172.21.15.37:0/23307860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc1038d0 0x7f35dc103cf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 -- 172.21.15.37:0/23307860 >> 172.21.15.37:0/23307860 conn(0x7f35dc0fee60 msgr2=0x7f35dc101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:02.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 -- 172.21.15.37:0/23307860 shutdown_connections 2024-03-21T09:11:02.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.047+0000 7f35e2200640 1 -- 172.21.15.37:0/23307860 wait complete. 2024-03-21T09:11:02.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.049+0000 7f35e2200640 1 Processor -- start 2024-03-21T09:11:02.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.049+0000 7f35e2200640 1 -- start start 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35e2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 0x7f35dc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35e2200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 0x7f35dc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35e2200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35dc105480 0x7f35dc1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35e2200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc10f570 con 0x7f35dc105480 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35e2200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc10f3f0 con 0x7f35dc104ac0 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35e2200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc10f6f0 con 0x7f35dc1038d0 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 0x7f35dc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35daa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 0x7f35dc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 0x7f35dc199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48692/0 (socket says 172.21.15.37:48692) 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35daa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 0x7f35dc19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46688/0 (socket says 172.21.15.37:46688) 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 -- 172.21.15.37:0/1092904912 learned_addr learned my addr 172.21.15.37:0/1092904912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35dbe00640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35dc105480 0x7f35dc1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 -- 172.21.15.37:0/1092904912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 msgr2=0x7f35dc19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 0x7f35dc19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 -- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35dc105480 msgr2=0x7f35dc1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35dc105480 0x7f35dc1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35db400640 1 -- 172.21.15.37:0/1092904912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc1a1800 con 0x7f35dc1038d0 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.050+0000 7f35daa00640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 0x7f35dc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:02.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.051+0000 7f35db400640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 0x7f35dc199be0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000c00 tx=0x7f35cc02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:02.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.051+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc043020 con 0x7f35dc1038d0 2024-03-21T09:11:02.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.051+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35cc0054e0 con 0x7f35dc1038d0 2024-03-21T09:11:02.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.051+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc1a1a90 con 0x7f35dc1038d0 2024-03-21T09:11:02.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.051+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc003ce0 con 0x7f35dc1038d0 2024-03-21T09:11:02.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.051+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1a2000 con 0x7f35dc1038d0 2024-03-21T09:11:02.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.053+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc068970 con 0x7f35dc1038d0 2024-03-21T09:11:02.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.054+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f35cc046020 con 0x7f35dc1038d0 2024-03-21T09:11:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.055+0000 7f35d3e00640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f35c00a2fa0 0x7f35c00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.055+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35cc0f1820 con 0x7f35dc1038d0 2024-03-21T09:11:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.056+0000 7f35daa00640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f35c00a2fa0 0x7f35c00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.056+0000 7f35daa00640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f35c00a2fa0 0x7f35c00a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35dc19b0f0 tx=0x7f35c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:02.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.059+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35cc04d020 con 0x7f35dc1038d0 2024-03-21T09:11:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.282+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35dc0622a0 con 0x7f35dc1038d0 2024-03-21T09:11:02.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.283+0000 7f35d3e00640 1 -- 172.21.15.37:0/1092904912 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35cc0be260 con 0x7f35dc1038d0 2024-03-21T09:11:02.277 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:11:02.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f35c00a2fa0 msgr2=0x7f35c00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f35c00a2fa0 0x7f35c00a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35dc19b0f0 tx=0x7f35c800b390 comp rx=0 tx=0).stop 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 msgr2=0x7f35dc199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 0x7f35dc199be0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000c00 tx=0x7f35cc02c6c0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 shutdown_connections 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f35c00a2fa0 0x7f35c00a5460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35dc105480 0x7f35dc1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35dc104ac0 0x7f35dc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 --2- 172.21.15.37:0/1092904912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35dc1038d0 0x7f35dc199be0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 >> 172.21.15.37:0/1092904912 conn(0x7f35dc0fee60 msgr2=0x7f35dc107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 shutdown_connections 2024-03-21T09:11:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.286+0000 7f35e2200640 1 -- 172.21.15.37:0/1092904912 wait complete. 2024-03-21T09:11:02.301 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 -- 172.21.15.37:0/3328884812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac101f70 msgr2=0x7f27ac104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3328884812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac101f70 0x7f27ac104360 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009900 tx=0x7f27a002bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 -- 172.21.15.37:0/3328884812 shutdown_connections 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3328884812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac1071d0 0x7f27ac109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3328884812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ac1048a0 0x7f27ac106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3328884812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac101f70 0x7f27ac104360 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.529+0000 7f27b2a00640 1 -- 172.21.15.37:0/3328884812 >> 172.21.15.37:0/3328884812 conn(0x7f27ac0fb980 msgr2=0x7f27ac0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.530+0000 7f27b2a00640 1 -- 172.21.15.37:0/3328884812 shutdown_connections 2024-03-21T09:11:02.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.530+0000 7f27b2a00640 1 -- 172.21.15.37:0/3328884812 wait complete. 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.531+0000 7f27b2a00640 1 Processor -- start 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.531+0000 7f27b2a00640 1 -- start start 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 0x7f27ac1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ac1048a0 0x7f27ac0fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac1071d0 0x7f27ac100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac10d1d0 con 0x7f27ac1048a0 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac10d050 con 0x7f27ac1071d0 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac10d350 con 0x7f27ac101f70 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 0x7f27ac1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 0x7f27ac1018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48712/0 (socket says 172.21.15.37:48712) 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b1600640 1 -- 172.21.15.37:0/3933604762 learned_addr learned my addr 172.21.15.37:0/3933604762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b0c00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ac1048a0 0x7f27ac0fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.532+0000 7f27b2000640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac1071d0 0x7f27ac100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b1600640 1 -- 172.21.15.37:0/3933604762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac1071d0 msgr2=0x7f27ac100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b1600640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac1071d0 0x7f27ac100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b1600640 1 -- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ac1048a0 msgr2=0x7f27ac0fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b1600640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ac1048a0 0x7f27ac0fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b1600640 1 -- 172.21.15.37:0/3933604762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac100cf0 con 0x7f27ac101f70 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b1600640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 0x7f27ac1018e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005ab0 tx=0x7f27a002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0043020 con 0x7f27ac101f70 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27a00054e0 con 0x7f27ac101f70 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0003ce0 con 0x7f27ac101f70 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.533+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac1a1910 con 0x7f27ac101f70 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.534+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac1a1eb0 con 0x7f27ac101f70 2024-03-21T09:11:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.535+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0fd470 con 0x7f27ac101f70 2024-03-21T09:11:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.537+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f27a0046020 con 0x7f27ac101f70 2024-03-21T09:11:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.538+0000 7f27aa000640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27940a2fa0 0x7f27940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.538+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f27a00f1ac0 con 0x7f27ac101f70 2024-03-21T09:11:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.538+0000 7f27b0c00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27940a2fa0 0x7f27940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.539+0000 7f27b0c00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27940a2fa0 0x7f27940a5460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f27ac101650 tx=0x7f279800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.540+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f27a004d020 con 0x7f27ac101f70 2024-03-21T09:11:02.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1889454756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:02.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:02 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1092904912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:02.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.756+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27ac02d200 con 0x7f27ac101f70 2024-03-21T09:11:02.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.757+0000 7f27aa000640 1 -- 172.21.15.37:0/3933604762 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27a00be500 con 0x7f27ac101f70 2024-03-21T09:11:02.750 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27940a2fa0 msgr2=0x7f27940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27940a2fa0 0x7f27940a5460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f27ac101650 tx=0x7f279800b390 comp rx=0 tx=0).stop 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 msgr2=0x7f27ac1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 0x7f27ac1018e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005ab0 tx=0x7f27a002c6c0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 shutdown_connections 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f27940a2fa0 0x7f27940a5460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27ac1071d0 0x7f27ac100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ac1048a0 0x7f27ac0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 --2- 172.21.15.37:0/3933604762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27ac101f70 0x7f27ac1018e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 >> 172.21.15.37:0/3933604762 conn(0x7f27ac0fb980 msgr2=0x7f27ac102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:02.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 shutdown_connections 2024-03-21T09:11:02.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:02.759+0000 7f27b2a00640 1 -- 172.21.15.37:0/3933604762 wait complete. 2024-03-21T09:11:02.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:02.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:02 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1889454756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:02.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:02 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1092904912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:02.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:02 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1889454756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:02.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:02 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1092904912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:02.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 -- 172.21.15.37:0/485927020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 msgr2=0x7fd8980744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:02.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 --2- 172.21.15.37:0/485927020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd8980744e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd880009900 tx=0x7fd88002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:11:02.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 -- 172.21.15.37:0/485927020 shutdown_connections 2024-03-21T09:11:02.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 --2- 172.21.15.37:0/485927020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd898109560 0x7fd89810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 --2- 172.21.15.37:0/485927020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd8980744e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 --2- 172.21.15.37:0/485927020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd898075660 0x7fd898073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 -- 172.21.15.37:0/485927020 >> 172.21.15.37:0/485927020 conn(0x7fd8980fcc20 msgr2=0x7fd8980ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 -- 172.21.15.37:0/485927020 shutdown_connections 2024-03-21T09:11:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.005+0000 7fd89f200640 1 -- 172.21.15.37:0/485927020 wait complete. 2024-03-21T09:11:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.007+0000 7fd89f200640 1 Processor -- start 2024-03-21T09:11:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.007+0000 7fd89f200640 1 -- start start 2024-03-21T09:11:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd898199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd898075660 0x7fd89819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd898109560 0x7fd8981a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89810f560 con 0x7fd898109560 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89810f3e0 con 0x7fd898075660 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89810f6e0 con 0x7fd898074080 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd898199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd898199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48724/0 (socket says 172.21.15.37:48724) 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 -- 172.21.15.37:0/1524551165 learned_addr learned my addr 172.21.15.37:0/1524551165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd88f400640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd898075660 0x7fd89819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd89d200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd898109560 0x7fd8981a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 -- 172.21.15.37:0/1524551165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd898075660 msgr2=0x7fd89819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd898075660 0x7fd89819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 -- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd898109560 msgr2=0x7fd8981a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd898109560 0x7fd8981a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.008+0000 7fd897e00640 1 -- 172.21.15.37:0/1524551165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8981a17d0 con 0x7fd898074080 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.009+0000 7fd897e00640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd898199bb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd88800e9f0 tx=0x7fd88800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.009+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88800cd80 con 0x7fd898074080 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.009+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd888010910 con 0x7fd898074080 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.009+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8981a1ac0 con 0x7fd898074080 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.009+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888018990 con 0x7fd898074080 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.009+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8981a2030 con 0x7fd898074080 2024-03-21T09:11:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.010+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd898108320 con 0x7fd898074080 2024-03-21T09:11:03.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.012+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd888010ac0 con 0x7fd898074080 2024-03-21T09:11:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.013+0000 7fd895600640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd87c0a2fa0 0x7fd87c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.013+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8880cfff0 con 0x7fd898074080 2024-03-21T09:11:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.014+0000 7fd88f400640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd87c0a2fa0 0x7fd87c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.014+0000 7fd88f400640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd87c0a2fa0 0x7fd87c0a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd88002c2d0 tx=0x7fd88002c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:03.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.016+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd88809ca70 con 0x7fd898074080 2024-03-21T09:11:03.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.237+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd89802d300 con 0x7fd898074080 2024-03-21T09:11:03.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.238+0000 7fd895600640 1 -- 172.21.15.37:0/1524551165 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd88809c370 con 0x7fd898074080 2024-03-21T09:11:03.231 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:03.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd87c0a2fa0 msgr2=0x7fd87c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd87c0a2fa0 0x7fd87c0a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd88002c2d0 tx=0x7fd88002c910 comp rx=0 tx=0).stop 2024-03-21T09:11:03.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 msgr2=0x7fd898199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd898199bb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd88800e9f0 tx=0x7fd88800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 shutdown_connections 2024-03-21T09:11:03.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd87c0a2fa0 0x7fd87c0a5460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd898109560 0x7fd8981a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd898075660 0x7fd89819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 --2- 172.21.15.37:0/1524551165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898074080 0x7fd898199bb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.240+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 >> 172.21.15.37:0/1524551165 conn(0x7fd8980fcc20 msgr2=0x7fd8981054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.241+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 shutdown_connections 2024-03-21T09:11:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.241+0000 7fd89f200640 1 -- 172.21.15.37:0/1524551165 wait complete. 2024-03-21T09:11:03.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 -- 172.21.15.37:0/2144831915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084102140 msgr2=0x7f0084104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 --2- 172.21.15.37:0/2144831915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084102140 0x7f0084104530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0074009900 tx=0x7f007402bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 -- 172.21.15.37:0/2144831915 shutdown_connections 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 --2- 172.21.15.37:0/2144831915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00841073a0 0x7f00841097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 --2- 172.21.15.37:0/2144831915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0084104a70 0x7f0084106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 --2- 172.21.15.37:0/2144831915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084102140 0x7f0084104530 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 -- 172.21.15.37:0/2144831915 >> 172.21.15.37:0/2144831915 conn(0x7f00840fbb30 msgr2=0x7f00840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:03.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 -- 172.21.15.37:0/2144831915 shutdown_connections 2024-03-21T09:11:03.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.488+0000 7f0088c00640 1 -- 172.21.15.37:0/2144831915 wait complete. 2024-03-21T09:11:03.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.489+0000 7f0088c00640 1 Processor -- start 2024-03-21T09:11:03.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.489+0000 7f0088c00640 1 -- start start 2024-03-21T09:11:03.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0088c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0084102140 0x7f0084101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0088c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 0x7f00841000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0088c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00841073a0 0x7f0084100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0088c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008410d3c0 con 0x7f0084102140 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0088c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008410d240 con 0x7f00841073a0 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0088c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008410d540 con 0x7f0084104a70 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 0x7f00841000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 0x7f00841000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48752/0 (socket says 172.21.15.37:48752) 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 -- 172.21.15.37:0/1356590974 learned_addr learned my addr 172.21.15.37:0/1356590974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0082800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00841073a0 0x7f0084100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081e00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0084102140 0x7f0084101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 -- 172.21.15.37:0/1356590974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00841073a0 msgr2=0x7f0084100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00841073a0 0x7f0084100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 -- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0084102140 msgr2=0x7f0084101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.490+0000 7f0081400640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0084102140 0x7f0084101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f0081400640 1 -- 172.21.15.37:0/1356590974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084100ec0 con 0x7f0084104a70 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f0081e00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0084102140 0x7f0084101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:03.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f0082800640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00841073a0 0x7f0084100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:03.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f0081400640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 0x7f00841000f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f007000e9f0 tx=0x7f007000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:03.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000ccc0 con 0x7f0084104a70 2024-03-21T09:11:03.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0070005500 con 0x7f0084104a70 2024-03-21T09:11:03.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00841a1b90 con 0x7f0084104a70 2024-03-21T09:11:03.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070010690 con 0x7f0084104a70 2024-03-21T09:11:03.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.491+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00841a20e0 con 0x7f0084104a70 2024-03-21T09:11:03.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.493+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00840fd620 con 0x7f0084104a70 2024-03-21T09:11:03.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.496+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f00700046c0 con 0x7f0084104a70 2024-03-21T09:11:03.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.497+0000 7f007aa00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f00680a2fa0 0x7f00680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.497+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f00700cfbb0 con 0x7f0084104a70 2024-03-21T09:11:03.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.498+0000 7f0081e00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f00680a2fa0 0x7f00680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.498+0000 7f0081e00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f00680a2fa0 0x7f00680a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f007402c240 tx=0x7f007402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:03.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.501+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f007009c5f0 con 0x7f0084104a70 2024-03-21T09:11:03.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:03 smithi037 ceph-mon[31830]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-21T09:11:03.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:03 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3933604762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:03.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:03 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1524551165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:03.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.724+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f00840622a0 con 0x7f0084104a70 2024-03-21T09:11:03.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.725+0000 7f007aa00640 1 -- 172.21.15.37:0/1356590974 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f007009bef0 con 0x7f0084104a70 2024-03-21T09:11:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:03.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.727+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f00680a2fa0 msgr2=0x7f00680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.727+0000 7f0088c00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f00680a2fa0 0x7f00680a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f007402c240 tx=0x7f007402df90 comp rx=0 tx=0).stop 2024-03-21T09:11:03.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.727+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 msgr2=0x7f00841000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.727+0000 7f0088c00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 0x7f00841000f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f007000e9f0 tx=0x7f007000edf0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 shutdown_connections 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f00680a2fa0 0x7f00680a5460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00841073a0 0x7f0084100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0084104a70 0x7f00841000f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 --2- 172.21.15.37:0/1356590974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0084102140 0x7f0084101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 >> 172.21.15.37:0/1356590974 conn(0x7f00840fbb30 msgr2=0x7f0084102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 shutdown_connections 2024-03-21T09:11:03.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.728+0000 7f0088c00640 1 -- 172.21.15.37:0/1356590974 wait complete. 2024-03-21T09:11:03.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:03.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:03 smithi071 ceph-mon[36597]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-21T09:11:03.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:03 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3933604762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:03.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:03 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1524551165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:03.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:03 smithi053 ceph-mon[36169]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-21T09:11:03.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:03 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3933604762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:03.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:03 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1524551165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 -- 172.21.15.37:0/2084858605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 msgr2=0x7fdf90073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 --2- 172.21.15.37:0/2084858605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90073ab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf8002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 -- 172.21.15.37:0/2084858605 shutdown_connections 2024-03-21T09:11:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 --2- 172.21.15.37:0/2084858605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf90109470 0x7fdf9010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 --2- 172.21.15.37:0/2084858605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90074080 0x7fdf900744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 --2- 172.21.15.37:0/2084858605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90073ab0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 -- 172.21.15.37:0/2084858605 >> 172.21.15.37:0/2084858605 conn(0x7fdf900fcb30 msgr2=0x7fdf900fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 -- 172.21.15.37:0/2084858605 shutdown_connections 2024-03-21T09:11:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.978+0000 7fdf8f200640 1 -- 172.21.15.37:0/2084858605 wait complete. 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.979+0000 7fdf8f200640 1 Processor -- start 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 -- start start 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf90074080 0x7fdf90199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90109470 0x7fdf901a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9010f470 con 0x7fdf90074080 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9010f2f0 con 0x7fdf90075660 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9010f5f0 con 0x7fdf90109470 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf90074080 0x7fdf90199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf8e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90109470 0x7fdf901a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59884/0 (socket says 172.21.15.37:59884) 2024-03-21T09:11:03.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.980+0000 7fdf87e00640 1 -- 172.21.15.37:0/3019489342 learned_addr learned my addr 172.21.15.37:0/3019489342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf87e00640 1 -- 172.21.15.37:0/3019489342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90109470 msgr2=0x7fdf901a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf87e00640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90109470 0x7fdf901a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf87e00640 1 -- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf90074080 msgr2=0x7fdf90199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf87e00640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf90074080 0x7fdf90199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf87e00640 1 -- 172.21.15.37:0/3019489342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf901a1680 con 0x7fdf90075660 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf87e00640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90199fa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800d950 tx=0x7fdf7800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78017070 con 0x7fdf90075660 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf901a1970 con 0x7fdf90075660 2024-03-21T09:11:03.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf78009c30 con 0x7fdf90075660 2024-03-21T09:11:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.981+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800f6b0 con 0x7fdf90075660 2024-03-21T09:11:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.982+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf901a1ec0 con 0x7fdf90075660 2024-03-21T09:11:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.983+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf90108230 con 0x7fdf90075660 2024-03-21T09:11:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.985+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdf780109d0 con 0x7fdf90075660 2024-03-21T09:11:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.986+0000 7fdf86a00640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdf740a2f10 0x7fdf740a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.986+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdf780cf3d0 con 0x7fdf90075660 2024-03-21T09:11:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.986+0000 7fdf8de00640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdf740a2f10 0x7fdf740a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.987+0000 7fdf8de00640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdf740a2f10 0x7fdf740a53d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf8002c2d0 tx=0x7fdf8002c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:03.990+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf78009de0 con 0x7fdf90075660 2024-03-21T09:11:04.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.239+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf9002d070 con 0x7fdf90075660 2024-03-21T09:11:04.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.240+0000 7fdf86a00640 1 -- 172.21.15.37:0/3019489342 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fdf7809b7c0 con 0x7fdf90075660 2024-03-21T09:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:09:20.795215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:11:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdf740a2f10 msgr2=0x7fdf740a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdf740a2f10 0x7fdf740a53d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf8002c2d0 tx=0x7fdf8002c910 comp rx=0 tx=0).stop 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 msgr2=0x7fdf90199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90199fa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800d950 tx=0x7fdf7800de10 comp rx=0 tx=0).stop 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 shutdown_connections 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdf740a2f10 0x7fdf740a53d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90109470 0x7fdf901a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90075660 0x7fdf90199fa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 --2- 172.21.15.37:0/3019489342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf90074080 0x7fdf90199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 >> 172.21.15.37:0/3019489342 conn(0x7fdf900fcb30 msgr2=0x7fdf90105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:04.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.243+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 shutdown_connections 2024-03-21T09:11:04.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.244+0000 7fdf8f200640 1 -- 172.21.15.37:0/3019489342 wait complete. 2024-03-21T09:11:04.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:04.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.476+0000 7f89ee400640 1 -- 172.21.15.37:0/1779631188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8102260 msgr2=0x7f89e8104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 --2- 172.21.15.37:0/1779631188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8102260 0x7f89e8104650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009900 tx=0x7f89d802bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 -- 172.21.15.37:0/1779631188 shutdown_connections 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 --2- 172.21.15.37:0/1779631188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e81074c0 0x7f89e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 --2- 172.21.15.37:0/1779631188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e8104b90 0x7f89e8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 --2- 172.21.15.37:0/1779631188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8102260 0x7f89e8104650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 -- 172.21.15.37:0/1779631188 >> 172.21.15.37:0/1779631188 conn(0x7f89e80fbbd0 msgr2=0x7f89e80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 -- 172.21.15.37:0/1779631188 shutdown_connections 2024-03-21T09:11:04.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.477+0000 7f89ee400640 1 -- 172.21.15.37:0/1779631188 wait complete. 2024-03-21T09:11:04.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.478+0000 7f89ee400640 1 Processor -- start 2024-03-21T09:11:04.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.478+0000 7f89ee400640 1 -- start start 2024-03-21T09:11:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89ee400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 0x7f89e8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89ee400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8104b90 0x7f89e8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89ee400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e81074c0 0x7f89e819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89ee400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e810d390 con 0x7f89e81074c0 2024-03-21T09:11:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89ee400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e810d210 con 0x7f89e8102260 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89ee400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e810d510 con 0x7f89e8104b90 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 0x7f89e8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8104b90 0x7f89e8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 0x7f89e8197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59912/0 (socket says 172.21.15.37:59912) 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89e7400640 1 -- 172.21.15.37:0/3620354875 learned_addr learned my addr 172.21.15.37:0/3620354875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89e7e00640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e81074c0 0x7f89e819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.479+0000 7f89e7400640 1 -- 172.21.15.37:0/3620354875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8104b90 msgr2=0x7f89e8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e7400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8104b90 0x7f89e8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e7400640 1 -- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e81074c0 msgr2=0x7f89e819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e7400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e81074c0 0x7f89e819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e7400640 1 -- 172.21.15.37:0/3620354875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e819f5b0 con 0x7f89e8102260 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e7e00640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e81074c0 0x7f89e819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e6a00640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8104b90 0x7f89e8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89e7400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 0x7f89e8197990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89d8000c00 tx=0x7f89d802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8043020 con 0x7f89e8102260 2024-03-21T09:11:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89d80054e0 con 0x7f89e8102260 2024-03-21T09:11:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003c40 con 0x7f89e8102260 2024-03-21T09:11:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.480+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e819f840 con 0x7f89e8102260 2024-03-21T09:11:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.481+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e819fd90 con 0x7f89e8102260 2024-03-21T09:11:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.482+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e80fd6c0 con 0x7f89e8102260 2024-03-21T09:11:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.483+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f89d8046020 con 0x7f89e8102260 2024-03-21T09:11:04.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.486+0000 7f89dfe00640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f89cc0a2f20 0x7f89cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.486+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f89d80f18b0 con 0x7f89e8102260 2024-03-21T09:11:04.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.486+0000 7f89e6a00640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f89cc0a2f20 0x7f89cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.487+0000 7f89e6a00640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f89cc0a2f20 0x7f89cc0a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f89e8198ea0 tx=0x7f89d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.489+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f89d804d020 con 0x7f89e8102260 2024-03-21T09:11:04.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:04 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1356590974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:04.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:04 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3019489342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:04.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.700+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e80622a0 con 0x7f89e8102260 2024-03-21T09:11:04.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.702+0000 7f89dfe00640 1 -- 172.21.15.37:0/3620354875 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7f89d80be2f0 con 0x7f89e8102260 2024-03-21T09:11:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:04.697 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f89cc0a2f20 msgr2=0x7f89cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f89cc0a2f20 0x7f89cc0a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f89e8198ea0 tx=0x7f89d4005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 msgr2=0x7f89e8197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 0x7f89e8197990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89d8000c00 tx=0x7f89d802c650 comp rx=0 tx=0).stop 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 shutdown_connections 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f89cc0a2f20 0x7f89cc0a53e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89e81074c0 0x7f89e819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89e8104b90 0x7f89e8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 --2- 172.21.15.37:0/3620354875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89e8102260 0x7f89e8197990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 >> 172.21.15.37:0/3620354875 conn(0x7f89e80fbbd0 msgr2=0x7f89e8102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:04.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 shutdown_connections 2024-03-21T09:11:04.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.706+0000 7f89ee400640 1 -- 172.21.15.37:0/3620354875 wait complete. 2024-03-21T09:11:04.719 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:04.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:04 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1356590974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:04.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:04 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3019489342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 -- 172.21.15.37:0/3132163488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184101120 msgr2=0x7f01841015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 --2- 172.21.15.37:0/3132163488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184101120 0x7f01841015a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f017c009ca0 tx=0x7f017c02c100 comp rx=0 tx=0).stop 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 -- 172.21.15.37:0/3132163488 shutdown_connections 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 --2- 172.21.15.37:0/3132163488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184101120 0x7f01841015a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 --2- 172.21.15.37:0/3132163488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0184100780 0x7f0184100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 --2- 172.21.15.37:0/3132163488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 -- 172.21.15.37:0/3132163488 >> 172.21.15.37:0/3132163488 conn(0x7f01840fba20 msgr2=0x7f01840fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 -- 172.21.15.37:0/3132163488 shutdown_connections 2024-03-21T09:11:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.936+0000 7f0183e00640 1 -- 172.21.15.37:0/3132163488 wait complete. 2024-03-21T09:11:04.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.937+0000 7f0183e00640 1 Processor -- start 2024-03-21T09:11:04.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 -- start start 2024-03-21T09:11:04.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184100780 0x7f0184199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0184101120 0x7f01841a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018410f400 con 0x7f0184101120 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018410f280 con 0x7f0184100780 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0183e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018410f580 con 0x7f01840ffe20 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.938+0000 7f0182a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f017be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184100780 0x7f0184199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46554/0 (socket says 172.21.15.37:46554) 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 -- 172.21.15.37:0/3211213477 learned_addr learned my addr 172.21.15.37:0/3211213477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0183400640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0184101120 0x7f01841a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 -- 172.21.15.37:0/3211213477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184100780 msgr2=0x7f0184199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184100780 0x7f0184199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 -- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0184101120 msgr2=0x7f01841a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0184101120 0x7f01841a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 -- 172.21.15.37:0/3211213477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01841a16c0 con 0x7f01840ffe20 2024-03-21T09:11:04.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0182a00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184199aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f017400ea60 tx=0x7f017400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:04.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400cde0 con 0x7f01840ffe20 2024-03-21T09:11:04.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01740035f0 con 0x7f01840ffe20 2024-03-21T09:11:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174010670 con 0x7f01840ffe20 2024-03-21T09:11:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.939+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01841a19b0 con 0x7f01840ffe20 2024-03-21T09:11:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.940+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01841a1eb0 con 0x7f01840ffe20 2024-03-21T09:11:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.941+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018404f470 con 0x7f01840ffe20 2024-03-21T09:11:04.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.942+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f01740037a0 con 0x7f01840ffe20 2024-03-21T09:11:04.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.944+0000 7f017b400640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f01680a2f10 0x7f01680a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:04.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.944+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f01740cfab0 con 0x7f01840ffe20 2024-03-21T09:11:04.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.944+0000 7f017be00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f01680a2f10 0x7f01680a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:04.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.945+0000 7f017be00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f01680a2f10 0x7f01680a53d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f018419afb0 tx=0x7f016c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:04.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:04 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1356590974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:04.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:04 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3019489342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:04.947+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f017409c4f0 con 0x7f01840ffe20 2024-03-21T09:11:05.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.202+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f018410ab70 con 0x7f01840ffe20 2024-03-21T09:11:05.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.203+0000 7f017b400640 1 -- 172.21.15.37:0/3211213477 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f017409bdf0 con 0x7f01840ffe20 2024-03-21T09:11:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:05.198 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:09:20.795215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:11:05.198 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f01680a2f10 msgr2=0x7f01680a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f01680a2f10 0x7f01680a53d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f018419afb0 tx=0x7f016c00b410 comp rx=0 tx=0).stop 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 msgr2=0x7f0184199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184199aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f017400ea60 tx=0x7f017400ef20 comp rx=0 tx=0).stop 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 shutdown_connections 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f01680a2f10 0x7f01680a53d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0184101120 0x7f01841a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0184100780 0x7f0184199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 --2- 172.21.15.37:0/3211213477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01840ffe20 0x7f0184199aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 >> 172.21.15.37:0/3211213477 conn(0x7f01840fba20 msgr2=0x7f01840fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 shutdown_connections 2024-03-21T09:11:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.207+0000 7f0183e00640 1 -- 172.21.15.37:0/3211213477 wait complete. 2024-03-21T09:11:05.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T09:08:46.306108+0000', 'modified': '2024-03-21T09:09:20.795215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24589, 'mds_1': 24575, 'mds_2': 24553}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6837/3665426355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3665426355}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3665426355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6835/2027751164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 2027751164}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2027751164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6837/2290408711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 2290408711}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 2290408711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.71:6839/543433986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 543433986}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 543433986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.53:6839/380096256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 380096256}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 380096256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24589': {'gid': 24589, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6841/600888190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 600888190}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 600888190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24589, 'qdb_cluster': [24553, 24575, 24589]} 2024-03-21T09:11:05.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-21T09:11:05.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 -- 172.21.15.37:0/2951456302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 msgr2=0x7efdb0105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 --2- 172.21.15.37:0/2951456302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb0105500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efda8009ca0 tx=0x7efda802c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 -- 172.21.15.37:0/2951456302 shutdown_connections 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 --2- 172.21.15.37:0/2951456302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb0105500 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 --2- 172.21.15.37:0/2951456302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb0104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 --2- 172.21.15.37:0/2951456302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb00ffd60 0x7efdb0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 -- 172.21.15.37:0/2951456302 >> 172.21.15.37:0/2951456302 conn(0x7efdb00fb980 msgr2=0x7efdb00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 -- 172.21.15.37:0/2951456302 shutdown_connections 2024-03-21T09:11:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.444+0000 7efdb6200640 1 -- 172.21.15.37:0/2951456302 wait complete. 2024-03-21T09:11:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 Processor -- start 2024-03-21T09:11:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 -- start start 2024-03-21T09:11:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb00ffd60 0x7efdb019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb01a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0113760 con 0x7efdb00ffd60 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb01135e0 con 0x7efdb01026d0 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.445+0000 7efdb6200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb01138e0 con 0x7efdb0105080 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdb5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb01a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59946/0 (socket says 172.21.15.37:59946) 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 -- 172.21.15.37:0/1038607624 learned_addr learned my addr 172.21.15.37:0/1038607624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdb4e00640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb00ffd60 0x7efdb019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 -- 172.21.15.37:0/1038607624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 msgr2=0x7efdb01a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb01a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 -- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb00ffd60 msgr2=0x7efdb019ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb00ffd60 0x7efdb019ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 -- 172.21.15.37:0/1038607624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb01a59d0 con 0x7efdb01026d0 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdb5800640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb01a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.446+0000 7efdafe00640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb019e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e9f0 tx=0x7efd9c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.447+0000 7efdad600640 1 -- 172.21.15.37:0/1038607624 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00cd80 con 0x7efdb01026d0 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.447+0000 7efdad600640 1 -- 172.21.15.37:0/1038607624 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd9c005500 con 0x7efdb01026d0 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.447+0000 7efdad600640 1 -- 172.21.15.37:0/1038607624 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c010690 con 0x7efdb01026d0 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.447+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb01a5cc0 con 0x7efdb01026d0 2024-03-21T09:11:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.447+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb01a61c0 con 0x7efdb01026d0 2024-03-21T09:11:05.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.450+0000 7efdad600640 1 -- 172.21.15.37:0/1038607624 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efd9c0046c0 con 0x7efdb01026d0 2024-03-21T09:11:05.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.450+0000 7efdba5c9740 1 Processor -- start 2024-03-21T09:11:05.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.450+0000 7efdba5c9740 1 -- start start 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdba5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ad9bac90 con 0x55a8ada55d80 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdba5c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ad963980 con 0x55a8ada56770 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdba5c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ad9b6d30 con 0x55a8ada5b8e0 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada5b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59952/0 (socket says 172.21.15.37:59952) 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 -- 172.21.15.37:0/1938197614 learned_addr learned my addr 172.21.15.37:0/1938197614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb5800640 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52104/0 (socket says 172.21.15.37:52104) 2024-03-21T09:11:05.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdafe00640 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 -- 172.21.15.37:0/1938197614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 msgr2=0x55a8ada5dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 -- 172.21.15.37:0/1938197614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 msgr2=0x55a8ada561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.451+0000 7efdb4e00640 1 -- 172.21.15.37:0/1938197614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ad97b510 con 0x55a8ada56770 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.452+0000 7efdb4e00640 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada5b270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efda4009cf0 tx=0x7efda4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09d1031acb36948 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.452+0000 7efda2000640 1 -- 172.21.15.37:0/1938197614 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4004230 con 0x55a8ada56770 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.452+0000 7efda2000640 1 -- 172.21.15.37:0/1938197614 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efda40043d0 con 0x55a8ada56770 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.452+0000 7efda2000640 1 -- 172.21.15.37:0/1938197614 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4019960 con 0x55a8ada56770 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.452+0000 7efdad600640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd980a2fa0 0x7efd980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.452+0000 7efdad600640 1 -- 172.21.15.37:0/1038607624 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efd9c0cfb40 con 0x7efdb01026d0 2024-03-21T09:11:05.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdb5800640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd980a2fa0 0x7efd980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 -- 172.21.15.37:0/1938197614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 msgr2=0x55a8ada5b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada5b270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efda4009cf0 tx=0x7efda4011810 comp rx=0 tx=0).stop 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 -- 172.21.15.37:0/1938197614 shutdown_connections 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada5b270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 --2- 172.21.15.37:0/1938197614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 -- 172.21.15.37:0/1938197614 >> 172.21.15.37:0/1938197614 conn(0x55a8ada512f0 msgr2=0x55a8ada516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 -- 172.21.15.37:0/1938197614 shutdown_connections 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdba5c9740 1 -- 172.21.15.37:0/1938197614 wait complete. 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.453+0000 7efdb5800640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd980a2fa0 0x7efd980a5460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efda8005e30 tx=0x7efda8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 -- start start 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 Processor -- start 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada60290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada56770 0x55a8ada607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 0x55a8ada67810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada5e890 con 0x55a8ada55d80 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ad976dd0 con 0x55a8ada5b8e0 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada5ea10 con 0x55a8ada56770 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.454+0000 7efdba5c9740 10 client.0 _ll_register_callbacks cb 0x55a8ad6ceac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdb5800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 0x55a8ada67810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdb5800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 0x55a8ada67810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59966/0 (socket says 172.21.15.37:59966) 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada56770 0x55a8ada607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdb5800640 1 -- 172.21.15.37:0/103211127 learned_addr learned my addr 172.21.15.37:0/103211127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdb4e00640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada60290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 -- 172.21.15.37:0/103211127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 msgr2=0x55a8ada67810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 0x55a8ada67810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 -- 172.21.15.37:0/103211127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 msgr2=0x55a8ada60290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada60290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 -- 172.21.15.37:0/103211127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8ad6cf2d0 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdb5800640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 0x55a8ada67810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efdafe00640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada56770 0x55a8ada607d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd250 tx=0x7efd9c004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e849dbd6f0f5b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efd96a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.455+0000 7efd96a00640 1 -- 172.21.15.37:0/103211127 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c09b7e0 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efd96a00640 1 -- 172.21.15.37:0/103211127 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efd9c0a4600 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efda40043d0 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efd96a00640 1 -- 172.21.15.37:0/103211127 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd9c09c580 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 10 client.25162 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a8ad7996b0 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efd96a00640 1 -- 172.21.15.37:0/103211127 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efda40043d0 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efd96a00640 1 -- 172.21.15.37:0/103211127 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7efd9c09b460 con 0x55a8ada56770 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 10 client.25162 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 10 client.25162 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 20 client.25162 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.456+0000 7efdba5c9740 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8ada6ca80 0x55a8ada6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.457+0000 7efdba5c9740 4 client.25162 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:11:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.457+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8ad95c520 con 0x55a8ada6ca80 2024-03-21T09:11:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.457+0000 7efdb4e00640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8ada6ca80 0x55a8ada6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.457+0000 7efdb4e00640 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8ada6ca80 0x55a8ada6bb00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ec750bfe3eff8b9 server_cookie=7312713ab34768b0 in_seq=0 out_seq=0 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.457+0000 7efd96a00640 10 client.25162 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.458+0000 7efd96a00640 1 -- 172.21.15.37:0/103211127 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a8ad95c520 con 0x55a8ada6ca80 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.458+0000 7efd96a00640 10 client.25162 handle_command_reply: tid=0 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.458+0000 7efdba5c9740 1 client.25162 shutdown 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada56770 msgr2=0x55a8ada607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada56770 0x55a8ada607d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd250 tx=0x7efd9c004ba0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 shutdown_connections 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada5b8e0 0x55a8ada67810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada56770 0x55a8ada607d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8ada6ca80 0x55a8ada6bb00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 --2- 172.21.15.37:0/103211127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada60290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efd96a00640 0 client.25162 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 >> 172.21.15.37:0/103211127 conn(0x55a8ada512f0 msgr2=0x55a8ada504f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 shutdown_connections 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 1 -- 172.21.15.37:0/103211127 wait complete. 2024-03-21T09:11:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.459+0000 7efdba5c9740 20 client.25162 trim_cache size 0 max 16384 2024-03-21T09:11:05.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.477+0000 7efdba5c9740 1 Processor -- start 2024-03-21T09:11:05.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 -- start start 2024-03-21T09:11:05.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 0x55a8ada5f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada6c2a0 con 0x55a8ada5b8e0 2024-03-21T09:11:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada6c900 con 0x55a8ada56770 2024-03-21T09:11:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdba5c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada6cf60 con 0x55a8ada55d80 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdb4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 0x55a8ada5f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdb4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 0x55a8ada5f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46586/0 (socket says 172.21.15.37:46586) 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdb4e00640 1 -- 172.21.15.37:0/2823713955 learned_addr learned my addr 172.21.15.37:0/2823713955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdb5800640 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.478+0000 7efdafe00640 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efdb4e00640 1 -- 172.21.15.37:0/2823713955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 msgr2=0x55a8ada677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efdb4e00640 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efdb4e00640 1 -- 172.21.15.37:0/2823713955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada5b8e0 msgr2=0x55a8ada5dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efdb4e00640 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efdb4e00640 1 -- 172.21.15.37:0/2823713955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ada43170 con 0x55a8ada55d80 2024-03-21T09:11:05.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efdb4e00640 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 0x55a8ada5f8b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efda4014040 tx=0x7efda4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fbb42ffbf6b22ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efda1600640 1 -- 172.21.15.37:0/2823713955 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4005040 con 0x55a8ada55d80 2024-03-21T09:11:05.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efda1600640 1 -- 172.21.15.37:0/2823713955 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efda40035f0 con 0x55a8ada55d80 2024-03-21T09:11:05.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.479+0000 7efda1600640 1 -- 172.21.15.37:0/2823713955 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda402b760 con 0x55a8ada55d80 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 -- 172.21.15.37:0/2823713955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 msgr2=0x55a8ada5f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 0x55a8ada5f8b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efda4014040 tx=0x7efda4011810 comp rx=0 tx=0).stop 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 -- 172.21.15.37:0/2823713955 shutdown_connections 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada5b8e0 0x55a8ada5dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 --2- 172.21.15.37:0/2823713955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada55d80 0x55a8ada5f8b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 -- 172.21.15.37:0/2823713955 >> 172.21.15.37:0/2823713955 conn(0x55a8ada512f0 msgr2=0x55a8ada6bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 -- 172.21.15.37:0/2823713955 shutdown_connections 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 -- 172.21.15.37:0/2823713955 wait complete. 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.480+0000 7efdba5c9740 1 -- start start 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 Processor -- start 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada63240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada6dc20 con 0x55a8ada55d80 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada6d5c0 con 0x55a8ada56770 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ada6e280 con 0x55a8ada5b8e0 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdba5c9740 10 client.0 _ll_register_callbacks cb 0x7efd9c0d3270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdafe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdb4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdafe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada63780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59990/0 (socket says 172.21.15.37:59990) 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdafe00640 1 -- 172.21.15.37:0/425917596 learned_addr learned my addr 172.21.15.37:0/425917596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.481+0000 7efdb5800640 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdafe00640 1 -- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 msgr2=0x55a8ada667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdafe00640 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdafe00640 1 -- 172.21.15.37:0/425917596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 msgr2=0x55a8ada63240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdafe00640 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada63240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdafe00640 1 -- 172.21.15.37:0/425917596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8ada66d90 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdafe00640 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada63780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55a8ada61770 tx=0x7efd9c0a47c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88dd01b3a7c3b051 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efd96a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efd96a00640 1 -- 172.21.15.37:0/425917596 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c09b7e0 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efd96a00640 1 -- 172.21.15.37:0/425917596 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efd9c097bd0 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efda40035f0 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efd96a00640 1 -- 172.21.15.37:0/425917596 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd9c0bc0b0 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.482+0000 7efdba5c9740 10 client.25148 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a8adac7e40 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efd96a00640 1 -- 172.21.15.37:0/425917596 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efda40035f0 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efd96a00640 1 -- 172.21.15.37:0/425917596 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7efd9c097dd0 con 0x55a8ada56770 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 10 client.25148 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 10 client.25148 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 20 client.25148 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8adacc470 0x55a8adac8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 4 client.25148 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:11:05.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8ad9c81d0 con 0x55a8adacc470 2024-03-21T09:11:05.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.483+0000 7efdb4e00640 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8adacc470 0x55a8adac8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdb4e00640 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8adacc470 0x55a8adac8070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd55931e8508f84f server_cookie=1844ed6449676719 in_seq=0 out_seq=0 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efd96a00640 10 client.25148 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efd96a00640 1 -- 172.21.15.37:0/425917596 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a8ad9c81d0 con 0x55a8adacc470 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efd96a00640 10 client.25148 handle_command_reply: tid=0 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 client.25148 shutdown 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 msgr2=0x55a8ada63780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada63780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55a8ada61770 tx=0x7efd9c0a47c0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 shutdown_connections 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8ada5b8e0 0x55a8ada667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8ada56770 0x55a8ada63780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8adacc470 0x55a8adac8070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 --2- 172.21.15.37:0/425917596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8ada55d80 0x55a8ada63240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 >> 172.21.15.37:0/425917596 conn(0x55a8ada512f0 msgr2=0x55a8ada6bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.484+0000 7efd96a00640 0 client.25148 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 shutdown_connections 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdba5c9740 1 -- 172.21.15.37:0/425917596 wait complete. 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdba5c9740 20 client.25148 trim_cache size 0 max 16384 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24553, 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:11:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 105.37872412999999, 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 94, 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 157.27255539399999 2024-03-21T09:11:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd980a2fa0 msgr2=0x7efd980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdb6200640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd980a2fa0 0x7efd980a5460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efda8005e30 tx=0x7efda8032590 comp rx=0 tx=0).stop 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 msgr2=0x7efdb019e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.485+0000 7efdb6200640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb019e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00e9f0 tx=0x7efd9c00edf0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 shutdown_connections 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efd980a2fa0 0x7efd980a5460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb0105080 0x7efdb01a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb01026d0 0x7efdb019e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 --2- 172.21.15.37:0/1038607624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb00ffd60 0x7efdb019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 >> 172.21.15.37:0/1038607624 conn(0x7efdb00fb980 msgr2=0x7efdb0100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 shutdown_connections 2024-03-21T09:11:05.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.486+0000 7efdb6200640 1 -- 172.21.15.37:0/1038607624 wait complete. 2024-03-21T09:11:05.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T09:11:05.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 -- 172.21.15.37:0/1490124104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec105290 msgr2=0x7f6aec10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 --2- 172.21.15.37:0/1490124104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec105290 0x7f6aec10bb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ca0 tx=0x7f6ae402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 -- 172.21.15.37:0/1490124104 shutdown_connections 2024-03-21T09:11:05.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 --2- 172.21.15.37:0/1490124104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec105290 0x7f6aec10bb20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 --2- 172.21.15.37:0/1490124104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 0x7f6aec104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 --2- 172.21.15.37:0/1490124104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec1036e0 0x7f6aec103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 -- 172.21.15.37:0/1490124104 >> 172.21.15.37:0/1490124104 conn(0x7f6aec0fec70 msgr2=0x7f6aec101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 -- 172.21.15.37:0/1490124104 shutdown_connections 2024-03-21T09:11:05.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.707+0000 7f6af2400640 1 -- 172.21.15.37:0/1490124104 wait complete. 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.708+0000 7f6af2400640 1 Processor -- start 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 -- start start 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 0x7f6aec199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 0x7f6aec199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec105290 0x7f6aec1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec10f340 con 0x7f6aec105290 2024-03-21T09:11:05.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec10f1c0 con 0x7f6aec1048d0 2024-03-21T09:11:05.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af2400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec10f4c0 con 0x7f6aec1036e0 2024-03-21T09:11:05.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 0x7f6aec199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 0x7f6aec199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 0x7f6aec199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46612/0 (socket says 172.21.15.37:46612) 2024-03-21T09:11:05.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.709+0000 7f6af1000640 1 -- 172.21.15.37:0/3209325849 learned_addr learned my addr 172.21.15.37:0/3209325849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1a00640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec105290 0x7f6aec1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1000640 1 -- 172.21.15.37:0/3209325849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 msgr2=0x7f6aec199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1000640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 0x7f6aec199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1000640 1 -- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec105290 msgr2=0x7f6aec1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1000640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec105290 0x7f6aec1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1000640 1 -- 172.21.15.37:0/3209325849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec1a1570 con 0x7f6aec1036e0 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1a00640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec105290 0x7f6aec1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6aebe00640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 0x7f6aec199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af1000640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 0x7f6aec199950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000e9c0 tx=0x7f6ae000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6ae9600640 1 -- 172.21.15.37:0/3209325849 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000cd80 con 0x7f6aec1036e0 2024-03-21T09:11:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6ae9600640 1 -- 172.21.15.37:0/3209325849 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae00035f0 con 0x7f6aec1036e0 2024-03-21T09:11:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6ae9600640 1 -- 172.21.15.37:0/3209325849 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0010690 con 0x7f6aec1036e0 2024-03-21T09:11:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.710+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec1a1860 con 0x7f6aec1036e0 2024-03-21T09:11:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.711+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec1a1d60 con 0x7f6aec1036e0 2024-03-21T09:11:05.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 Processor -- start 2024-03-21T09:11:05.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 -- start start 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b89d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b89fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b7fcc90 con 0x55fa9b89d8e0 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b7a5980 con 0x55fa9b897d80 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6af67cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b7f8d30 con 0x55fa9b898770 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.715+0000 7f6ae9600640 1 -- 172.21.15.37:0/3209325849 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ae00037a0 con 0x7f6aec1036e0 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:60018/0 (socket says 172.21.15.37:60018) 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 -- 172.21.15.37:0/1974516847 learned_addr learned my addr 172.21.15.37:0/1974516847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1000640 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b89d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6aebe00640 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b89fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 -- 172.21.15.37:0/1974516847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 msgr2=0x55fa9b89d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b89d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 -- 172.21.15.37:0/1974516847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 msgr2=0x55fa9b89fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b89fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 -- 172.21.15.37:0/1974516847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa9b7bd510 con 0x55fa9b897d80 2024-03-21T09:11:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.716+0000 7f6af1a00640 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8981a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ca0 tx=0x7f6ae4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48c187301054457 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6ade000640 1 -- 172.21.15.37:0/1974516847 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4043020 con 0x55fa9b897d80 2024-03-21T09:11:05.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6ade000640 1 -- 172.21.15.37:0/1974516847 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4032c90 con 0x55fa9b897d80 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6ade000640 1 -- 172.21.15.37:0/1974516847 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4039040 con 0x55fa9b897d80 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6ae9600640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ad40a2fa0 0x7f6ad40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6ae9600640 1 -- 172.21.15.37:0/3209325849 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ae00cfbf0 con 0x7f6aec1036e0 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 -- 172.21.15.37:0/1974516847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 msgr2=0x55fa9b8981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8981a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ca0 tx=0x7f6ae4009800 comp rx=0 tx=0).stop 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af1000640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ad40a2fa0 0x7f6ad40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 -- 172.21.15.37:0/1974516847 shutdown_connections 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b89fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b89d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1974516847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8981a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.717+0000 7f6af67cb740 1 -- 172.21.15.37:0/1974516847 >> 172.21.15.37:0/1974516847 conn(0x55fa9b8932f0 msgr2=0x55fa9b8936f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.718+0000 7f6af67cb740 1 -- 172.21.15.37:0/1974516847 shutdown_connections 2024-03-21T09:11:05.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.718+0000 7f6af67cb740 1 -- 172.21.15.37:0/1974516847 wait complete. 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.718+0000 7f6af1000640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ad40a2fa0 0x7f6ad40a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ae009bec0 tx=0x7f6ae0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.718+0000 7f6af67cb740 1 -- start start 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.718+0000 7f6af67cb740 1 Processor -- start 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8a0830 con 0x55fa9b89d8e0 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b7b8dd0 con 0x55fa9b897d80 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8a09b0 con 0x55fa9b898770 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af67cb740 10 client.0 _ll_register_callbacks cb 0x55fa9b510ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46632/0 (socket says 172.21.15.37:46632) 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6aebe00640 1 -- 172.21.15.37:0/49126472 learned_addr learned my addr 172.21.15.37:0/49126472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.719+0000 7f6af1a00640 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6aebe00640 1 -- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 msgr2=0x55fa9b8a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6aebe00640 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6aebe00640 1 -- 172.21.15.37:0/49126472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 msgr2=0x55fa9b8a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6aebe00640 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6aebe00640 1 -- 172.21.15.37:0/49126472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa9b5112d0 con 0x55fa9b898770 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6aebe00640 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a2770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8012960 tx=0x7f6ad8012e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b2e1f0a757cf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6ad2a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6ad2a00640 1 -- 172.21.15.37:0/49126472 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800f980 con 0x55fa9b898770 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6ad2a00640 1 -- 172.21.15.37:0/49126472 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ad800fb20 con 0x55fa9b898770 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.720+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ae4032c90 con 0x55fa9b898770 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6ad2a00640 1 -- 172.21.15.37:0/49126472 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad800fd20 con 0x55fa9b898770 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 10 client.25180 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fa9b5db6b0 con 0x55fa9b898770 2024-03-21T09:11:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6ad2a00640 1 -- 172.21.15.37:0/49126472 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4032c90 con 0x55fa9b898770 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6ad2a00640 1 -- 172.21.15.37:0/49126472 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6ad801c4b0 con 0x55fa9b898770 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 10 client.25180 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 10 client.25180 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 20 client.25180 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b8adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 4 client.25180 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa9b79e520 con 0x55fa9b8aea70 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.721+0000 7f6af1a00640 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b8adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.722+0000 7f6af1a00640 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b8adb50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d7c79c5184c5806 server_cookie=66c3efb2f84d9466 in_seq=0 out_seq=0 2024-03-21T09:11:05.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.722+0000 7f6ad2a00640 10 client.25180 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6ad2a00640 1 -- 172.21.15.37:0/49126472 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fa9b79e520 con 0x55fa9b8aea70 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6ad2a00640 10 client.25180 handle_command_reply: tid=0 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 client.25180 shutdown 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 msgr2=0x55fa9b8a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a2770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8012960 tx=0x7f6ad8012e20 comp rx=0 tx=0).stop 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 shutdown_connections 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b8adb50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6ad2a00640 0 client.25180 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a2770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 --2- 172.21.15.37:0/49126472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 >> 172.21.15.37:0/49126472 conn(0x55fa9b8932f0 msgr2=0x55fa9b8924f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 shutdown_connections 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 1 -- 172.21.15.37:0/49126472 wait complete. 2024-03-21T09:11:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.723+0000 7f6af67cb740 20 client.25180 trim_cache size 0 max 16384 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.741+0000 7f6af67cb740 1 Processor -- start 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 -- start start 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b897d80 0x55fa9b896c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8ae290 con 0x55fa9b897d80 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8ae8f0 con 0x55fa9b89d8e0 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af67cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8aef50 con 0x55fa9b898770 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46644/0 (socket says 172.21.15.37:46644) 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 -- 172.21.15.37:0/1988223401 learned_addr learned my addr 172.21.15.37:0/1988223401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6af1a00640 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b897d80 0x55fa9b896c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 -- 172.21.15.37:0/1988223401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b89d8e0 msgr2=0x55fa9b8b6f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8b6f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 -- 172.21.15.37:0/1988223401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b897d80 msgr2=0x55fa9b896c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b897d80 0x55fa9b896c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.742+0000 7f6aebe00640 1 -- 172.21.15.37:0/1988223401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa9b894060 con 0x55fa9b898770 2024-03-21T09:11:05.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.743+0000 7f6aebe00640 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a9770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005b60 tx=0x7f6ad800ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e4a892025f3c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.743+0000 7f6add600640 1 -- 172.21.15.37:0/1988223401 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800fa80 con 0x55fa9b898770 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.743+0000 7f6add600640 1 -- 172.21.15.37:0/1988223401 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad801c420 con 0x55fa9b898770 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.743+0000 7f6add600640 1 -- 172.21.15.37:0/1988223401 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8003850 con 0x55fa9b898770 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 -- 172.21.15.37:0/1988223401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 msgr2=0x55fa9b8a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a9770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005b60 tx=0x7f6ad800ff90 comp rx=0 tx=0).stop 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 -- 172.21.15.37:0/1988223401 shutdown_connections 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b898770 0x55fa9b8a9770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 --2- 172.21.15.37:0/1988223401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b897d80 0x55fa9b896c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 -- 172.21.15.37:0/1988223401 >> 172.21.15.37:0/1988223401 conn(0x55fa9b8932f0 msgr2=0x55fa9b896550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 -- 172.21.15.37:0/1988223401 shutdown_connections 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 -- 172.21.15.37:0/1988223401 wait complete. 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 -- start start 2024-03-21T09:11:05.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.744+0000 7f6af67cb740 1 Processor -- start 2024-03-21T09:11:05.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b898770 0x55fa9b8a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8afc10 con 0x55fa9b898770 2024-03-21T09:11:05.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8af5b0 con 0x55fa9b897d80 2024-03-21T09:11:05.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa9b8b0270 con 0x55fa9b89d8e0 2024-03-21T09:11:05.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af67cb740 10 client.0 _ll_register_callbacks cb 0x7f6ad8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:05.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b898770 0x55fa9b8a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:60044/0 (socket says 172.21.15.37:60044) 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af1a00640 1 -- 172.21.15.37:0/2385201101 learned_addr learned my addr 172.21.15.37:0/2385201101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af1a00640 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.745+0000 7f6af1a00640 1 -- 172.21.15.37:0/2385201101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b89d8e0 msgr2=0x55fa9b8a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af1a00640 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af1a00640 1 -- 172.21.15.37:0/2385201101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b898770 msgr2=0x55fa9b8a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af1a00640 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b898770 0x55fa9b8a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af1a00640 1 -- 172.21.15.37:0/2385201101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa9b8a69c0 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af1a00640 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a93c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4002230 tx=0x7f6ae40093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5607ca885ae5c3cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6ad2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6ad2a00640 1 -- 172.21.15.37:0/2385201101 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4039070 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ad801c420 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6ad2a00640 1 -- 172.21.15.37:0/2385201101 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ae4032690 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6ad2a00640 1 -- 172.21.15.37:0/2385201101 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae4052040 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af67cb740 10 client.25163 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.746+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fa9b90aaa0 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6ad2a00640 1 -- 172.21.15.37:0/2385201101 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad801c420 con 0x55fa9b897d80 2024-03-21T09:11:05.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6ad2a00640 1 -- 172.21.15.37:0/2385201101 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6ae4004030 con 0x55fa9b897d80 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6af67cb740 10 client.25163 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6af67cb740 10 client.25163 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6af67cb740 20 client.25163 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6af67cb740 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b90ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6af67cb740 4 client.25163 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fa9b80a1d0 con 0x55fa9b8aea70 2024-03-21T09:11:05.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.747+0000 7f6aebe00640 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b90ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6aebe00640 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b90ac70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fd28cda70f1ef8c server_cookie=f9890d80c38abc63 in_seq=0 out_seq=0 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6ad2a00640 10 client.25163 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6ad2a00640 1 -- 172.21.15.37:0/2385201101 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fa9b80a1d0 con 0x55fa9b8aea70 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6ad2a00640 10 client.25163 handle_command_reply: tid=0 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 client.25163 shutdown 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 msgr2=0x55fa9b8a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a93c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4002230 tx=0x7f6ae40093f0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 shutdown_connections 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55fa9b8aea70 0x55fa9b90ac70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa9b89d8e0 0x55fa9b8a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fa9b898770 0x55fa9b8a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6ad2a00640 0 client.25163 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:05.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 --2- 172.21.15.37:0/2385201101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa9b897d80 0x55fa9b8a93c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 >> 172.21.15.37:0/2385201101 conn(0x55fa9b8932f0 msgr2=0x55fa9b896de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 shutdown_connections 2024-03-21T09:11:05.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.748+0000 7f6af67cb740 1 -- 172.21.15.37:0/2385201101 wait complete. 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af67cb740 20 client.25163 trim_cache size 0 max 16384 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24575, 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:11:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:11:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 106.655865797, 2024-03-21T09:11:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-03-21T09:11:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 94, 2024-03-21T09:11:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-03-21T09:11:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 146.37054321900001 2024-03-21T09:11:05.752 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:11:05.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ad40a2fa0 msgr2=0x7f6ad40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ad40a2fa0 0x7f6ad40a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ae009bec0 tx=0x7f6ae0000bc0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 msgr2=0x7f6aec199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 0x7f6aec199950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000e9c0 tx=0x7f6ae000ee80 comp rx=0 tx=0).stop 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 shutdown_connections 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ad40a2fa0 0x7f6ad40a5460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aec105290 0x7f6aec1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec1048d0 0x7f6aec199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 --2- 172.21.15.37:0/3209325849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aec1036e0 0x7f6aec199950 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.749+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 >> 172.21.15.37:0/3209325849 conn(0x7f6aec0fec70 msgr2=0x7f6aec107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.750+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 shutdown_connections 2024-03-21T09:11:05.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.750+0000 7f6af2400640 1 -- 172.21.15.37:0/3209325849 wait complete. 2024-03-21T09:11:05.759 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:05 smithi037 ceph-mon[31830]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:05.759 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:05 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3620354875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:05.760 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:05 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3211213477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:05.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-21T09:11:05.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:05 smithi071 ceph-mon[36597]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:05.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:05 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3620354875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:05.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:05 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3211213477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:05.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:05 smithi053 ceph-mon[36169]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:05.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:05 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3620354875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:05.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:05 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3211213477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 -- 172.21.15.37:0/1559751955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8101510 msgr2=0x7fbcb8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/1559751955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8101510 0x7fbcb8101930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009900 tx=0x7fbca802bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 -- 172.21.15.37:0/1559751955 shutdown_connections 2024-03-21T09:11:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/1559751955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcb81030c0 0x7fbcb8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/1559751955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcb8102700 0x7fbcb8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/1559751955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8101510 0x7fbcb8101930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 -- 172.21.15.37:0/1559751955 >> 172.21.15.37:0/1559751955 conn(0x7fbcb80fca80 msgr2=0x7fbcb80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 -- 172.21.15.37:0/1559751955 shutdown_connections 2024-03-21T09:11:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.982+0000 7fbcb7e00640 1 -- 172.21.15.37:0/1559751955 wait complete. 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.983+0000 7fbcb7e00640 1 Processor -- start 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.983+0000 7fbcb7e00640 1 -- start start 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcb8101510 0x7fbcb8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 0x7fbcb8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcb81030c0 0x7fbcb81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb810d1f0 con 0x7fbcb8101510 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb810d070 con 0x7fbcb81030c0 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb810d370 con 0x7fbcb8102700 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 0x7fbcb8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcb8101510 0x7fbcb8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcb81030c0 0x7fbcb81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 0x7fbcb8199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46682/0 (socket says 172.21.15.37:46682) 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 -- 172.21.15.37:0/4179120754 learned_addr learned my addr 172.21.15.37:0/4179120754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 -- 172.21.15.37:0/4179120754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcb81030c0 msgr2=0x7fbcb81a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcb81030c0 0x7fbcb81a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 -- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcb8101510 msgr2=0x7fbcb8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcb8101510 0x7fbcb8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.984+0000 7fbcae000640 1 -- 172.21.15.37:0/4179120754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb81a1650 con 0x7fbcb8102700 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.985+0000 7fbcae000640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 0x7fbcb8199f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbca000e9f0 tx=0x7fbca000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.985+0000 7fbcbca00640 1 -- 172.21.15.37:0/4179120754 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000cd80 con 0x7fbcb8102700 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.985+0000 7fbcbca00640 1 -- 172.21.15.37:0/4179120754 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbca0010910 con 0x7fbcb8102700 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.985+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb81a1940 con 0x7fbcb8102700 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.985+0000 7fbcbca00640 1 -- 172.21.15.37:0/4179120754 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0018990 con 0x7fbcb8102700 2024-03-21T09:11:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.985+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb81a1ee0 con 0x7fbcb8102700 2024-03-21T09:11:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.988+0000 7fbcc0bc0740 1 Processor -- start 2024-03-21T09:11:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.988+0000 7fbcbca00640 1 -- 172.21.15.37:0/4179120754 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbca0010ac0 con 0x7fbcb8102700 2024-03-21T09:11:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.988+0000 7fbcc0bc0740 1 -- start start 2024-03-21T09:11:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d251d80 0x558e6d2521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 0x558e6d257270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d259d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d1b6c90 con 0x558e6d251d80 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d15f980 con 0x558e6d2578e0 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d1b2d30 con 0x558e6d252770 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d251d80 0x558e6d2521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 0x558e6d257270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcae000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d259d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 0x558e6d257270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46696/0 (socket says 172.21.15.37:46696) 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1026871163 learned_addr learned my addr 172.21.15.37:0/1026871163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1026871163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 msgr2=0x558e6d259d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.989+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d259d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1026871163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d251d80 msgr2=0x558e6d2521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d251d80 0x558e6d2521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1026871163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e6d177510 con 0x558e6d252770 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcae000640 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d259d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcb7400640 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d251d80 0x558e6d2521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 0x558e6d257270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbca8000c00 tx=0x7fbca8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c1d2a862d48e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcbca00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbc980a2fa0 0x7fbc980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbca7e00640 1 -- 172.21.15.37:0/1026871163 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca802c980 con 0x558e6d252770 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbca7e00640 1 -- 172.21.15.37:0/1026871163 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbca802cb20 con 0x558e6d252770 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbcbca00640 1 -- 172.21.15.37:0/4179120754 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbca00cf9e0 con 0x7fbcb8102700 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.990+0000 7fbca7e00640 1 -- 172.21.15.37:0/1026871163 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80046c0 con 0x558e6d252770 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcb7400640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbc980a2fa0 0x7fbc980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1026871163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 msgr2=0x558e6d257270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcb7400640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbc980a2fa0 0x7fbc980a5460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x558e6d252fd0 tx=0x7fbcb0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 0x558e6d257270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbca8000c00 tx=0x7fbca8009560 comp rx=0 tx=0).stop 2024-03-21T09:11:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1026871163 shutdown_connections 2024-03-21T09:11:05.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d259d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d252770 0x558e6d257270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1026871163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d251d80 0x558e6d2521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.991+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1026871163 >> 172.21.15.37:0/1026871163 conn(0x558e6d24d2f0 msgr2=0x558e6d24d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.992+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1026871163 shutdown_connections 2024-03-21T09:11:05.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.992+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1026871163 wait complete. 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.992+0000 7fbcc0bc0740 1 -- start start 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 Processor -- start 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d25a830 con 0x558e6d252770 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d172dd0 con 0x558e6d251d80 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d25a9b0 con 0x558e6d2578e0 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcc0bc0740 10 client.0 _ll_register_callbacks cb 0x558e6cecaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:60068/0 (socket says 172.21.15.37:60068) 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.993+0000 7fbcb6a00640 1 -- 172.21.15.37:0/3612424472 learned_addr learned my addr 172.21.15.37:0/3612424472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb7400640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcae000640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb6a00640 1 -- 172.21.15.37:0/3612424472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 msgr2=0x558e6d2637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb6a00640 1 -- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 msgr2=0x558e6d25c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb6a00640 1 -- 172.21.15.37:0/3612424472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e6cecb2d0 con 0x558e6d251d80 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcae000640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25c270 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbca802c240 tx=0x7fbca803f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cf48a84c19902e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbca4c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.994+0000 7fbca4c00640 1 -- 172.21.15.37:0/3612424472 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca804f070 con 0x558e6d251d80 2024-03-21T09:11:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbca802cb20 con 0x558e6d251d80 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbca4c00640 1 -- 172.21.15.37:0/3612424472 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbca803fca0 con 0x558e6d251d80 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbca4c00640 1 -- 172.21.15.37:0/3612424472 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca8058050 con 0x558e6d251d80 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbcc0bc0740 10 client.25175 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558e6cf956b0 con 0x558e6d251d80 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbca4c00640 1 -- 172.21.15.37:0/3612424472 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbca802cb20 con 0x558e6d251d80 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbca4c00640 1 -- 172.21.15.37:0/3612424472 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbca8052020 con 0x558e6d251d80 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbcc0bc0740 10 client.25175 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.995+0000 7fbcc0bc0740 10 client.25175 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbcc0bc0740 20 client.25175 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d267950 0x558e6d26ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbcc0bc0740 4 client.25175 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e6d158520 con 0x558e6d267950 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbcae000640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d267950 0x558e6d26ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbcae000640 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d267950 0x558e6d26ca30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a0bcefdf25c84a server_cookie=6aaf6c891b413047 in_seq=0 out_seq=0 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.996+0000 7fbca4c00640 10 client.25175 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.997+0000 7fbca4c00640 1 -- 172.21.15.37:0/3612424472 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558e6d158520 con 0x558e6d267950 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.997+0000 7fbca4c00640 10 client.25175 handle_command_reply: tid=0 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.997+0000 7fbcc0bc0740 1 client.25175 shutdown 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.997+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 msgr2=0x558e6d25c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.997+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25c270 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbca802c240 tx=0x7fbca803f8b0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 shutdown_connections 2024-03-21T09:11:05.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d267950 0x558e6d26ca30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/3612424472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25c270 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 >> 172.21.15.37:0/3612424472 conn(0x558e6d24d2f0 msgr2=0x558e6d24c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbca4c00640 0 client.25175 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 shutdown_connections 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/3612424472 wait complete. 2024-03-21T09:11:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:05.998+0000 7fbcc0bc0740 20 client.25175 trim_cache size 0 max 16384 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.021+0000 7fbcc0bc0740 1 Processor -- start 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 -- start start 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 0x558e6d24d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d2637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d24bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d267dd0 con 0x558e6d252770 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d268430 con 0x558e6d2578e0 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d268a90 con 0x558e6d251d80 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 0x558e6d24d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d24bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 0x558e6d24d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46724/0 (socket says 172.21.15.37:46724) 2024-03-21T09:11:06.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1678356262 learned_addr learned my addr 172.21.15.37:0/1678356262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:06.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.022+0000 7fbcae000640 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d2637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1678356262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 msgr2=0x558e6d24bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d24bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1678356262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 msgr2=0x558e6d2637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d2637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbcb6a00640 1 -- 172.21.15.37:0/1678356262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e6d252b50 con 0x558e6d251d80 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 0x558e6d24d6d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbca802c170 tx=0x7fbca8005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721d157cde68ea6e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbca7400640 1 -- 172.21.15.37:0/1678356262 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca804f040 con 0x558e6d251d80 2024-03-21T09:11:06.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbca7400640 1 -- 172.21.15.37:0/1678356262 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbca80496d0 con 0x558e6d251d80 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.023+0000 7fbca7400640 1 -- 172.21.15.37:0/1678356262 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8034650 con 0x558e6d251d80 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1678356262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 msgr2=0x558e6d24d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 0x558e6d24d6d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbca802c170 tx=0x7fbca8005560 comp rx=0 tx=0).stop 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1678356262 shutdown_connections 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d2578e0 0x558e6d24bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d2637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1678356262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d251d80 0x558e6d24d6d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1678356262 >> 172.21.15.37:0/1678356262 conn(0x558e6d24d2f0 msgr2=0x558e6d2531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1678356262 shutdown_connections 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.024+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1678356262 wait complete. 2024-03-21T09:11:06.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.025+0000 7fbcc0bc0740 1 -- start start 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.025+0000 7fbcc0bc0740 1 Processor -- start 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d269750 con 0x558e6d252770 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d2690f0 con 0x558e6d251d80 2024-03-21T09:11:06.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6d269db0 con 0x558e6d2578e0 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcc0bc0740 10 client.0 _ll_register_callbacks cb 0x7fbcb0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2627f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46732/0 (socket says 172.21.15.37:46732) 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:60092/0 (socket says 172.21.15.37:60092) 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 -- 172.21.15.37:0/1134058398 learned_addr learned my addr 172.21.15.37:0/1134058398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcae000640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 -- 172.21.15.37:0/1134058398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 msgr2=0x558e6d25f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 -- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 msgr2=0x558e6d25f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb7400640 1 -- 172.21.15.37:0/1134058398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e6d262de0 con 0x558e6d2578e0 2024-03-21T09:11:06.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.026+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbcb7400640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2627f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0003f20 tx=0x7fbcb0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39abcc659541c44b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbca4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbca4c00640 1 -- 172.21.15.37:0/1134058398 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb002e060 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbca4c00640 1 -- 172.21.15.37:0/1134058398 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbcb00045c0 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbca80496d0 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbca4c00640 1 -- 172.21.15.37:0/1134058398 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb0035050 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbcc0bc0740 10 client.25204 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.027+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558e6d2c3f70 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbca4c00640 1 -- 172.21.15.37:0/1134058398 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbca80496d0 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbca4c00640 1 -- 172.21.15.37:0/1134058398 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbcb001d420 con 0x558e6d2578e0 2024-03-21T09:11:06.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcc0bc0740 10 client.25204 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:06.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcc0bc0740 10 client.25204 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T09:11:06.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcc0bc0740 20 client.25204 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:06.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d2c86d0 0x558e6d2c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:06.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcc0bc0740 4 client.25204 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:11:06.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e6d1c41d0 con 0x558e6d2c86d0 2024-03-21T09:11:06.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.028+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d2c86d0 0x558e6d2c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbcb6a00640 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d2c86d0 0x558e6d2c42b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c220b618dab93c9 server_cookie=c3f5e47ff904e67f in_seq=0 out_seq=0 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbca4c00640 10 client.25204 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbca4c00640 1 -- 172.21.15.37:0/1134058398 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x558e6d1c41d0 con 0x558e6d2c86d0 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbca4c00640 10 client.25204 handle_command_reply: tid=0 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbcc0bc0740 1 client.25204 shutdown 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 msgr2=0x558e6d2627f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:06.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.029+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2627f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0003f20 tx=0x7fbcb0004a40 comp rx=0 tx=0).stop 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 shutdown_connections 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558e6d2578e0 0x558e6d2627f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e6d252770 0x558e6d25f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558e6d2c86d0 0x558e6d2c42b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 --2- 172.21.15.37:0/1134058398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e6d251d80 0x558e6d25f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbca4c00640 0 client.25204 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 >> 172.21.15.37:0/1134058398 conn(0x558e6d24d2f0 msgr2=0x558e6d24d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 shutdown_connections 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 1 -- 172.21.15.37:0/1134058398 wait complete. 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.030+0000 7fbcc0bc0740 20 client.25204 trim_cache size 0 max 16384 2024-03-21T09:11:06.027 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24589, 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 139.139560434, 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 94, 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 149.5522522 2024-03-21T09:11:06.028 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbc980a2fa0 msgr2=0x7fbc980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbc980a2fa0 0x7fbc980a5460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x558e6d252fd0 tx=0x7fbcb0009500 comp rx=0 tx=0).stop 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 msgr2=0x7fbcb8199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 0x7fbcb8199f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbca000e9f0 tx=0x7fbca000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 shutdown_connections 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fbc980a2fa0 0x7fbc980a5460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcb81030c0 0x7fbcb81a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcb8102700 0x7fbcb8199f70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 --2- 172.21.15.37:0/4179120754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcb8101510 0x7fbcb8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:06.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 >> 172.21.15.37:0/4179120754 conn(0x7fbcb80fca80 msgr2=0x7fbcb80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:06.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 shutdown_connections 2024-03-21T09:11:06.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:06.031+0000 7fbcb7e00640 1 -- 172.21.15.37:0/4179120754 wait complete. 2024-03-21T09:11:06.044 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T09:11:06.045 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T09:11:06.055 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:11:06.056 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-21T09:11:06.507 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:07.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:07 smithi053 ceph-mon[36169]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:07.768 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:07 smithi037 ceph-mon[31830]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:07.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:07 smithi071 ceph-mon[36597]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:08.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 -- 172.21.15.37:0/2915193448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01021e0 msgr2=0x7fcac0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2915193448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01021e0 0x7fcac0102600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcab40099c0 tx=0x7fcab4009cd0 comp rx=0 tx=0).stop 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 -- 172.21.15.37:0/2915193448 shutdown_connections 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2915193448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac0103d80 0x7fcac010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2915193448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01021e0 0x7fcac0102600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2915193448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac01033e0 0x7fcac0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 -- 172.21.15.37:0/2915193448 >> 172.21.15.37:0/2915193448 conn(0x7fcac00fd740 msgr2=0x7fcac00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 -- 172.21.15.37:0/2915193448 shutdown_connections 2024-03-21T09:11:08.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.915+0000 7fcac7e00700 1 -- 172.21.15.37:0/2915193448 wait complete. 2024-03-21T09:11:08.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.916+0000 7fcac7e00700 1 Processor -- start 2024-03-21T09:11:08.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 -- start start 2024-03-21T09:11:08.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 0x7fcac01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01033e0 0x7fcac01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac0103d80 0x7fcac019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac010ddf0 con 0x7fcac0103d80 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac010dc70 con 0x7fcac01033e0 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac7e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac010df70 con 0x7fcac01021e0 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 0x7fcac01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 0x7fcac01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46744/0 (socket says 172.21.15.37:46744) 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.917+0000 7fcac5400700 1 -- 172.21.15.37:0/2045642634 learned_addr learned my addr 172.21.15.37:0/2045642634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac4a00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01033e0 0x7fcac01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac0103d80 0x7fcac019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5400700 1 -- 172.21.15.37:0/2045642634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01033e0 msgr2=0x7fcac01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5400700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01033e0 0x7fcac01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:08.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5400700 1 -- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac0103d80 msgr2=0x7fcac019fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:08.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5400700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac0103d80 0x7fcac019fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:08.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5400700 1 -- 172.21.15.37:0/2045642634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac01a00c0 con 0x7fcac01021e0 2024-03-21T09:11:08.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac4a00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01033e0 0x7fcac01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:08.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac0103d80 0x7fcac019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:08.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcac5400700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 0x7fcac01984b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcab400bee0 tx=0x7fcab400bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:08.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.918+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4003e30 con 0x7fcac01021e0 2024-03-21T09:11:08.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.919+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac01a0350 con 0x7fcac01021e0 2024-03-21T09:11:08.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.919+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcab400f460 con 0x7fcac01021e0 2024-03-21T09:11:08.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.919+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4017630 con 0x7fcac01021e0 2024-03-21T09:11:08.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.919+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a07a0 con 0x7fcac01021e0 2024-03-21T09:11:08.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.920+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0068e50 con 0x7fcac01021e0 2024-03-21T09:11:08.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.922+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcab400f980 con 0x7fcac01021e0 2024-03-21T09:11:08.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.923+0000 7fcabe000700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcaa80a2fb0 0x7fcaa80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:08.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.923+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcab40d0670 con 0x7fcac01021e0 2024-03-21T09:11:08.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.924+0000 7fcac4a00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcaa80a2fb0 0x7fcaa80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:08.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.924+0000 7fcac4a00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcaa80a2fb0 0x7fcaa80a5470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009cb0 tx=0x7fcaac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:08.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:08.927+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcab4013070 con 0x7fcac01021e0 2024-03-21T09:11:09.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.174+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcac00625d0 con 0x7fcaa80a2fb0 2024-03-21T09:11:09.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.176+0000 7fcabe000700 1 -- 172.21.15.37:0/2045642634 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fcac00625d0 con 0x7fcaa80a2fb0 2024-03-21T09:11:09.170 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-03-21T09:11:09.170 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-03-21T09:11:09.170 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcaa80a2fb0 msgr2=0x7fcaa80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcaa80a2fb0 0x7fcaa80a5470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009cb0 tx=0x7fcaac009260 comp rx=0 tx=0).stop 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 msgr2=0x7fcac01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 0x7fcac01984b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcab400bee0 tx=0x7fcab400bfc0 comp rx=0 tx=0).stop 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 shutdown_connections 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac0103d80 0x7fcac019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac01021e0 0x7fcac01984b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcaa80a2fb0 0x7fcaa80a5470 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 --2- 172.21.15.37:0/2045642634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac01033e0 0x7fcac01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 >> 172.21.15.37:0/2045642634 conn(0x7fcac00fd740 msgr2=0x7fcac01066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:09.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.179+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 shutdown_connections 2024-03-21T09:11:09.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:09.180+0000 7fcac7e00700 1 -- 172.21.15.37:0/2045642634 wait complete. 2024-03-21T09:11:09.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:09 smithi071 ceph-mon[36597]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:09.888 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:09 smithi037 ceph-mon[31830]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:09.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:09 smithi053 ceph-mon[36169]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:09.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-21T09:11:10.390 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:10.771 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:10 smithi037 ceph-mon[31830]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:10.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:10 smithi071 ceph-mon[36597]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:10.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:10 smithi053 ceph-mon[36169]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:11.772 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:11 smithi037 ceph-mon[31830]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:11.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:11 smithi071 ceph-mon[36597]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:11.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:11 smithi053 ceph-mon[36169]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.632+0000 7fd588e00700 1 -- 172.21.15.37:0/3396589680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841021d0 msgr2=0x7fd5841025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.632+0000 7fd588e00700 1 --2- 172.21.15.37:0/3396589680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841021d0 0x7fd5841025f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd574009920 tx=0x7fd574009c30 comp rx=0 tx=0).stop 2024-03-21T09:11:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 -- 172.21.15.37:0/3396589680 shutdown_connections 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 --2- 172.21.15.37:0/3396589680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd584103d70 0x7fd58410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 --2- 172.21.15.37:0/3396589680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841021d0 0x7fd5841025f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 --2- 172.21.15.37:0/3396589680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5841033d0 0x7fd584103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 -- 172.21.15.37:0/3396589680 >> 172.21.15.37:0/3396589680 conn(0x7fd5840fd730 msgr2=0x7fd5840ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 -- 172.21.15.37:0/3396589680 shutdown_connections 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.633+0000 7fd588e00700 1 -- 172.21.15.37:0/3396589680 wait complete. 2024-03-21T09:11:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 Processor -- start 2024-03-21T09:11:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 -- start start 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5841021d0 0x7fd58419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 0x7fd58419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd584103d70 0x7fd5841a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58410de00 con 0x7fd5841021d0 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58410dc80 con 0x7fd5841033d0 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.634+0000 7fd588e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58410df80 con 0x7fd584103d70 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 0x7fd58419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 0x7fd58419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:60136/0 (socket says 172.21.15.37:60136) 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 -- 172.21.15.37:0/1021949307 learned_addr learned my addr 172.21.15.37:0/1021949307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd582800700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd584103d70 0x7fd5841a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:12.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5841021d0 0x7fd58419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 -- 172.21.15.37:0/1021949307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd584103d70 msgr2=0x7fd5841a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd584103d70 0x7fd5841a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 -- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5841021d0 msgr2=0x7fd58419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5841021d0 0x7fd58419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.635+0000 7fd581400700 1 -- 172.21.15.37:0/1021949307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5841a2370 con 0x7fd5841033d0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.636+0000 7fd581400700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 0x7fd58419ac50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9c0 tx=0x7fd57000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.636+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570004a30 con 0x7fd5841033d0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.636+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57000cca0 con 0x7fd5841033d0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.636+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5841a2660 con 0x7fd5841033d0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.636+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570005680 con 0x7fd5841033d0 2024-03-21T09:11:12.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.636+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841a2bf0 con 0x7fd5841033d0 2024-03-21T09:11:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.637+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd584068e50 con 0x7fd5841033d0 2024-03-21T09:11:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.639+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd57000c7c0 con 0x7fd5841033d0 2024-03-21T09:11:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.640+0000 7fd57aa00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5680a3030 0x7fd5680a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.640+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5700cf7f0 con 0x7fd5841033d0 2024-03-21T09:11:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.640+0000 7fd581e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5680a3030 0x7fd5680a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:12.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.641+0000 7fd581e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5680a3030 0x7fd5680a54f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd57400fb70 tx=0x7fd574019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.644+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd57009c230 con 0x7fd5841033d0 2024-03-21T09:11:12.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.882+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5841a2eb0 con 0x7fd5680a3030 2024-03-21T09:11:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.900+0000 7fd57aa00700 1 -- 172.21.15.37:0/1021949307 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7fd5841a2eb0 con 0x7fd5680a3030 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (2m) 43s ago 2m 24.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 69efc5c45f13 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi053 running (2m) 84s ago 2m 18.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 1aa4f9723303 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi071 running (2m) 84s ago 2m 18.3M - 19.0.0-2231-g2edfee68 237e43c7a65f 7250c6777bcb 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (2m) 43s ago 2m 12.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 6c7cf65ee58e 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi053 running (2m) 84s ago 2m 13.8M - 19.0.0-2231-g2edfee68 237e43c7a65f 3408ef0175f3 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi071 running (2m) 84s ago 2m 15.5M - 19.0.0-2231-g2edfee68 237e43c7a65f b4766b14ca93 2024-03-21T09:11:12.895 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (2m) 43s ago 2m 12.2M - 19.0.0-2231-g2edfee68 237e43c7a65f 842b11d99e53 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi053 running (2m) 84s ago 2m 14.7M - 19.0.0-2231-g2edfee68 237e43c7a65f ca54b8f9e05a 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi071 running (2m) 84s ago 2m 16.6M - 19.0.0-2231-g2edfee68 237e43c7a65f d9b59ea58aaf 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (3m) 43s ago 3m 12.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 09024db02b67 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi053 running (3m) 84s ago 3m 14.0M - 19.0.0-2231-g2edfee68 237e43c7a65f f4cb4bb953a1 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi071 running (3m) 84s ago 3m 15.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 329a41c2a897 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (11m) 43s ago 11m 513M - 19.0.0-2231-g2edfee68 237e43c7a65f 2d661ad69bc3 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi053 *:8443,8765 running (9m) 84s ago 9m 455M - 19.0.0-2231-g2edfee68 237e43c7a65f 27f6ceb034e8 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi071 *:8443,8765 running (9m) 84s ago 9m 450M - 19.0.0-2231-g2edfee68 237e43c7a65f be3dcb7258ec 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (11m) 43s ago 11m 48.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 70bc41de4721 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi053 running (9m) 84s ago 9m 39.4M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f fe569e9ff565 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi071 running (9m) 84s ago 9m 40.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f afdc6583bd51 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (8m) 43s ago 8m 44.9M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 50b219b1586c 2024-03-21T09:11:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi053 running (8m) 84s ago 8m 46.0M 4271M 19.0.0-2231-g2edfee68 237e43c7a65f f7648173707c 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi071 running (7m) 84s ago 7m 61.4M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 69ef66ea9b00 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (7m) 43s ago 7m 54.9M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 248f5c3a4f3d 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi053 running (6m) 84s ago 6m 57.3M 4271M 19.0.0-2231-g2edfee68 237e43c7a65f fade87b91fd0 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi071 running (6m) 84s ago 6m 47.0M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 29461e290e60 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (5m) 43s ago 5m 53.1M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 3bbf905b3400 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi053 running (5m) 84s ago 5m 56.7M 4271M 19.0.0-2231-g2edfee68 237e43c7a65f 515f4d74c876 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi071 running (5m) 84s ago 5m 45.6M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f a16f126fc91b 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (4m) 43s ago 4m 40.3M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f c85d11914a52 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi053 running (3m) 84s ago 3m 53.3M 4271M 19.0.0-2231-g2edfee68 237e43c7a65f 6da9ca9edd1e 2024-03-21T09:11:12.897 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi071 running (3m) 84s ago 3m 53.4M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f da17d48af0a8 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5680a3030 msgr2=0x7fd5680a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5680a3030 0x7fd5680a54f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd57400fb70 tx=0x7fd574019040 comp rx=0 tx=0).stop 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 msgr2=0x7fd58419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 0x7fd58419ac50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9c0 tx=0x7fd57000ecd0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 shutdown_connections 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd584103d70 0x7fd5841a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5841021d0 0x7fd58419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd5680a3030 0x7fd5680a54f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 --2- 172.21.15.37:0/1021949307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841033d0 0x7fd58419ac50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 >> 172.21.15.37:0/1021949307 conn(0x7fd5840fd730 msgr2=0x7fd584108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 shutdown_connections 2024-03-21T09:11:12.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:12.903+0000 7fd588e00700 1 -- 172.21.15.37:0/1021949307 wait complete. 2024-03-21T09:11:13.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:13 smithi037 ceph-mon[31830]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:13.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-21T09:11:13.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:13 smithi071 ceph-mon[36597]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:13.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:13 smithi053 ceph-mon[36169]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:14.196 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:14.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:14 smithi037 ceph-mon[31830]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:14.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:14 smithi071 ceph-mon[36597]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:14.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:14 smithi053 ceph-mon[36169]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:15.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:15 smithi071 ceph-mon[36597]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:15.914 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:15 smithi037 ceph-mon[31830]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:15.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:15 smithi053 ceph-mon[36169]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:16.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 -- 172.21.15.37:0/1074466570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0104410 msgr2=0x7f12b0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:16.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 --2- 172.21.15.37:0/1074466570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0104410 0x7f12b0104830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12a40099c0 tx=0x7f12a4009cd0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 -- 172.21.15.37:0/1074466570 shutdown_connections 2024-03-21T09:11:16.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 --2- 172.21.15.37:0/1074466570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0105fb0 0x7f12b010c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 --2- 172.21.15.37:0/1074466570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0104410 0x7f12b0104830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 --2- 172.21.15.37:0/1074466570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 -- 172.21.15.37:0/1074466570 >> 172.21.15.37:0/1074466570 conn(0x7f12b00ff9b0 msgr2=0x7f12b0101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 -- 172.21.15.37:0/1074466570 shutdown_connections 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.498+0000 7f12b8000700 1 -- 172.21.15.37:0/1074466570 wait complete. 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.499+0000 7f12b8000700 1 Processor -- start 2024-03-21T09:11:16.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 -- start start 2024-03-21T09:11:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0104410 0x7f12b019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0105fb0 0x7f12b01a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b0110030 con 0x7f12b0104410 2024-03-21T09:11:16.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b010feb0 con 0x7f12b0105fb0 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b8000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01101b0 con 0x7f12b0105610 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b4c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b6000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0105fb0 0x7f12b01a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b4c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55550/0 (socket says 172.21.15.37:55550) 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.500+0000 7f12b4c00700 1 -- 172.21.15.37:0/2746653691 learned_addr learned my addr 172.21.15.37:0/2746653691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b5600700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0104410 0x7f12b019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b4c00700 1 -- 172.21.15.37:0/2746653691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0105fb0 msgr2=0x7f12b01a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b4c00700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0105fb0 0x7f12b01a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b4c00700 1 -- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0104410 msgr2=0x7f12b019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b4c00700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0104410 0x7f12b019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b4c00700 1 -- 172.21.15.37:0/2746653691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b01a2270 con 0x7f12b0105610 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b5600700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0104410 0x7f12b019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:16.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12b4c00700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b019ab50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f129c00eb60 tx=0x7f129c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:16.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.501+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00cc90 con 0x7f12b0105610 2024-03-21T09:11:16.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.502+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b01a2560 con 0x7f12b0105610 2024-03-21T09:11:16.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.502+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f129c00ce30 con 0x7f12b0105610 2024-03-21T09:11:16.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.502+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0188e0 con 0x7f12b0105610 2024-03-21T09:11:16.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.502+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b01a2aa0 con 0x7f12b0105610 2024-03-21T09:11:16.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.503+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0068e50 con 0x7f12b0105610 2024-03-21T09:11:16.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.506+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f129c018a80 con 0x7f12b0105610 2024-03-21T09:11:16.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.508+0000 7f12ae000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12980a2fb0 0x7f12980a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:16.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.508+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f129c0cf9e0 con 0x7f12b0105610 2024-03-21T09:11:16.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.508+0000 7f12b5600700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12980a2fb0 0x7f12980a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:16.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.509+0000 7f12b5600700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12980a2fb0 0x7f12980a5470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12a400b580 tx=0x7f12a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:16.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.511+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f129c09c420 con 0x7f12b0105610 2024-03-21T09:11:16.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.751+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f12b00625d0 con 0x7f12980a2fb0 2024-03-21T09:11:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.761+0000 7f12ae000700 1 -- 172.21.15.37:0/2746653691 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f12b00625d0 con 0x7f12980a2fb0 2024-03-21T09:11:16.755 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-21T09:11:16.755 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 88s ago 3m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-03-21T09:11:16.755 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 88s ago 9m smithi037=x;smithi053=y;smithi071=z;count:3 2024-03-21T09:11:16.755 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 88s ago 9m smithi037:172.21.15.37=a;smithi053:172.21.15.53=b;smithi071:172.21.15.71=c;count:3 2024-03-21T09:11:16.755 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 88s ago - 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12980a2fb0 msgr2=0x7f12980a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12980a2fb0 0x7f12980a5470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12a400b580 tx=0x7f12a4005fb0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 msgr2=0x7f12b019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b019ab50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f129c00eb60 tx=0x7f129c00ee70 comp rx=0 tx=0).stop 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 shutdown_connections 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12980a2fb0 0x7f12980a5470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b0105fb0 0x7f12b01a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b0104410 0x7f12b019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 --2- 172.21.15.37:0/2746653691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b0105610 0x7f12b019ab50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 >> 172.21.15.37:0/2746653691 conn(0x7f12b00ff9b0 msgr2=0x7f12b0108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.764+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 shutdown_connections 2024-03-21T09:11:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:16.765+0000 7f12b8000700 1 -- 172.21.15.37:0/2746653691 wait complete. 2024-03-21T09:11:17.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-21T09:11:17.833 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:17 smithi037 ceph-mon[31830]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:17.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:17 smithi071 ceph-mon[36597]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:17.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:17 smithi053 ceph-mon[36169]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:18.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:18 smithi037 ceph-mon[31830]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:18.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:18 smithi071 ceph-mon[36597]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:18.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:18 smithi053 ceph-mon[36169]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:19.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:19 smithi071 ceph-mon[36597]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:19.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:19 smithi037 ceph-mon[31830]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:19.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:19 smithi053 ceph-mon[36169]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 -- 172.21.15.37:0/3230460798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934102c50 msgr2=0x7f9934105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 --2- 172.21.15.37:0/3230460798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934102c50 0x7f9934105040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9924009990 tx=0x7f9924009ca0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 -- 172.21.15.37:0/3230460798 shutdown_connections 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 --2- 172.21.15.37:0/3230460798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934107eb0 0x7f993410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 --2- 172.21.15.37:0/3230460798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934102c50 0x7f9934105040 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 --2- 172.21.15.37:0/3230460798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9934105580 0x7f9934107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 -- 172.21.15.37:0/3230460798 >> 172.21.15.37:0/3230460798 conn(0x7f99340fc660 msgr2=0x7f99340feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 -- 172.21.15.37:0/3230460798 shutdown_connections 2024-03-21T09:11:20.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.531+0000 7f9939c00700 1 -- 172.21.15.37:0/3230460798 wait complete. 2024-03-21T09:11:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.532+0000 7f9939c00700 1 Processor -- start 2024-03-21T09:11:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 -- start start 2024-03-21T09:11:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 0x7f99341025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934105580 0x7f9934100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9934107eb0 0x7f9934101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993410de20 con 0x7f9934107eb0 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993410dca0 con 0x7f9934105580 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9939c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993410dfa0 con 0x7f9934102c50 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9932a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 0x7f99341025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9932a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 0x7f99341025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55568/0 (socket says 172.21.15.37:55568) 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9932a00700 1 -- 172.21.15.37:0/3960814943 learned_addr learned my addr 172.21.15.37:0/3960814943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9932000700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934105580 0x7f9934100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.533+0000 7f9933400700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9934107eb0 0x7f9934101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9932a00700 1 -- 172.21.15.37:0/3960814943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934105580 msgr2=0x7f9934100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9932a00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934105580 0x7f9934100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9932a00700 1 -- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9934107eb0 msgr2=0x7f9934101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9932a00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9934107eb0 0x7f9934101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9932a00700 1 -- 172.21.15.37:0/3960814943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99341019f0 con 0x7f9934102c50 2024-03-21T09:11:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9932a00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 0x7f99341025b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f992400b2e0 tx=0x7f992400bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924026070 con 0x7f9934102c50 2024-03-21T09:11:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9924003d70 con 0x7f9934102c50 2024-03-21T09:11:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924021410 con 0x7f9934102c50 2024-03-21T09:11:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.534+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99341a25a0 con 0x7f9934102c50 2024-03-21T09:11:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.535+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99341a29f0 con 0x7f9934102c50 2024-03-21T09:11:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.536+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9934068e50 con 0x7f9934102c50 2024-03-21T09:11:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.538+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9924029070 con 0x7f9934102c50 2024-03-21T09:11:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.540+0000 7f992b400700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f99180a3030 0x7f99180a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.541+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f99240d04e0 con 0x7f9934102c50 2024-03-21T09:11:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.541+0000 7f9932000700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f99180a3030 0x7f99180a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:20.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.541+0000 7f9932000700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f99180a3030 0x7f99180a54f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9920009c60 tx=0x7f9920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.543+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f992405d740 con 0x7f9934102c50 2024-03-21T09:11:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.779+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f99340625d0 con 0x7f99180a3030 2024-03-21T09:11:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.782+0000 7f992b400700 1 -- 172.21.15.37:0/3960814943 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7f99340625d0 con 0x7f99180a3030 2024-03-21T09:11:20.775 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-03-21T09:11:20.775 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-03-21T09:11:20.775 INFO:teuthology.orchestra.run.smithi037.stdout:smithi053 172.21.15.53 2024-03-21T09:11:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:smithi071 172.21.15.71 2024-03-21T09:11:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-03-21T09:11:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.785+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f99180a3030 msgr2=0x7f99180a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.785+0000 7f9939c00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f99180a3030 0x7f99180a54f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9920009c60 tx=0x7f9920009260 comp rx=0 tx=0).stop 2024-03-21T09:11:20.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.785+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 msgr2=0x7f99341025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.785+0000 7f9939c00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 0x7f99341025b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f992400b2e0 tx=0x7f992400bbb0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 shutdown_connections 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9934107eb0 0x7f9934101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9934102c50 0x7f99341025b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f99180a3030 0x7f99180a54f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 --2- 172.21.15.37:0/3960814943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9934105580 0x7f9934100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 >> 172.21.15.37:0/3960814943 conn(0x7f99340fc660 msgr2=0x7f9934103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 shutdown_connections 2024-03-21T09:11:20.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:20.786+0000 7f9939c00700 1 -- 172.21.15.37:0/3960814943 wait complete. 2024-03-21T09:11:21.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-21T09:11:21.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:21 smithi037 ceph-mon[31830]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:21.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:21 smithi071 ceph-mon[36597]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:21.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:21 smithi053 ceph-mon[36169]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:22.263 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:22.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:22 smithi037 ceph-mon[31830]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:22.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:22 smithi071 ceph-mon[36597]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:22.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:22 smithi053 ceph-mon[36169]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:23.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:23 smithi037 ceph-mon[31830]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:23.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:23 smithi071 ceph-mon[36597]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:23.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:23 smithi053 ceph-mon[36169]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 -- 172.21.15.37:0/1177123119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494101d60 msgr2=0x7f34941021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 --2- 172.21.15.37:0/1177123119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494101d60 0x7f34941021e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b0a0 tx=0x7f348c00b3b0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 -- 172.21.15.37:0/1177123119 shutdown_connections 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 --2- 172.21.15.37:0/1177123119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494101d60 0x7f34941021e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 --2- 172.21.15.37:0/1177123119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3494100a40 0x7f3494100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 --2- 172.21.15.37:0/1177123119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34941013a0 0x7f3494101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 -- 172.21.15.37:0/1177123119 >> 172.21.15.37:0/1177123119 conn(0x7f34940fc660 msgr2=0x7f34940feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 -- 172.21.15.37:0/1177123119 shutdown_connections 2024-03-21T09:11:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.679+0000 7f3499c00700 1 -- 172.21.15.37:0/1177123119 wait complete. 2024-03-21T09:11:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.680+0000 7f3499c00700 1 Processor -- start 2024-03-21T09:11:24.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 -- start start 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 0x7f34941065c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34941013a0 0x7f3494106b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3494101d60 0x7f3494103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349410ffc0 con 0x7f34941013a0 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349410fe40 con 0x7f3494101d60 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3499c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494110140 con 0x7f3494100a40 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 0x7f34941065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 0x7f34941065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33598/0 (socket says 172.21.15.37:33598) 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3492a00700 1 -- 172.21.15.37:0/657087455 learned_addr learned my addr 172.21.15.37:0/657087455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3492000700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34941013a0 0x7f3494106b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:24.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.681+0000 7f3493400700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3494101d60 0x7f3494103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3492a00700 1 -- 172.21.15.37:0/657087455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3494101d60 msgr2=0x7f3494103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3492a00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3494101d60 0x7f3494103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3492a00700 1 -- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34941013a0 msgr2=0x7f3494106b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3492a00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34941013a0 0x7f3494106b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3492a00700 1 -- 172.21.15.37:0/657087455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494103a10 con 0x7f3494100a40 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3492a00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 0x7f34941065c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f348400eb30 tx=0x7f348400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348400cc90 con 0x7f3494100a40 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3484004500 con 0x7f3494100a40 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34941a69e0 con 0x7f3494100a40 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.682+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484010430 con 0x7f3494100a40 2024-03-21T09:11:24.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.683+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34941a6e90 con 0x7f3494100a40 2024-03-21T09:11:24.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.684+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494068e50 con 0x7f3494100a40 2024-03-21T09:11:24.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.685+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3484004020 con 0x7f3494100a40 2024-03-21T09:11:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.686+0000 7f348b400700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34780a3030 0x7f34780a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.687+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f34840cfa90 con 0x7f3494100a40 2024-03-21T09:11:24.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.687+0000 7f3492000700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34780a3030 0x7f34780a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:24.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.687+0000 7f3492000700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34780a3030 0x7f34780a54f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3480005950 tx=0x7f348000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:24.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.690+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f348409c4d0 con 0x7f3494100a40 2024-03-21T09:11:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.927+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f34940625d0 con 0x7f34780a3030 2024-03-21T09:11:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.931+0000 7f348b400700 1 -- 172.21.15.37:0/657087455 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f34940625d0 con 0x7f34780a3030 2024-03-21T09:11:24.925 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-21T09:11:24.926 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-21T09:11:24.926 INFO:teuthology.orchestra.run.smithi037.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-03-21T09:11:24.926 INFO:teuthology.orchestra.run.smithi037.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-21T09:11:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34780a3030 msgr2=0x7f34780a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34780a3030 0x7f34780a54f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3480005950 tx=0x7f348000b410 comp rx=0 tx=0).stop 2024-03-21T09:11:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 msgr2=0x7f34941065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 0x7f34941065c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f348400eb30 tx=0x7f348400ee40 comp rx=0 tx=0).stop 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 shutdown_connections 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f34780a3030 0x7f34780a54f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3494101d60 0x7f3494103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494100a40 0x7f34941065c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 --2- 172.21.15.37:0/657087455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34941013a0 0x7f3494106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.935+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 >> 172.21.15.37:0/657087455 conn(0x7f34940fc660 msgr2=0x7f34940fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.936+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 shutdown_connections 2024-03-21T09:11:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:24.936+0000 7f3499c00700 1 -- 172.21.15.37:0/657087455 wait complete. 2024-03-21T09:11:25.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:25 smithi037 ceph-mon[31830]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:25.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:25 smithi071 ceph-mon[36597]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:25.928 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T09:11:25.938 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:11:25.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T09:11:25.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:25 smithi053 ceph-mon[36169]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:26.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:26 smithi037 ceph-mon[31830]: from='client.25234 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:26.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:26 smithi071 ceph-mon[36597]: from='client.25234 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:26.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:26 smithi053 ceph-mon[36169]: from='client.25234 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:27.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:27 smithi071 ceph-mon[36597]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:27.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:27 smithi037 ceph-mon[31830]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:27.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:27 smithi053 ceph-mon[36169]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:28.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 -- 172.21.15.37:0/435505812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 msgr2=0x7fcc14103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:28.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 --2- 172.21.15.37:0/435505812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14103760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc000099c0 tx=0x7fcc00009cd0 comp rx=0 tx=0).stop 2024-03-21T09:11:28.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 -- 172.21.15.37:0/435505812 shutdown_connections 2024-03-21T09:11:28.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 --2- 172.21.15.37:0/435505812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc14103ca0 0x7fcc1410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:28.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 --2- 172.21.15.37:0/435505812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:28.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 --2- 172.21.15.37:0/435505812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14103760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:28.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 -- 172.21.15.37:0/435505812 >> 172.21.15.37:0/435505812 conn(0x7fcc140fd6a0 msgr2=0x7fcc140ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:28.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 -- 172.21.15.37:0/435505812 shutdown_connections 2024-03-21T09:11:28.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.802+0000 7fcc1a800700 1 -- 172.21.15.37:0/435505812 wait complete. 2024-03-21T09:11:28.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.803+0000 7fcc1a800700 1 Processor -- start 2024-03-21T09:11:28.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.803+0000 7fcc1a800700 1 -- start start 2024-03-21T09:11:28.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc1a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:28.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc1a800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:28.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc1a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc14103ca0 0x7fcc14074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc1a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1410dd20 con 0x7fcc14103ca0 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc1a800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1410dba0 con 0x7fcc14103300 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc1a800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1410dea0 con 0x7fcc14102100 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc19400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc18a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc19400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33610/0 (socket says 172.21.15.37:33610) 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc19400700 1 -- 172.21.15.37:0/1847775761 learned_addr learned my addr 172.21.15.37:0/1847775761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.804+0000 7fcc19e00700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc14103ca0 0x7fcc14074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc19400700 1 -- 172.21.15.37:0/1847775761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 msgr2=0x7fcc14074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc19400700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc19400700 1 -- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc14103ca0 msgr2=0x7fcc14074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc19400700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc14103ca0 0x7fcc14074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc19400700 1 -- 172.21.15.37:0/1847775761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14075370 con 0x7fcc14102100 2024-03-21T09:11:28.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc18a00700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc19400700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14075f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e970 tx=0x7fcc0800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004220 con 0x7fcc14102100 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc080043c0 con 0x7fcc14102100 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc141a2460 con 0x7fcc14102100 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.805+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08003dc0 con 0x7fcc14102100 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.806+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc141a29a0 con 0x7fcc14102100 2024-03-21T09:11:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.807+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc14068e50 con 0x7fcc14102100 2024-03-21T09:11:28.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.808+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcc0800c7e0 con 0x7fcc14102100 2024-03-21T09:11:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.810+0000 7fcc12000700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcbfc0a3030 0x7fcbfc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.810+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcc080cf930 con 0x7fcc14102100 2024-03-21T09:11:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.810+0000 7fcc18a00700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcbfc0a3030 0x7fcbfc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.811+0000 7fcc18a00700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcbfc0a3030 0x7fcbfc0a54f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc00003760 tx=0x7fcc00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:28.818+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc0809c370 con 0x7fcc14102100 2024-03-21T09:11:29.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.097+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcc14062da0 con 0x7fcc14102100 2024-03-21T09:11:29.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.098+0000 7fcc12000700 1 -- 172.21.15.37:0/1847775761 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2980 (secure 0 0 0) 0x7fcc0809bc70 con 0x7fcc14102100 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:e23 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 23 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-21T09:08:46.306108+0000 2024-03-21T09:11:29.094 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-21T09:09:20.795215+0000 2024-03-21T09:11:29.095 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 3 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2 2024-03-21T09:11:29.098 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24589,1=24575,2=24553} 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3,4] 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24589 members: 24553,24575,24589 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{0:24589} state up:active seq 4 addr [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{0:24535} state up:standby-replay seq 1 addr [v2:172.21.15.71:6836/2290408711,v1:172.21.15.71:6837/2290408711] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{1:24575} state up:active seq 11 addr [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{1:24523} state up:standby-replay seq 1 addr [v2:172.21.15.71:6834/2027751164,v1:172.21.15.71:6835/2027751164] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.099 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{2:24553} state up:active seq 14 addr [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{2:24518} state up:standby-replay seq 1 addr [v2:172.21.15.53:6836/3665426355,v1:172.21.15.53:6837/3665426355] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.37:6834/3418620449,v1:172.21.15.37:6835/3418620449] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14400} state up:standby seq 1 addr [v2:172.21.15.37:6836/495294644,v1:172.21.15.37:6837/495294644] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.37:6838/508705459,v1:172.21.15.37:6839/508705459] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.53:6832/1353662837,v1:172.21.15.53:6833/1353662837] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24463} state up:standby seq 1 addr [v2:172.21.15.71:6832/3927519167,v1:172.21.15.71:6833/3927519167] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.100 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{-1:24491} state up:standby seq 1 addr [v2:172.21.15.53:6834/1616524011,v1:172.21.15.53:6835/1616524011] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcbfc0a3030 msgr2=0x7fcbfc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcbfc0a3030 0x7fcbfc0a54f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc00003760 tx=0x7fcc00005fb0 comp rx=0 tx=0).stop 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 msgr2=0x7fcc14075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14075f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e970 tx=0x7fcc0800ec80 comp rx=0 tx=0).stop 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 shutdown_connections 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc14103ca0 0x7fcc14074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14102100 0x7fcc14075f30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:29.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fcbfc0a3030 0x7fcbfc0a54f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 --2- 172.21.15.37:0/1847775761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc14103300 0x7fcc14074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 >> 172.21.15.37:0/1847775761 conn(0x7fcc140fd6a0 msgr2=0x7fcc14106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 shutdown_connections 2024-03-21T09:11:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:29.103+0000 7fcc1a800700 1 -- 172.21.15.37:0/1847775761 wait complete. 2024-03-21T09:11:29.102 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:11:29.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:29 smithi071 ceph-mon[36597]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:29.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:29 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1847775761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T09:11:29.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:29 smithi037 ceph-mon[31830]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:29.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:29 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1847775761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T09:11:29.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:29 smithi053 ceph-mon[36169]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:29.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:29 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1847775761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T09:11:29.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-21T09:11:30.485 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:31.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:31 smithi071 ceph-mon[36597]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:31.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:31 smithi037 ceph-mon[31830]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:31.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:31 smithi053 ceph-mon[36169]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 -- 172.21.15.37:0/1465101460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 msgr2=0x7f43d0078fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 --2- 172.21.15.37:0/1465101460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 0x7f43d0078fd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43c800b0a0 tx=0x7f43c800b3b0 comp rx=0 tx=0).stop 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 -- 172.21.15.37:0/1465101460 shutdown_connections 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 --2- 172.21.15.37:0/1465101460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 0x7f43d0078fd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 --2- 172.21.15.37:0/1465101460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d0076f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 --2- 172.21.15.37:0/1465101460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 0x7f43d0078210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 -- 172.21.15.37:0/1465101460 >> 172.21.15.37:0/1465101460 conn(0x7f43d00ff950 msgr2=0x7f43d0101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:32.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 -- 172.21.15.37:0/1465101460 shutdown_connections 2024-03-21T09:11:32.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.746+0000 7f43d6200700 1 -- 172.21.15.37:0/1465101460 wait complete. 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.747+0000 7f43d6200700 1 Processor -- start 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.747+0000 7f43d6200700 1 -- start start 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.747+0000 7f43d6200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43d6200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 0x7f43d019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43d6200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 0x7f43d01a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43d6200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0114450 con 0x7f43d0077d90 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43d6200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01142d0 con 0x7f43d0078750 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43d6200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01145d0 con 0x7f43d0076b40 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33640/0 (socket says 172.21.15.37:33640) 2024-03-21T09:11:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 -- 172.21.15.37:0/2525724939 learned_addr learned my addr 172.21.15.37:0/2525724939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43c7e00700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 0x7f43d019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 -- 172.21.15.37:0/2525724939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 msgr2=0x7f43d01a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 0x7f43d01a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 -- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 msgr2=0x7f43d019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 0x7f43d019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43cf400700 1 -- 172.21.15.37:0/2525724939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d01a65a0 con 0x7f43d0076b40 2024-03-21T09:11:32.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.748+0000 7f43c7e00700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 0x7f43d019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.749+0000 7f43cf400700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d019e940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.749+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000cbd0 con 0x7f43d0076b40 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.749+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43c000cd70 con 0x7f43d0076b40 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.749+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d01a6890 con 0x7f43d0076b40 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.749+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0010430 con 0x7f43d0076b40 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.749+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d01a6dd0 con 0x7f43d0076b40 2024-03-21T09:11:32.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.751+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d0068e50 con 0x7f43d0076b40 2024-03-21T09:11:32.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.753+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f43c0004830 con 0x7f43d0076b40 2024-03-21T09:11:32.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.756+0000 7f43ccc00700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f43b40a3030 0x7f43b40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:32.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.756+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f43c00cfb70 con 0x7f43d0076b40 2024-03-21T09:11:32.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.756+0000 7f43c7e00700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f43b40a3030 0x7f43b40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:32.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.757+0000 7f43c7e00700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f43b40a3030 0x7f43b40a54f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009be0 tx=0x7f43b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:32.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:32.763+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f43c009c5b0 con 0x7f43d0076b40 2024-03-21T09:11:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:11:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:32 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:32 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:32.941 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:32 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.028+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f43d002da30 con 0x7f43d0076b40 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.029+0000 7f43ccc00700 1 -- 172.21.15.37:0/2525724939 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) v1 ==== 56+0+5625 (secure 0 0 0) 0x7f43c009beb0 con 0x7f43d0076b40 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 94 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-21T08:59:29.362739+0000 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-21T09:10:47.397472+0000 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 27 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/2693651580,v1:172.21.15.37:6803/2693651580] [v2:172.21.15.37:6804/2693651580,v1:172.21.15.37:6805/2693651580] exists,up f8ca1c1e-a53f-43bb-b2d7-0df467be85b4 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/916967711,v1:172.21.15.53:6801/916967711] [v2:172.21.15.53:6802/916967711,v1:172.21.15.53:6803/916967711] exists,up 97aea20a-8dd3-4a5d-a5a5-341cd7508568 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/1968250372,v1:172.21.15.71:6801/1968250372] [v2:172.21.15.71:6802/1968250372,v1:172.21.15.71:6803/1968250372] exists,up 3d5b09e0-acc1-488c-8e3d-34c05699a51e 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/3211293526,v1:172.21.15.37:6811/3211293526] [v2:172.21.15.37:6812/3211293526,v1:172.21.15.37:6813/3211293526] exists,up 879c9049-bfc3-41fa-be8e-354edcc68a14 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/1561544362,v1:172.21.15.53:6809/1561544362] [v2:172.21.15.53:6810/1561544362,v1:172.21.15.53:6811/1561544362] exists,up 721729f1-98cf-4ce4-b0e6-3a0d9736d2d0 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 38 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/2412287007,v1:172.21.15.71:6809/2412287007] [v2:172.21.15.71:6810/2412287007,v1:172.21.15.71:6811/2412287007] exists,up ee6f4f1f-25a9-4e57-a211-178500d9cf88 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 46 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/1203908785,v1:172.21.15.37:6819/1203908785] [v2:172.21.15.37:6820/1203908785,v1:172.21.15.37:6821/1203908785] exists,up 6de23659-fb28-4c49-988a-84c26500a853 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 51 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/884972742,v1:172.21.15.53:6817/884972742] [v2:172.21.15.53:6818/884972742,v1:172.21.15.53:6819/884972742] exists,up 344d4a58-75bc-4741-b385-e590feb5fcd8 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 59 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/4216652236,v1:172.21.15.71:6817/4216652236] [v2:172.21.15.71:6818/4216652236,v1:172.21.15.71:6819/4216652236] exists,up e2d7e521-1571-45c7-98d2-3a7b3e876488 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/3963720627,v1:172.21.15.37:6827/3963720627] [v2:172.21.15.37:6828/3963720627,v1:172.21.15.37:6829/3963720627] exists,up c7ee448c-645a-44ee-aa9d-f8e9b5898ce0 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/3560446684,v1:172.21.15.53:6825/3560446684] [v2:172.21.15.53:6826/3560446684,v1:172.21.15.53:6827/3560446684] exists,up a4c10007-9a20-4448-981e-e97b3650beca 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 81 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6824/3952075855,v1:172.21.15.71:6825/3952075855] [v2:172.21.15.71:6826/3952075855,v1:172.21.15.71:6827/3952075855] exists,up ae4df7bc-7bbd-421b-8031-0599545220d8 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.13 [2,11] 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1c [8,11] 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.0 [5,9] 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.19 [3,7] 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.1d [4,6] 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.21 [4,8] 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.38 [3,9] 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.3c [4,9] 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.c [9,10] 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.e [5,8] 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.13 [9,8] 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.1f [9,8] 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2468741411 expires 2024-03-22T09:00:45.623535+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/798132403 expires 2024-03-22T09:00:45.623535+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2055653180 expires 2024-03-22T09:00:11.515361+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1907385341 expires 2024-03-22T09:00:11.515361+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2644652589 expires 2024-03-22T09:00:11.515361+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/276911571 expires 2024-03-22T09:00:11.515361+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/276911571 expires 2024-03-22T09:00:11.515361+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3643193292 expires 2024-03-22T09:00:45.623535+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2468741411 expires 2024-03-22T09:00:45.623535+0000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/295866901 expires 2024-03-22T09:00:45.623535+0000 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f43b40a3030 msgr2=0x7f43b40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f43b40a3030 0x7f43b40a54f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009be0 tx=0x7f43b8009260 comp rx=0 tx=0).stop 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 msgr2=0x7f43d019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d019e940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e9f0 tx=0x7f43c000edb0 comp rx=0 tx=0).stop 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 shutdown_connections 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f43b40a3030 0x7f43b40a54f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43d0078750 0x7f43d01a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0076b40 0x7f43d019e940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 --2- 172.21.15.37:0/2525724939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d0077d90 0x7f43d019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 >> 172.21.15.37:0/2525724939 conn(0x7f43d00ff950 msgr2=0x7f43d0101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 shutdown_connections 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:33.032+0000 7f43d6200700 1 -- 172.21.15.37:0/2525724939 wait complete. 2024-03-21T09:11:33.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:33 smithi053 ceph-mon[36169]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:33.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:33 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2525724939' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T09:11:33.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:33 smithi071 ceph-mon[36597]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:33.852 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:33 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2525724939' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T09:11:33.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:33 smithi037 ceph-mon[31830]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:33.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:33 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2525724939' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T09:11:34.065 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-21T09:11:34.079 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:34.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3878400076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b81032e0 msgr2=0x7fa2b8109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3878400076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b81032e0 0x7fa2b8109b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009be0 tx=0x7fa2b002bef0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3878400076 shutdown_connections 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3878400076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b81032e0 0x7fa2b8109b70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3878400076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 0x7fa2b8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3878400076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b8101730 0x7fa2b8101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3878400076 >> 172.21.15.37:0/3878400076 conn(0x7fa2b80fccc0 msgr2=0x7fa2b80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3878400076 shutdown_connections 2024-03-21T09:11:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.287+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3878400076 wait complete. 2024-03-21T09:11:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.288+0000 7fa2bfe00640 1 Processor -- start 2024-03-21T09:11:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.288+0000 7fa2bfe00640 1 -- start start 2024-03-21T09:11:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.288+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 0x7fa2b8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.288+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 0x7fa2b819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b81032e0 0x7fa2b81a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bfe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810d370 con 0x7fa2b81032e0 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bfe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810d1f0 con 0x7fa2b8101730 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bfe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810d4f0 con 0x7fa2b8102920 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 0x7fa2b8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 0x7fa2b819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 0x7fa2b8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54432/0 (socket says 172.21.15.37:54432) 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 0x7fa2b819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47650/0 (socket says 172.21.15.37:47650) 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 -- 172.21.15.37:0/3635993739 learned_addr learned my addr 172.21.15.37:0/3635993739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bde00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b81032e0 0x7fa2b81a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 -- 172.21.15.37:0/3635993739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 msgr2=0x7fa2b819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 0x7fa2b819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 -- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b81032e0 msgr2=0x7fa2b81a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b81032e0 0x7fa2b81a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.289+0000 7fa2bd400640 1 -- 172.21.15.37:0/3635993739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b81a1760 con 0x7fa2b8101730 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.290+0000 7fa2bd400640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 0x7fa2b8199b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e9c0 tx=0x7fa2ac00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.290+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac00ccc0 con 0x7fa2b8101730 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.290+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac0035f0 con 0x7fa2b8101730 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.290+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac00ce60 con 0x7fa2b8101730 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.290+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b81a1a50 con 0x7fa2b8101730 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.290+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b81a1ff0 con 0x7fa2b8101730 2024-03-21T09:11:34.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.292+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b81a2280 con 0x7fa2b8101730 2024-03-21T09:11:34.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.295+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa2ac0037a0 con 0x7fa2b8101730 2024-03-21T09:11:34.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.297+0000 7fa2b6000640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2a00a2fa0 0x7fa2a00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.297+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa2ac0cfa20 con 0x7fa2b8101730 2024-03-21T09:11:34.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.297+0000 7fa2bca00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2a00a2fa0 0x7fa2a00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.298+0000 7fa2bca00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2a00a2fa0 0x7fa2a00a5460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa2b819b050 tx=0x7fa2a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.301+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2ac09c4c0 con 0x7fa2b8101730 2024-03-21T09:11:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.534+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2b81a2450 con 0x7fa2b8101730 2024-03-21T09:11:34.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.536+0000 7fa2b6000640 1 -- 172.21.15.37:0/3635993739 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7fa2ac09bdc0 con 0x7fa2b8101730 2024-03-21T09:11:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:34.532 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2a00a2fa0 msgr2=0x7fa2a00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2a00a2fa0 0x7fa2a00a5460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa2b819b050 tx=0x7fa2a4009260 comp rx=0 tx=0).stop 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 msgr2=0x7fa2b8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 0x7fa2b8199b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e9c0 tx=0x7fa2ac00edc0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 shutdown_connections 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2a00a2fa0 0x7fa2a00a5460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b81032e0 0x7fa2b81a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b8102920 0x7fa2b819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 --2- 172.21.15.37:0/3635993739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8101730 0x7fa2b8199b40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 >> 172.21.15.37:0/3635993739 conn(0x7fa2b80fccc0 msgr2=0x7fa2b80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 shutdown_connections 2024-03-21T09:11:34.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.540+0000 7fa2bfe00640 1 -- 172.21.15.37:0/3635993739 wait complete. 2024-03-21T09:11:34.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:11:34.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 -- 172.21.15.37:0/2748407404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f41048f0 msgr2=0x7f12f4104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 --2- 172.21.15.37:0/2748407404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f41048f0 0x7f12f4104d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc02bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 -- 172.21.15.37:0/2748407404 shutdown_connections 2024-03-21T09:11:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 --2- 172.21.15.37:0/2748407404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12f41052b0 0x7f12f410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 --2- 172.21.15.37:0/2748407404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f41048f0 0x7f12f4104d70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 --2- 172.21.15.37:0/2748407404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103700 0x7f12f4103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.770+0000 7f12f3e00640 1 -- 172.21.15.37:0/2748407404 >> 172.21.15.37:0/2748407404 conn(0x7f12f40fec70 msgr2=0x7f12f41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:34.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.771+0000 7f12f3e00640 1 -- 172.21.15.37:0/2748407404 shutdown_connections 2024-03-21T09:11:34.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.771+0000 7f12f3e00640 1 -- 172.21.15.37:0/2748407404 wait complete. 2024-03-21T09:11:34.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 Processor -- start 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 -- start start 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f4103700 0x7f12f4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12f41048f0 0x7f12f4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41052b0 0x7f12f41a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f410f3b0 con 0x7f12f41048f0 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f410f230 con 0x7f12f4103700 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f410f530 con 0x7f12f41052b0 2024-03-21T09:11:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f4103700 0x7f12f4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f4103700 0x7f12f4199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54456/0 (socket says 172.21.15.37:54456) 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.772+0000 7f12f2a00640 1 -- 172.21.15.37:0/1526810013 learned_addr learned my addr 172.21.15.37:0/1526810013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41052b0 0x7f12f41a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12ebe00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12f41048f0 0x7f12f4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 -- 172.21.15.37:0/1526810013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f4103700 msgr2=0x7f12f4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f4103700 0x7f12f4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 -- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12f41048f0 msgr2=0x7f12f4199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12f41048f0 0x7f12f4199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 -- 172.21.15.37:0/1526810013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f41a1620 con 0x7f12f41052b0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.773+0000 7f12f3400640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41052b0 0x7f12f41a0fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00da10 tx=0x7f12ec00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.774+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00bd70 con 0x7f12f41052b0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.774+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12ec003e50 con 0x7f12f41052b0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.774+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f41a1910 con 0x7f12f41052b0 2024-03-21T09:11:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.774+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0106a0 con 0x7f12f41052b0 2024-03-21T09:11:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.774+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f41a1e30 con 0x7f12f41052b0 2024-03-21T09:11:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.775+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f410aa20 con 0x7f12f41052b0 2024-03-21T09:11:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.778+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f12ec003970 con 0x7f12f41052b0 2024-03-21T09:11:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.780+0000 7f12eb400640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12d80a2fa0 0x7f12d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:34.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.780+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f12ec0cfb10 con 0x7f12f41052b0 2024-03-21T09:11:34.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.780+0000 7f12f2a00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12d80a2fa0 0x7f12d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:34.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.781+0000 7f12f2a00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12d80a2fa0 0x7f12d80a5460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12e40058c0 tx=0x7f12e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:34.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:34.784+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12ec09c550 con 0x7f12f41052b0 2024-03-21T09:11:35.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.009+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12f402d0c0 con 0x7f12f41052b0 2024-03-21T09:11:35.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.009+0000 7f12eb400640 1 -- 172.21.15.37:0/1526810013 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12ec09be50 con 0x7f12f41052b0 2024-03-21T09:11:35.004 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12d80a2fa0 msgr2=0x7f12d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12d80a2fa0 0x7f12d80a5460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12e40058c0 tx=0x7f12e4005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41052b0 msgr2=0x7f12f41a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41052b0 0x7f12f41a0fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00da10 tx=0x7f12ec00ded0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 shutdown_connections 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f12d80a2fa0 0x7f12d80a5460 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41052b0 0x7f12f41a0fb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12f41048f0 0x7f12f4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 --2- 172.21.15.37:0/1526810013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12f4103700 0x7f12f4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 >> 172.21.15.37:0/1526810013 conn(0x7f12f40fec70 msgr2=0x7f12f4107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 shutdown_connections 2024-03-21T09:11:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.012+0000 7f12f3e00640 1 -- 172.21.15.37:0/1526810013 wait complete. 2024-03-21T09:11:35.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:35.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:34 smithi037 ceph-mon[31830]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:35.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:34 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:34 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:34 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:34 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:34 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3635993739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:35.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:34 smithi053 ceph-mon[36169]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:35.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:34 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:34 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:34 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:34 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:34 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3635993739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 -- 172.21.15.37:0/2987427715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241038d0 msgr2=0x7f6424103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 --2- 172.21.15.37:0/2987427715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241038d0 0x7f6424103cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6414009900 tx=0x7f641402bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 -- 172.21.15.37:0/2987427715 shutdown_connections 2024-03-21T09:11:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 --2- 172.21.15.37:0/2987427715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424105480 0x7f642410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 --2- 172.21.15.37:0/2987427715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6424104ac0 0x7f6424104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 --2- 172.21.15.37:0/2987427715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241038d0 0x7f6424103cf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 -- 172.21.15.37:0/2987427715 >> 172.21.15.37:0/2987427715 conn(0x7f64240fee60 msgr2=0x7f6424101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:35.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 -- 172.21.15.37:0/2987427715 shutdown_connections 2024-03-21T09:11:35.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.249+0000 7f6428a00640 1 -- 172.21.15.37:0/2987427715 wait complete. 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.250+0000 7f6428a00640 1 Processor -- start 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.250+0000 7f6428a00640 1 -- start start 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6428a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64241038d0 0x7f6424199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6428a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 0x7f642419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6428a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6424105480 0x7f64241a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6428a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642410f570 con 0x7f64241038d0 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6428a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642410f3f0 con 0x7f6424105480 2024-03-21T09:11:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6428a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642410f6f0 con 0x7f6424104ac0 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f641be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 0x7f642419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f641be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 0x7f642419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47680/0 (socket says 172.21.15.37:47680) 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6422800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6424105480 0x7f64241a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f641be00640 1 -- 172.21.15.37:0/1322283551 learned_addr learned my addr 172.21.15.37:0/1322283551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.251+0000 7f6421e00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64241038d0 0x7f6424199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641be00640 1 -- 172.21.15.37:0/1322283551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6424105480 msgr2=0x7f64241a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641be00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6424105480 0x7f64241a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641be00640 1 -- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64241038d0 msgr2=0x7f6424199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641be00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64241038d0 0x7f6424199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641be00640 1 -- 172.21.15.37:0/1322283551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64241a1840 con 0x7f6424104ac0 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f6421e00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64241038d0 0x7f6424199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f6422800640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6424105480 0x7f64241a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641be00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 0x7f642419a170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e9f0 tx=0x7f640c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00ccc0 con 0x7f6424104ac0 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f640c010910 con 0x7f6424104ac0 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64241a1b30 con 0x7f6424104ac0 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c0188a0 con 0x7f6424104ac0 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.252+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a2050 con 0x7f6424104ac0 2024-03-21T09:11:35.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.254+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64000052c0 con 0x7f6424104ac0 2024-03-21T09:11:35.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.255+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f640c010430 con 0x7f6424104ac0 2024-03-21T09:11:35.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.258+0000 7f641aa00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64080a2f20 0x7f64080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.258+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f640c0cfbf0 con 0x7f6424104ac0 2024-03-21T09:11:35.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.258+0000 7f6421e00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64080a2f20 0x7f64080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.259+0000 7f6421e00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64080a2f20 0x7f64080a53e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f641402c170 tx=0x7f641402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:35.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.260+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f640c09c630 con 0x7f6424104ac0 2024-03-21T09:11:35.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:34 smithi071 ceph-mon[36597]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:35.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:34 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:34 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:34 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:34 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:35.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:34 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3635993739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.486+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6400005dc0 con 0x7f6424104ac0 2024-03-21T09:11:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.487+0000 7f641aa00640 1 -- 172.21.15.37:0/1322283551 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f640c09bf30 con 0x7f6424104ac0 2024-03-21T09:11:35.482 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64080a2f20 msgr2=0x7f64080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64080a2f20 0x7f64080a53e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f641402c170 tx=0x7f641402df90 comp rx=0 tx=0).stop 2024-03-21T09:11:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 msgr2=0x7f642419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 0x7f642419a170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e9f0 tx=0x7f640c00edf0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 shutdown_connections 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f64080a2f20 0x7f64080a53e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6424105480 0x7f64241a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6424104ac0 0x7f642419a170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 --2- 172.21.15.37:0/1322283551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64241038d0 0x7f6424199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 >> 172.21.15.37:0/1322283551 conn(0x7f64240fee60 msgr2=0x7f6424107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.490+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 shutdown_connections 2024-03-21T09:11:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.491+0000 7f6428a00640 1 -- 172.21.15.37:0/1322283551 wait complete. 2024-03-21T09:11:35.510 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:35.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 -- 172.21.15.37:0/2222594321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 msgr2=0x7fa17010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 --2- 172.21.15.37:0/2222594321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 0x7fa17010bd20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa168009ca0 tx=0x7fa16802c100 comp rx=0 tx=0).stop 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 -- 172.21.15.37:0/2222594321 shutdown_connections 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 --2- 172.21.15.37:0/2222594321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 0x7fa17010bd20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 --2- 172.21.15.37:0/2222594321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa170104ad0 0x7fa170104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 --2- 172.21.15.37:0/2222594321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1701038e0 0x7fa170103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 -- 172.21.15.37:0/2222594321 >> 172.21.15.37:0/2222594321 conn(0x7fa1700fee50 msgr2=0x7fa170101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.735+0000 7fa174e00640 1 -- 172.21.15.37:0/2222594321 shutdown_connections 2024-03-21T09:11:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.736+0000 7fa174e00640 1 -- 172.21.15.37:0/2222594321 wait complete. 2024-03-21T09:11:35.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.736+0000 7fa174e00640 1 Processor -- start 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 -- start start 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1701038e0 0x7fa170199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 0x7fa17019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 0x7fa1701a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17010f540 con 0x7fa1701038e0 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17010f3c0 con 0x7fa170105490 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa174e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17010f6c0 con 0x7fa170104ad0 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa16d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 0x7fa17019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa16de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1701038e0 0x7fa170199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa16d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 0x7fa17019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47692/0 (socket says 172.21.15.37:47692) 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa16d400640 1 -- 172.21.15.37:0/1538452053 learned_addr learned my addr 172.21.15.37:0/1538452053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.737+0000 7fa16e800640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 0x7fa1701a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa16d400640 1 -- 172.21.15.37:0/1538452053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 msgr2=0x7fa1701a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa16d400640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 0x7fa1701a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa16d400640 1 -- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1701038e0 msgr2=0x7fa170199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa16d400640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1701038e0 0x7fa170199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa16d400640 1 -- 172.21.15.37:0/1538452053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1701a1780 con 0x7fa170104ad0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa16d400640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 0x7fa17019a0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00d950 tx=0x7fa15c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c017070 con 0x7fa170104ad0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c009c30 con 0x7fa170104ad0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1701a1a70 con 0x7fa170104ad0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00f6b0 con 0x7fa170104ad0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.738+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1701a2010 con 0x7fa170104ad0 2024-03-21T09:11:35.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.739+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17010ac00 con 0x7fa170104ad0 2024-03-21T09:11:35.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.741+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa15c0109d0 con 0x7fa170104ad0 2024-03-21T09:11:35.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.745+0000 7fa166a00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa1540a2fa0 0x7fa1540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:35.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.745+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa15c0cf300 con 0x7fa170104ad0 2024-03-21T09:11:35.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.745+0000 7fa16de00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa1540a2fa0 0x7fa1540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:35.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.746+0000 7fa16de00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa1540a2fa0 0x7fa1540a5460 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa1600095f0 tx=0x7fa160005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:35.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.747+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa15c009de0 con 0x7fa170104ad0 2024-03-21T09:11:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.983+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa170062340 con 0x7fa170104ad0 2024-03-21T09:11:35.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.983+0000 7fa166a00640 1 -- 172.21.15.37:0/1538452053 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa15c09b6f0 con 0x7fa170104ad0 2024-03-21T09:11:35.978 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa1540a2fa0 msgr2=0x7fa1540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa1540a2fa0 0x7fa1540a5460 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa1600095f0 tx=0x7fa160005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 msgr2=0x7fa17019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 0x7fa17019a0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00d950 tx=0x7fa15c00de10 comp rx=0 tx=0).stop 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 shutdown_connections 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa1540a2fa0 0x7fa1540a5460 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170105490 0x7fa1701a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa170104ad0 0x7fa17019a0a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 --2- 172.21.15.37:0/1538452053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1701038e0 0x7fa170199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:35.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 >> 172.21.15.37:0/1538452053 conn(0x7fa1700fee50 msgr2=0x7fa170107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:35.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 shutdown_connections 2024-03-21T09:11:35.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:35.986+0000 7fa174e00640 1 -- 172.21.15.37:0/1538452053 wait complete. 2024-03-21T09:11:35.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:36.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:35 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1526810013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:36.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:35 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1322283551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:36.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:35 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1526810013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:36.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:35 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1322283551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:36.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.209+0000 7fa81e400640 1 -- 172.21.15.37:0/2836224657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8181038e0 msgr2=0x7fa818103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.209+0000 7fa81e400640 1 --2- 172.21.15.37:0/2836224657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8181038e0 0x7fa818103d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa808009ca0 tx=0x7fa80802c100 comp rx=0 tx=0).stop 2024-03-21T09:11:36.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.209+0000 7fa81e400640 1 -- 172.21.15.37:0/2836224657 shutdown_connections 2024-03-21T09:11:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.209+0000 7fa81e400640 1 --2- 172.21.15.37:0/2836224657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa818105490 0x7fa81810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.209+0000 7fa81e400640 1 --2- 172.21.15.37:0/2836224657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa818104ad0 0x7fa818104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.209+0000 7fa81e400640 1 --2- 172.21.15.37:0/2836224657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8181038e0 0x7fa818103d00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.210+0000 7fa81e400640 1 -- 172.21.15.37:0/2836224657 >> 172.21.15.37:0/2836224657 conn(0x7fa8180fee50 msgr2=0x7fa818101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.210+0000 7fa81e400640 1 -- 172.21.15.37:0/2836224657 shutdown_connections 2024-03-21T09:11:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.210+0000 7fa81e400640 1 -- 172.21.15.37:0/2836224657 wait complete. 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 Processor -- start 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 -- start start 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8181038e0 0x7fa818199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 0x7fa81819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa818105490 0x7fa8181a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81810f560 con 0x7fa8181038e0 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81810f3e0 con 0x7fa818104ad0 2024-03-21T09:11:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa81e400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81810f6e0 con 0x7fa818105490 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 0x7fa81819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 0x7fa81819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54506/0 (socket says 172.21.15.37:54506) 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa80ea00640 1 -- 172.21.15.37:0/1080542134 learned_addr learned my addr 172.21.15.37:0/1080542134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa817400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8181038e0 0x7fa818199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.211+0000 7fa817e00640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa818105490 0x7fa8181a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa80ea00640 1 -- 172.21.15.37:0/1080542134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa818105490 msgr2=0x7fa8181a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa80ea00640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa818105490 0x7fa8181a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa80ea00640 1 -- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8181038e0 msgr2=0x7fa818199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa80ea00640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8181038e0 0x7fa818199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa80ea00640 1 -- 172.21.15.37:0/1080542134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8181a1830 con 0x7fa818104ad0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa80ea00640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 0x7fa81819a160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa80000d950 tx=0x7fa80000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.212+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800017070 con 0x7fa818104ad0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.213+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa800009c30 con 0x7fa818104ad0 2024-03-21T09:11:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.213+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8181a1b20 con 0x7fa818104ad0 2024-03-21T09:11:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.213+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80000f6b0 con 0x7fa818104ad0 2024-03-21T09:11:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.213+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8181a2090 con 0x7fa818104ad0 2024-03-21T09:11:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.214+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81810ac00 con 0x7fa818104ad0 2024-03-21T09:11:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.218+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa8000109d0 con 0x7fa818104ad0 2024-03-21T09:11:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.219+0000 7fa814c00640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa7fc0a2fa0 0x7fa7fc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.219+0000 7fa817400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa7fc0a2fa0 0x7fa7fc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.219+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa8000cf360 con 0x7fa818104ad0 2024-03-21T09:11:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.220+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa800009de0 con 0x7fa818104ad0 2024-03-21T09:11:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.220+0000 7fa817400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa7fc0a2fa0 0x7fa7fc0a5460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa80802c640 tx=0x7fa808009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:36.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:35 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1526810013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:36.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:35 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1322283551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:36.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.470+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa81802d2c0 con 0x7fa818104ad0 2024-03-21T09:11:36.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.471+0000 7fa814c00640 1 -- 172.21.15.37:0/1080542134 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa80009b7d0 con 0x7fa818104ad0 2024-03-21T09:11:36.466 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa7fc0a2fa0 msgr2=0x7fa7fc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa7fc0a2fa0 0x7fa7fc0a5460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa80802c640 tx=0x7fa808009360 comp rx=0 tx=0).stop 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 msgr2=0x7fa81819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 0x7fa81819a160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa80000d950 tx=0x7fa80000de10 comp rx=0 tx=0).stop 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 shutdown_connections 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa7fc0a2fa0 0x7fa7fc0a5460 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa818105490 0x7fa8181a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa818104ad0 0x7fa81819a160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 --2- 172.21.15.37:0/1080542134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8181038e0 0x7fa818199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 >> 172.21.15.37:0/1080542134 conn(0x7fa8180fee50 msgr2=0x7fa818107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 shutdown_connections 2024-03-21T09:11:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.475+0000 7fa81e400640 1 -- 172.21.15.37:0/1080542134 wait complete. 2024-03-21T09:11:36.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:36.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 -- 172.21.15.37:0/3629820006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c107420 msgr2=0x7f7c5c109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3629820006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c107420 0x7f7c5c109870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009ca0 tx=0x7f7c5402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 -- 172.21.15.37:0/3629820006 shutdown_connections 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3629820006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c107420 0x7f7c5c109870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3629820006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104af0 0x7f7c5c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3629820006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 -- 172.21.15.37:0/3629820006 >> 172.21.15.37:0/3629820006 conn(0x7f7c5c0fbbd0 msgr2=0x7f7c5c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 -- 172.21.15.37:0/3629820006 shutdown_connections 2024-03-21T09:11:36.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.712+0000 7f7c62c00640 1 -- 172.21.15.37:0/3629820006 wait complete. 2024-03-21T09:11:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.713+0000 7f7c62c00640 1 Processor -- start 2024-03-21T09:11:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 -- start start 2024-03-21T09:11:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c104af0 0x7f7c5c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c107420 0x7f7c5c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10d370 con 0x7f7c5c107420 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10d1f0 con 0x7f7c5c1021c0 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c62c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10d4f0 con 0x7f7c5c104af0 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c5be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c5be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54530/0 (socket says 172.21.15.37:54530) 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c5be00640 1 -- 172.21.15.37:0/3982229654 learned_addr learned my addr 172.21.15.37:0/3982229654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c60c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c107420 0x7f7c5c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.714+0000 7f7c5b400640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c104af0 0x7f7c5c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c5be00640 1 -- 172.21.15.37:0/3982229654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c104af0 msgr2=0x7f7c5c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c5be00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c104af0 0x7f7c5c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c5be00640 1 -- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c107420 msgr2=0x7f7c5c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c5be00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c107420 0x7f7c5c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c5be00640 1 -- 172.21.15.37:0/3982229654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c1a17e0 con 0x7f7c5c1021c0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c5be00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c199bc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00e9f0 tx=0x7f7c4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00cd80 con 0x7f7c5c1021c0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c010910 con 0x7f7c5c1021c0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c1a1ad0 con 0x7f7c5c1021c0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.715+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c018990 con 0x7f7c5c1021c0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.716+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c1a2070 con 0x7f7c5c1021c0 2024-03-21T09:11:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.717+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c108f10 con 0x7f7c5c1021c0 2024-03-21T09:11:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.719+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7c4c010ac0 con 0x7f7c5c1021c0 2024-03-21T09:11:36.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.721+0000 7f7c58c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7c400a2fa0 0x7f7c400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:36.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.721+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c4c0cfa40 con 0x7f7c5c1021c0 2024-03-21T09:11:36.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.721+0000 7f7c5b400640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7c400a2fa0 0x7f7c400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:36.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.722+0000 7f7c5b400640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7c400a2fa0 0x7f7c400a5460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c19b0d0 tx=0x7f7c48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:36.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.724+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c4c09be30 con 0x7f7c5c1021c0 2024-03-21T09:11:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.989+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c5c0622a0 con 0x7f7c5c1021c0 2024-03-21T09:11:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.990+0000 7f7c58c00640 1 -- 172.21.15.37:0/3982229654 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f7c4c09fed0 con 0x7f7c5c1021c0 2024-03-21T09:11:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:36.984 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:09:20.795215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:11:36.985 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7c400a2fa0 msgr2=0x7f7c400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7c400a2fa0 0x7f7c400a5460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c19b0d0 tx=0x7f7c48005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 msgr2=0x7f7c5c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c199bc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00e9f0 tx=0x7f7c4c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 shutdown_connections 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7c400a2fa0 0x7f7c400a5460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c107420 0x7f7c5c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c104af0 0x7f7c5c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 --2- 172.21.15.37:0/3982229654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c1021c0 0x7f7c5c199bc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 >> 172.21.15.37:0/3982229654 conn(0x7f7c5c0fbbd0 msgr2=0x7f7c5c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 shutdown_connections 2024-03-21T09:11:36.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:36.993+0000 7f7c62c00640 1 -- 172.21.15.37:0/3982229654 wait complete. 2024-03-21T09:11:37.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:37.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:36 smithi037 ceph-mon[31830]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:37.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:36 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1538452053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:37.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:36 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:37.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:36 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1080542134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:36 smithi053 ceph-mon[36169]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:37.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:36 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1538452053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:37.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:36 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:37.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:36 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1080542134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 -- 172.21.15.37:0/1476510514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0756c0 msgr2=0x7faf8c073b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 --2- 172.21.15.37:0/1476510514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0756c0 0x7faf8c073b10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faf80009900 tx=0x7faf8002bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 -- 172.21.15.37:0/1476510514 shutdown_connections 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 --2- 172.21.15.37:0/1476510514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c1095c0 0x7faf8c10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 --2- 172.21.15.37:0/1476510514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0740e0 0x7faf8c074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 --2- 172.21.15.37:0/1476510514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0756c0 0x7faf8c073b10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 -- 172.21.15.37:0/1476510514 >> 172.21.15.37:0/1476510514 conn(0x7faf8c0fcc80 msgr2=0x7faf8c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 -- 172.21.15.37:0/1476510514 shutdown_connections 2024-03-21T09:11:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.231+0000 7faf93800640 1 -- 172.21.15.37:0/1476510514 wait complete. 2024-03-21T09:11:37.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.232+0000 7faf93800640 1 Processor -- start 2024-03-21T09:11:37.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.232+0000 7faf93800640 1 -- start start 2024-03-21T09:11:37.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf93800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 0x7faf8c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf93800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0756c0 0x7faf8c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf93800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c1095c0 0x7faf8c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf93800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c10f550 con 0x7faf8c0756c0 2024-03-21T09:11:37.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf93800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c10f3d0 con 0x7faf8c1095c0 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf93800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c10f6d0 con 0x7faf8c0740e0 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf90e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 0x7faf8c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf90e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 0x7faf8c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47724/0 (socket says 172.21.15.37:47724) 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf90e00640 1 -- 172.21.15.37:0/2312516933 learned_addr learned my addr 172.21.15.37:0/2312516933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf91800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c1095c0 0x7faf8c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.233+0000 7faf8be00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0756c0 0x7faf8c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf90e00640 1 -- 172.21.15.37:0/2312516933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c1095c0 msgr2=0x7faf8c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf90e00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c1095c0 0x7faf8c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf90e00640 1 -- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0756c0 msgr2=0x7faf8c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf90e00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0756c0 0x7faf8c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf90e00640 1 -- 172.21.15.37:0/2312516933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c1a17a0 con 0x7faf8c0740e0 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf8be00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0756c0 0x7faf8c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf90e00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 0x7faf8c199b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faf80000c00 tx=0x7faf8002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80043020 con 0x7faf8c0740e0 2024-03-21T09:11:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf800054e0 con 0x7faf8c0740e0 2024-03-21T09:11:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c1a1a30 con 0x7faf8c0740e0 2024-03-21T09:11:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80003c40 con 0x7faf8c0740e0 2024-03-21T09:11:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.234+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c1a1fd0 con 0x7faf8c0740e0 2024-03-21T09:11:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.236+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c108340 con 0x7faf8c0740e0 2024-03-21T09:11:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.237+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faf80046020 con 0x7faf8c0740e0 2024-03-21T09:11:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.239+0000 7faf89600640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faf740a2fa0 0x7faf740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.239+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf800f1870 con 0x7faf8c0740e0 2024-03-21T09:11:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.239+0000 7faf8be00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faf740a2fa0 0x7faf740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.240+0000 7faf8be00640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faf740a2fa0 0x7faf740a5460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf8c19b090 tx=0x7faf78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.243+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faf8004d020 con 0x7faf8c0740e0 2024-03-21T09:11:37.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:36 smithi071 ceph-mon[36597]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:37.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:36 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1538452053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:37.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:36 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:37.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:36 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1080542134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.450+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf8c0622a0 con 0x7faf8c0740e0 2024-03-21T09:11:37.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.451+0000 7faf89600640 1 -- 172.21.15.37:0/2312516933 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7faf800be2b0 con 0x7faf8c0740e0 2024-03-21T09:11:37.447 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:37.447 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:37.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.454+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faf740a2fa0 msgr2=0x7faf740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.454+0000 7faf93800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faf740a2fa0 0x7faf740a5460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf8c19b090 tx=0x7faf78005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.454+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 msgr2=0x7faf8c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.454+0000 7faf93800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 0x7faf8c199b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faf80000c00 tx=0x7faf8002c650 comp rx=0 tx=0).stop 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 shutdown_connections 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7faf740a2fa0 0x7faf740a5460 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf8c1095c0 0x7faf8c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf8c0756c0 0x7faf8c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 --2- 172.21.15.37:0/2312516933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c0740e0 0x7faf8c199b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 >> 172.21.15.37:0/2312516933 conn(0x7faf8c0fcc80 msgr2=0x7faf8c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 shutdown_connections 2024-03-21T09:11:37.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.455+0000 7faf93800640 1 -- 172.21.15.37:0/2312516933 wait complete. 2024-03-21T09:11:37.469 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:11:37.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 -- 172.21.15.37:0/3489122877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1003f0 msgr2=0x7f1bcc100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/3489122877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc100810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009900 tx=0x7f1bbc02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 -- 172.21.15.37:0/3489122877 shutdown_connections 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/3489122877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc101fa0 0x7f1bcc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/3489122877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bcc1015e0 0x7f1bcc101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/3489122877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc100810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 -- 172.21.15.37:0/3489122877 >> 172.21.15.37:0/3489122877 conn(0x7f1bcc0fb980 msgr2=0x7f1bcc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 -- 172.21.15.37:0/3489122877 shutdown_connections 2024-03-21T09:11:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.697+0000 7f1bcbe00640 1 -- 172.21.15.37:0/3489122877 wait complete. 2024-03-21T09:11:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.698+0000 7f1bcbe00640 1 Processor -- start 2024-03-21T09:11:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.698+0000 7f1bcbe00640 1 -- start start 2024-03-21T09:11:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1015e0 0x7f1bcc199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bcc101fa0 0x7f1bcc1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcbe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc10c0d0 con 0x7f1bcc101fa0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc10bf50 con 0x7f1bcc1003f0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc10c250 con 0x7f1bcc1015e0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54570/0 (socket says 172.21.15.37:54570) 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bca000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1015e0 0x7f1bcc199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcaa00640 1 -- 172.21.15.37:0/734413098 learned_addr learned my addr 172.21.15.37:0/734413098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.699+0000 7f1bcb400640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bcc101fa0 0x7f1bcc1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcaa00640 1 -- 172.21.15.37:0/734413098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1015e0 msgr2=0x7f1bcc199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcaa00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1015e0 0x7f1bcc199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcaa00640 1 -- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bcc101fa0 msgr2=0x7f1bcc1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcaa00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bcc101fa0 0x7f1bcc1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcaa00640 1 -- 172.21.15.37:0/734413098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc1a1680 con 0x7f1bcc1003f0 2024-03-21T09:11:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bca000640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1015e0 0x7f1bcc199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:37.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcaa00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc199a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc043020 con 0x7f1bcc1003f0 2024-03-21T09:11:37.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc1a1910 con 0x7f1bcc1003f0 2024-03-21T09:11:37.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0054e0 con 0x7f1bcc1003f0 2024-03-21T09:11:37.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.700+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc003bd0 con 0x7f1bcc1003f0 2024-03-21T09:11:37.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.701+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc1a1e60 con 0x7f1bcc1003f0 2024-03-21T09:11:37.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.702+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc1a2030 con 0x7f1bcc1003f0 2024-03-21T09:11:37.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.703+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1bbc046020 con 0x7f1bcc1003f0 2024-03-21T09:11:37.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.705+0000 7f1bd0a00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1bb00a2f20 0x7f1bb00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.705+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1bbc0f1800 con 0x7f1bcc1003f0 2024-03-21T09:11:37.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.705+0000 7f1bca000640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1bb00a2f20 0x7f1bb00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.706+0000 7f1bca000640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1bb00a2f20 0x7f1bb00a53e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc19af70 tx=0x7f1bb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.708+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1bbc04d020 con 0x7f1bcc1003f0 2024-03-21T09:11:37.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.916+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bcc1a26b0 con 0x7f1bcc1003f0 2024-03-21T09:11:37.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.917+0000 7f1bd0a00640 1 -- 172.21.15.37:0/734413098 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bbc0be240 con 0x7f1bcc1003f0 2024-03-21T09:11:37.912 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:11:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1bb00a2f20 msgr2=0x7f1bb00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1bb00a2f20 0x7f1bb00a53e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc19af70 tx=0x7f1bb8005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 msgr2=0x7f1bcc199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc199a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc02c650 comp rx=0 tx=0).stop 2024-03-21T09:11:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 shutdown_connections 2024-03-21T09:11:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1bb00a2f20 0x7f1bb00a53e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bcc101fa0 0x7f1bcc1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bcc1015e0 0x7f1bcc199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 --2- 172.21.15.37:0/734413098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bcc1003f0 0x7f1bcc199a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 >> 172.21.15.37:0/734413098 conn(0x7f1bcc0fb980 msgr2=0x7f1bcc0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 shutdown_connections 2024-03-21T09:11:37.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:37.920+0000 7f1bcbe00640 1 -- 172.21.15.37:0/734413098 wait complete. 2024-03-21T09:11:37.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.153+0000 7ff32aa00640 1 -- 172.21.15.37:0/3975063823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff324101f70 msgr2=0x7ff324104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 --2- 172.21.15.37:0/3975063823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff324101f70 0x7ff324104360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff318009900 tx=0x7ff31802bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 -- 172.21.15.37:0/3975063823 shutdown_connections 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 --2- 172.21.15.37:0/3975063823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3241071d0 0x7ff324109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 --2- 172.21.15.37:0/3975063823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3241048a0 0x7ff324106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 --2- 172.21.15.37:0/3975063823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff324101f70 0x7ff324104360 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 -- 172.21.15.37:0/3975063823 >> 172.21.15.37:0/3975063823 conn(0x7ff3240fb980 msgr2=0x7ff3240fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 -- 172.21.15.37:0/3975063823 shutdown_connections 2024-03-21T09:11:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.154+0000 7ff32aa00640 1 -- 172.21.15.37:0/3975063823 wait complete. 2024-03-21T09:11:38.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.156+0000 7ff32aa00640 1 Processor -- start 2024-03-21T09:11:38.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.156+0000 7ff32aa00640 1 -- start start 2024-03-21T09:11:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.156+0000 7ff32aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 0x7ff324199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff32aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3241048a0 0x7ff324199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff32aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3241071d0 0x7ff3241a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff32aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32410d180 con 0x7ff3241048a0 2024-03-21T09:11:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff32aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32410d000 con 0x7ff3241071d0 2024-03-21T09:11:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff32aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32410d300 con 0x7ff324101f70 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 0x7ff324199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 0x7ff324199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47760/0 (socket says 172.21.15.37:47760) 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 -- 172.21.15.37:0/4278840159 learned_addr learned my addr 172.21.15.37:0/4278840159 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff32a000640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3241071d0 0x7ff3241a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff328c00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3241048a0 0x7ff324199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 -- 172.21.15.37:0/4278840159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3241071d0 msgr2=0x7ff3241a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3241071d0 0x7ff3241a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 -- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3241048a0 msgr2=0x7ff324199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3241048a0 0x7ff324199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.157+0000 7ff329600640 1 -- 172.21.15.37:0/4278840159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3241a1650 con 0x7ff324101f70 2024-03-21T09:11:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.158+0000 7ff329600640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 0x7ff324199a30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff318009230 tx=0x7ff31802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.158+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318041070 con 0x7ff324101f70 2024-03-21T09:11:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.158+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff318004450 con 0x7ff324101f70 2024-03-21T09:11:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.158+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3241a18e0 con 0x7ff324101f70 2024-03-21T09:11:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.158+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318003e20 con 0x7ff324101f70 2024-03-21T09:11:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.158+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3241a1dd0 con 0x7ff324101f70 2024-03-21T09:11:38.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.159+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff324108d20 con 0x7ff324101f70 2024-03-21T09:11:38.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.161+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff318044070 con 0x7ff324101f70 2024-03-21T09:11:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.162+0000 7ff322000640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff30c0a2f20 0x7ff30c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.163+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3180f1370 con 0x7ff324101f70 2024-03-21T09:11:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.163+0000 7ff328c00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff30c0a2f20 0x7ff30c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.164+0000 7ff328c00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff30c0a2f20 0x7ff30c0a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff32419af40 tx=0x7ff310005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.166+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff3180bd760 con 0x7ff324101f70 2024-03-21T09:11:38.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3982229654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2312516933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:11:38.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:11:38.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:37 smithi037 ceph-mon[31830]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:38.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3982229654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2312516933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:11:38.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:11:38.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:37 smithi053 ceph-mon[36169]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:38.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3982229654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2312516933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:11:38.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:11:38.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:37 smithi071 ceph-mon[36597]: from='mgr.14152 172.21.15.37:0/1080297340' entity='mgr.x' 2024-03-21T09:11:38.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.369+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3240622a0 con 0x7ff324101f70 2024-03-21T09:11:38.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.370+0000 7ff322000640 1 -- 172.21.15.37:0/4278840159 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3180c1800 con 0x7ff324101f70 2024-03-21T09:11:38.365 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff30c0a2f20 msgr2=0x7ff30c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff30c0a2f20 0x7ff30c0a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff32419af40 tx=0x7ff310005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 msgr2=0x7ff324199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 0x7ff324199a30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff318009230 tx=0x7ff31802c650 comp rx=0 tx=0).stop 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 shutdown_connections 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff30c0a2f20 0x7ff30c0a53e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3241071d0 0x7ff3241a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3241048a0 0x7ff324199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 --2- 172.21.15.37:0/4278840159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff324101f70 0x7ff324199a30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 >> 172.21.15.37:0/4278840159 conn(0x7ff3240fb980 msgr2=0x7ff324101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 shutdown_connections 2024-03-21T09:11:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.373+0000 7ff32aa00640 1 -- 172.21.15.37:0/4278840159 wait complete. 2024-03-21T09:11:38.394 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 -- 172.21.15.37:0/485674865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c104940 msgr2=0x7f9d0c106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 --2- 172.21.15.37:0/485674865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c104940 0x7f9d0c106d30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009860 tx=0x7f9cf802bb70 comp rx=0 tx=0).stop 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 -- 172.21.15.37:0/485674865 shutdown_connections 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 --2- 172.21.15.37:0/485674865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d0c107270 0x7f9d0c109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 --2- 172.21.15.37:0/485674865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c104940 0x7f9d0c106d30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 --2- 172.21.15.37:0/485674865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c102010 0x7f9d0c104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 -- 172.21.15.37:0/485674865 >> 172.21.15.37:0/485674865 conn(0x7f9d0c0fb980 msgr2=0x7f9d0c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 -- 172.21.15.37:0/485674865 shutdown_connections 2024-03-21T09:11:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.627+0000 7f9d13200640 1 -- 172.21.15.37:0/485674865 wait complete. 2024-03-21T09:11:38.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.628+0000 7f9d13200640 1 Processor -- start 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.628+0000 7f9d13200640 1 -- start start 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.628+0000 7f9d13200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 0x7f9d0c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.628+0000 7f9d13200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 0x7f9d0c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d13200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d0c107270 0x7f9d0c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d13200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c10d160 con 0x7f9d0c107270 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d13200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c10cfe0 con 0x7f9d0c102010 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d13200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c10d2e0 con 0x7f9d0c104940 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 0x7f9d0c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 0x7f9d0c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 0x7f9d0c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47774/0 (socket says 172.21.15.37:47774) 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 0x7f9d0c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54608/0 (socket says 172.21.15.37:54608) 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 -- 172.21.15.37:0/632006694 learned_addr learned my addr 172.21.15.37:0/632006694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d12800640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d0c107270 0x7f9d0c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 -- 172.21.15.37:0/632006694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 msgr2=0x7f9d0c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 0x7f9d0c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 -- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d0c107270 msgr2=0x7f9d0c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d0c107270 0x7f9d0c1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11400640 1 -- 172.21.15.37:0/632006694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c1a1630 con 0x7f9d0c104940 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.629+0000 7f9d11e00640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 0x7f9d0c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.630+0000 7f9d11400640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 0x7f9d0c199f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80349e0 tx=0x7f9cf80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.630+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8041020 con 0x7f9d0c104940 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.630+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cf80323b0 con 0x7f9d0c104940 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.630+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8039040 con 0x7f9d0c104940 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.630+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c1a18c0 con 0x7f9d0c104940 2024-03-21T09:11:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.630+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c1a1d60 con 0x7f9d0c104940 2024-03-21T09:11:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.631+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c108d60 con 0x7f9d0c104940 2024-03-21T09:11:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.633+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9cf8044020 con 0x7f9d0c104940 2024-03-21T09:11:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.634+0000 7f9d06a00640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f9cf40a2fa0 0x7f9cf40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.634+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9cf80f0ff0 con 0x7f9d0c104940 2024-03-21T09:11:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.634+0000 7f9d11e00640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f9cf40a2fa0 0x7f9cf40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.635+0000 7f9d11e00640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f9cf40a2fa0 0x7f9cf40a5460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9d000096c0 tx=0x7f9d00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.638+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9cf804b020 con 0x7f9d0c104940 2024-03-21T09:11:38.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.853+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d0c0622a0 con 0x7f9d0c104940 2024-03-21T09:11:38.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.853+0000 7f9d06a00640 1 -- 172.21.15.37:0/632006694 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cf80bda50 con 0x7f9d0c104940 2024-03-21T09:11:38.848 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.856+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f9cf40a2fa0 msgr2=0x7f9cf40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.856+0000 7f9d13200640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f9cf40a2fa0 0x7f9cf40a5460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9d000096c0 tx=0x7f9d00009260 comp rx=0 tx=0).stop 2024-03-21T09:11:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.856+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 msgr2=0x7f9d0c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.856+0000 7f9d13200640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 0x7f9d0c199f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80349e0 tx=0x7f9cf80042f0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 shutdown_connections 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f9cf40a2fa0 0x7f9cf40a5460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d0c107270 0x7f9d0c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d0c104940 0x7f9d0c199f50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 --2- 172.21.15.37:0/632006694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0c102010 0x7f9d0c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 >> 172.21.15.37:0/632006694 conn(0x7f9d0c0fb980 msgr2=0x7f9d0c1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 shutdown_connections 2024-03-21T09:11:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:38.857+0000 7f9d13200640 1 -- 172.21.15.37:0/632006694 wait complete. 2024-03-21T09:11:38.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 -- 172.21.15.37:0/2560902309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1030b0 msgr2=0x7efc7c109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 --2- 172.21.15.37:0/2560902309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1030b0 0x7efc7c109940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efc74009be0 tx=0x7efc7402bef0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 -- 172.21.15.37:0/2560902309 shutdown_connections 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 --2- 172.21.15.37:0/2560902309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1030b0 0x7efc7c109940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 --2- 172.21.15.37:0/2560902309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1026f0 0x7efc7c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 --2- 172.21.15.37:0/2560902309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 -- 172.21.15.37:0/2560902309 >> 172.21.15.37:0/2560902309 conn(0x7efc7c0fcab0 msgr2=0x7efc7c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.103+0000 7efc82a00640 1 -- 172.21.15.37:0/2560902309 shutdown_connections 2024-03-21T09:11:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.104+0000 7efc82a00640 1 -- 172.21.15.37:0/2560902309 wait complete. 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.104+0000 7efc82a00640 1 Processor -- start 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 -- start start 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1026f0 0x7efc7c197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1030b0 0x7efc7c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c10d1b0 con 0x7efc7c1030b0 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c10d030 con 0x7efc7c101500 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c10d330 con 0x7efc7c1026f0 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc81600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc81600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54628/0 (socket says 172.21.15.37:54628) 2024-03-21T09:11:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc81600640 1 -- 172.21.15.37:0/3749283808 learned_addr learned my addr 172.21.15.37:0/3749283808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc80c00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1026f0 0x7efc7c197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.105+0000 7efc82000640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1030b0 0x7efc7c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc81600640 1 -- 172.21.15.37:0/3749283808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1026f0 msgr2=0x7efc7c197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc81600640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1026f0 0x7efc7c197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc81600640 1 -- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1030b0 msgr2=0x7efc7c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc81600640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1030b0 0x7efc7c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc81600640 1 -- 172.21.15.37:0/3749283808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c19f3b0 con 0x7efc7c101500 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc82000640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1030b0 0x7efc7c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc80c00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1026f0 0x7efc7c197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc81600640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c197790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e9a0 tx=0x7efc7000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000cd80 con 0x7efc7c101500 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc70010910 con 0x7efc7c101500 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c19f6a0 con 0x7efc7c101500 2024-03-21T09:11:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.106+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70018a30 con 0x7efc7c101500 2024-03-21T09:11:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.107+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c19fbf0 con 0x7efc7c101500 2024-03-21T09:11:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.108+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c108750 con 0x7efc7c101500 2024-03-21T09:11:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.111+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efc70010ac0 con 0x7efc7c101500 2024-03-21T09:11:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.113+0000 7efc7a000640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efc640a2f20 0x7efc640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.113+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efc700cfb80 con 0x7efc7c101500 2024-03-21T09:11:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.113+0000 7efc80c00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efc640a2f20 0x7efc640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.114+0000 7efc80c00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efc640a2f20 0x7efc640a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efc7c198ca0 tx=0x7efc68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.117+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efc70053740 con 0x7efc7c101500 2024-03-21T09:11:39.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:38 smithi037 ceph-mon[31830]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:39.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/734413098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4278840159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/632006694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:38 smithi053 ceph-mon[36169]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:39.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/734413098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4278840159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/632006694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.322+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efc7c0622a0 con 0x7efc7c101500 2024-03-21T09:11:39.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.323+0000 7efc7a000640 1 -- 172.21.15.37:0/3749283808 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7efc7009c620 con 0x7efc7c101500 2024-03-21T09:11:39.318 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efc640a2f20 msgr2=0x7efc640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efc640a2f20 0x7efc640a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efc7c198ca0 tx=0x7efc68005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 msgr2=0x7efc7c197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c197790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e9a0 tx=0x7efc7000ee60 comp rx=0 tx=0).stop 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 shutdown_connections 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7efc640a2f20 0x7efc640a53e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc7c1030b0 0x7efc7c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc7c1026f0 0x7efc7c197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 --2- 172.21.15.37:0/3749283808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c101500 0x7efc7c197790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 >> 172.21.15.37:0/3749283808 conn(0x7efc7c0fcab0 msgr2=0x7efc7c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:39.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 shutdown_connections 2024-03-21T09:11:39.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.327+0000 7efc82a00640 1 -- 172.21.15.37:0/3749283808 wait complete. 2024-03-21T09:11:39.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:39.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:38 smithi071 ceph-mon[36597]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:39.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/734413098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4278840159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/632006694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 -- 172.21.15.37:0/830460445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa01012a0 msgr2=0x7f5aa0101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 --2- 172.21.15.37:0/830460445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa01012a0 0x7f5aa0101720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009900 tx=0x7f5a9802bc50 comp rx=0 tx=0).stop 2024-03-21T09:11:39.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 -- 172.21.15.37:0/830460445 shutdown_connections 2024-03-21T09:11:39.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 --2- 172.21.15.37:0/830460445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa01012a0 0x7f5aa0101720 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 --2- 172.21.15.37:0/830460445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa0100900 0x7f5aa0100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 --2- 172.21.15.37:0/830460445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5aa00fffa0 0x7f5aa01003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 -- 172.21.15.37:0/830460445 >> 172.21.15.37:0/830460445 conn(0x7f5aa00fbb40 msgr2=0x7f5aa00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 -- 172.21.15.37:0/830460445 shutdown_connections 2024-03-21T09:11:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.565+0000 7f5aa6800640 1 -- 172.21.15.37:0/830460445 wait complete. 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.566+0000 7f5aa6800640 1 Processor -- start 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.566+0000 7f5aa6800640 1 -- start start 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5aa6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5aa00fffa0 0x7f5aa0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5aa6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 0x7f5aa019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5aa6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa01012a0 0x7f5aa01a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5aa6800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa010f520 con 0x7f5aa00fffa0 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5aa6800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa010f3a0 con 0x7f5aa0100900 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5aa6800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa010f6a0 con 0x7f5aa01012a0 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5a96a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 0x7f5aa019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5a9f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5aa00fffa0 0x7f5aa0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5a9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa01012a0 0x7f5aa01a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5a96a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 0x7f5aa019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54660/0 (socket says 172.21.15.37:54660) 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.567+0000 7f5a96a00640 1 -- 172.21.15.37:0/2581900466 learned_addr learned my addr 172.21.15.37:0/2581900466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9fe00640 1 -- 172.21.15.37:0/2581900466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 msgr2=0x7f5aa019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9fe00640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 0x7f5aa019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9fe00640 1 -- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5aa00fffa0 msgr2=0x7f5aa0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9fe00640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5aa00fffa0 0x7f5aa0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9fe00640 1 -- 172.21.15.37:0/2581900466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa01a1800 con 0x7f5aa01012a0 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a96a00640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 0x7f5aa019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9fe00640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa01012a0 0x7f5aa01a1190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a9802c190 tx=0x7f5a98005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98042020 con 0x7f5aa01012a0 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a980044d0 con 0x7f5aa01012a0 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9803e460 con 0x7f5aa01012a0 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa01a1a90 con 0x7f5aa01012a0 2024-03-21T09:11:39.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.568+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01a1fb0 con 0x7f5aa01012a0 2024-03-21T09:11:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.569+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa010ae40 con 0x7f5aa01012a0 2024-03-21T09:11:39.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.571+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5a98045020 con 0x7f5aa01012a0 2024-03-21T09:11:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.572+0000 7f5a9cc00640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5a840a2f20 0x7f5a840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.573+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5a980f17c0 con 0x7f5aa01012a0 2024-03-21T09:11:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.573+0000 7f5a9f400640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5a840a2f20 0x7f5a840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:39.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.573+0000 7f5a9f400640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5a840a2f20 0x7f5a840a53e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a90005fd0 tx=0x7f5a90009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:39.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.576+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a9804c020 con 0x7f5aa01012a0 2024-03-21T09:11:39.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.821+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aa00622a0 con 0x7f5aa01012a0 2024-03-21T09:11:39.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.822+0000 7f5a9cc00640 1 -- 172.21.15.37:0/2581900466 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f5a980be200 con 0x7f5aa01012a0 2024-03-21T09:11:39.817 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:39.817 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:09:20.795215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:11:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-21T09:11:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5a840a2f20 msgr2=0x7f5a840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5a840a2f20 0x7f5a840a53e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a90005fd0 tx=0x7f5a90009680 comp rx=0 tx=0).stop 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa01012a0 msgr2=0x7f5aa01a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa01012a0 0x7f5aa01a1190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a9802c190 tx=0x7f5a98005d50 comp rx=0 tx=0).stop 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 shutdown_connections 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5a840a2f20 0x7f5a840a53e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa01012a0 0x7f5aa01a1190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa0100900 0x7f5aa019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 --2- 172.21.15.37:0/2581900466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5aa00fffa0 0x7f5aa0199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 >> 172.21.15.37:0/2581900466 conn(0x7f5aa00fbb40 msgr2=0x7f5aa00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.825+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 shutdown_connections 2024-03-21T09:11:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:39.826+0000 7f5aa6800640 1 -- 172.21.15.37:0/2581900466 wait complete. 2024-03-21T09:11:39.842 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 -- 172.21.15.37:0/3881569726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281004b0 msgr2=0x7fe1281008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 --2- 172.21.15.37:0/3881569726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281004b0 0x7fe1281008d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe118009900 tx=0x7fe11802bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 -- 172.21.15.37:0/3881569726 shutdown_connections 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 --2- 172.21.15.37:0/3881569726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 0x7fe1281088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 --2- 172.21.15.37:0/3881569726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281016a0 0x7fe128101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 --2- 172.21.15.37:0/3881569726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281004b0 0x7fe1281008d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.063+0000 7fe12da00640 1 -- 172.21.15.37:0/3881569726 >> 172.21.15.37:0/3881569726 conn(0x7fe1280fba40 msgr2=0x7fe1280fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.064+0000 7fe12da00640 1 -- 172.21.15.37:0/3881569726 shutdown_connections 2024-03-21T09:11:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.064+0000 7fe12da00640 1 -- 172.21.15.37:0/3881569726 wait complete. 2024-03-21T09:11:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.065+0000 7fe12da00640 1 Processor -- start 2024-03-21T09:11:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 -- start start 2024-03-21T09:11:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 0x7fe128196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281016a0 0x7fe128196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 0x7fe12819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12810c190 con 0x7fe128102060 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12810c010 con 0x7fe1281004b0 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.066+0000 7fe12da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12810c310 con 0x7fe1281016a0 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 0x7fe128196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 0x7fe128196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54674/0 (socket says 172.21.15.37:54674) 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 -- 172.21.15.37:0/4134993080 learned_addr learned my addr 172.21.15.37:0/4134993080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127400640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281016a0 0x7fe128196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 -- 172.21.15.37:0/4134993080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281016a0 msgr2=0x7fe128196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281016a0 0x7fe128196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe12d000640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 0x7fe12819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 -- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 msgr2=0x7fe12819dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 0x7fe12819dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 -- 172.21.15.37:0/4134993080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12819e450 con 0x7fe1281004b0 2024-03-21T09:11:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe12d000640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 0x7fe12819dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:11:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.067+0000 7fe127e00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 0x7fe128196830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe11802efd0 tx=0x7fe118005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.068+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118041070 con 0x7fe1281004b0 2024-03-21T09:11:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.068+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1180044d0 con 0x7fe1281004b0 2024-03-21T09:11:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.068+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12819e6e0 con 0x7fe1281004b0 2024-03-21T09:11:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.068+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118034440 con 0x7fe1281004b0 2024-03-21T09:11:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.068+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12819ebf0 con 0x7fe1281004b0 2024-03-21T09:11:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.069+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1281077d0 con 0x7fe1281004b0 2024-03-21T09:11:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.071+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe118044070 con 0x7fe1281004b0 2024-03-21T09:11:40.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.072+0000 7fe124c00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe10c0a2f90 0x7fe10c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.072+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1180f1370 con 0x7fe1281004b0 2024-03-21T09:11:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.072+0000 7fe127400640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe10c0a2f90 0x7fe10c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.073+0000 7fe127400640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe10c0a2f90 0x7fe10c0a5450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe128197d40 tx=0x7fe114005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.076+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe1180bddb0 con 0x7fe1281004b0 2024-03-21T09:11:40.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:39 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3749283808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:40.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:39 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2581900466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:40.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:39 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3749283808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:40.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:39 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2581900466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:40.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.300+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1280622a0 con 0x7fe1281004b0 2024-03-21T09:11:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.302+0000 7fe124c00640 1 -- 172.21.15.37:0/4134993080 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7fe1180a80c0 con 0x7fe1281004b0 2024-03-21T09:11:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:40.297 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.305+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe10c0a2f90 msgr2=0x7fe10c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.305+0000 7fe12da00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe10c0a2f90 0x7fe10c0a5450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe128197d40 tx=0x7fe114005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.305+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 msgr2=0x7fe128196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.305+0000 7fe12da00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 0x7fe128196830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe11802efd0 tx=0x7fe118005d50 comp rx=0 tx=0).stop 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 shutdown_connections 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fe10c0a2f90 0x7fe10c0a5450 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe128102060 0x7fe12819dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1281016a0 0x7fe128196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 --2- 172.21.15.37:0/4134993080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1281004b0 0x7fe128196830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 >> 172.21.15.37:0/4134993080 conn(0x7fe1280fba40 msgr2=0x7fe1281049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 shutdown_connections 2024-03-21T09:11:40.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.306+0000 7fe12da00640 1 -- 172.21.15.37:0/4134993080 wait complete. 2024-03-21T09:11:40.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T09:08:46.306108+0000', 'modified': '2024-03-21T09:09:20.795215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24589, 'mds_1': 24575, 'mds_2': 24553}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6837/3665426355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3665426355}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3665426355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6835/2027751164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 2027751164}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2027751164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6837/2290408711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 2290408711}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 2290408711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.71:6839/543433986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 543433986}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 543433986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.53:6839/380096256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 380096256}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 380096256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24589': {'gid': 24589, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6841/600888190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 600888190}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 600888190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24589, 'qdb_cluster': [24553, 24575, 24589]} 2024-03-21T09:11:40.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-21T09:11:40.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-21T09:11:40.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:39 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3749283808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:40.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:39 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2581900466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 -- 172.21.15.37:0/3166356813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 msgr2=0x7f461010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 --2- 172.21.15.37:0/3166356813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 0x7f461010bb40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4608009900 tx=0x7f460802bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 -- 172.21.15.37:0/3166356813 shutdown_connections 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 --2- 172.21.15.37:0/3166356813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 0x7f461010bb40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 --2- 172.21.15.37:0/3166356813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 0x7f4610104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 --2- 172.21.15.37:0/3166356813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 -- 172.21.15.37:0/3166356813 >> 172.21.15.37:0/3166356813 conn(0x7f46100fec70 msgr2=0x7f46101010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 -- 172.21.15.37:0/3166356813 shutdown_connections 2024-03-21T09:11:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.556+0000 7f460f200640 1 -- 172.21.15.37:0/3166356813 wait complete. 2024-03-21T09:11:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 Processor -- start 2024-03-21T09:11:40.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 -- start start 2024-03-21T09:11:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 0x7f4610199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 0x7f46101a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461010f3d0 con 0x7f4610103700 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461010f250 con 0x7f46101052b0 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461010f550 con 0x7f46101048f0 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46358/0 (socket says 172.21.15.37:46358) 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460de00640 1 -- 172.21.15.37:0/3349312204 learned_addr learned my addr 172.21.15.37:0/3349312204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f4605400640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 0x7f4610199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.559+0000 7f460e800640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 0x7f46101a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f460de00640 1 -- 172.21.15.37:0/3349312204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 msgr2=0x7f4610199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f460de00640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 0x7f4610199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f460de00640 1 -- 172.21.15.37:0/3349312204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 msgr2=0x7f46101a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f460de00640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 0x7f46101a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f460de00640 1 -- 172.21.15.37:0/3349312204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46101a15a0 con 0x7f4610103700 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f4605400640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 0x7f4610199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f460de00640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610199980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f460000ebc0 tx=0x7f460000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f4607400640 1 -- 172.21.15.37:0/3349312204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003b70 con 0x7f4610103700 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f4607400640 1 -- 172.21.15.37:0/3349312204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4600003d10 con 0x7f4610103700 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.560+0000 7f4607400640 1 -- 172.21.15.37:0/3349312204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600019980 con 0x7f4610103700 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.561+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46101a1890 con 0x7f4610103700 2024-03-21T09:11:40.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.561+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46101a1d90 con 0x7f4610103700 2024-03-21T09:11:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 Processor -- start 2024-03-21T09:11:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4607400640 1 -- 172.21.15.37:0/3349312204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4600019b20 con 0x7f4610103700 2024-03-21T09:11:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 -- start start 2024-03-21T09:11:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b0421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b047270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b049d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8afa6c90 con 0x55ae8b0478e0 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.564+0000 7f4617c2d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8af4f980 con 0x55ae8b041d80 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f4617c2d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8afa2d30 con 0x55ae8b042770 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b047270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b0421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b0421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54696/0 (socket says 172.21.15.37:54696) 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 -- 172.21.15.37:0/2195969249 learned_addr learned my addr 172.21.15.37:0/2195969249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f4605400640 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b049d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 -- 172.21.15.37:0/2195969249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 msgr2=0x55ae8b047270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b047270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 -- 172.21.15.37:0/2195969249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 msgr2=0x55ae8b049d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b049d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.565+0000 7f460e800640 1 -- 172.21.15.37:0/2195969249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8af67510 con 0x55ae8b041d80 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f4607400640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f45f40a2f20 0x7f45f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f460e800640 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b0421a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f460802c8e0 tx=0x7f460802c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cd8badcfe62d6e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f4607400640 1 -- 172.21.15.37:0/3349312204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f46000cff80 con 0x7f4610103700 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f45ff400640 1 -- 172.21.15.37:0/2195969249 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460802c980 con 0x55ae8b041d80 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f45ff400640 1 -- 172.21.15.37:0/2195969249 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f460802cb20 con 0x55ae8b041d80 2024-03-21T09:11:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f45ff400640 1 -- 172.21.15.37:0/2195969249 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608004920 con 0x55ae8b041d80 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.566+0000 7f460de00640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f45f40a2f20 0x7f45f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 -- 172.21.15.37:0/2195969249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 msgr2=0x55ae8b0421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f460de00640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f45f40a2f20 0x7f45f40a53e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46000cda60 tx=0x7f46000d3a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b0421a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f460802c8e0 tx=0x7f460802c680 comp rx=0 tx=0).stop 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 -- 172.21.15.37:0/2195969249 shutdown_connections 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b049d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b047270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 --2- 172.21.15.37:0/2195969249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b0421a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 -- 172.21.15.37:0/2195969249 >> 172.21.15.37:0/2195969249 conn(0x55ae8b03d2f0 msgr2=0x55ae8b03d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 -- 172.21.15.37:0/2195969249 shutdown_connections 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 -- 172.21.15.37:0/2195969249 wait complete. 2024-03-21T09:11:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.567+0000 7f4617c2d740 1 -- start start 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 Processor -- start 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b04c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b053790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b04a830 con 0x55ae8b0478e0 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8af62dd0 con 0x55ae8b041d80 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b04a9b0 con 0x55ae8b042770 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.568+0000 7f4617c2d740 10 client.0 _ll_register_callbacks cb 0x55ae8acbaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b04c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b04c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47858/0 (socket says 172.21.15.37:47858) 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f460e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 -- 172.21.15.37:0/3027564342 learned_addr learned my addr 172.21.15.37:0/3027564342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f460e800640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b053790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 -- 172.21.15.37:0/3027564342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 msgr2=0x55ae8b04c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 -- 172.21.15.37:0/3027564342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 msgr2=0x55ae8b053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b053790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 -- 172.21.15.37:0/3027564342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8acbb2d0 con 0x55ae8b042770 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f460e800640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b053790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f4605400640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b04c750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45f8012d20 tx=0x7f45f800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545a87beb5a35756 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.569+0000 7f45f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f45f3e00640 1 -- 172.21.15.37:0/3027564342 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80120d0 con 0x55ae8b042770 2024-03-21T09:11:40.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f45f3e00640 1 -- 172.21.15.37:0/3027564342 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f45f8010430 con 0x55ae8b042770 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f460802cb20 con 0x55ae8b042770 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f45f3e00640 1 -- 172.21.15.37:0/3027564342 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f8012270 con 0x55ae8b042770 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f4617c2d740 10 client.25336 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ae8ad856b0 con 0x55ae8b042770 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f45f3e00640 1 -- 172.21.15.37:0/3027564342 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f460802cb20 con 0x55ae8b042770 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f45f3e00640 1 -- 172.21.15.37:0/3027564342 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f45f801c520 con 0x55ae8b042770 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.570+0000 7f4617c2d740 10 client.25336 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f4617c2d740 10 client.25336 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f4617c2d740 20 client.25336 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f4617c2d740 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b058aa0 0x55ae8b057b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f4617c2d740 4 client.25336 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae8af48520 con 0x55ae8b058aa0 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f460e800640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b058aa0 0x55ae8b057b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f460e800640 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b058aa0 0x55ae8b057b30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=378e61556af79f80 server_cookie=37f39c1dd54f375d in_seq=0 out_seq=0 2024-03-21T09:11:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.571+0000 7f45f3e00640 10 client.25336 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f45f3e00640 1 -- 172.21.15.37:0/3027564342 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae8af48520 con 0x55ae8b058aa0 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f45f3e00640 10 client.25336 handle_command_reply: tid=0 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 client.25336 shutdown 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 msgr2=0x55ae8b04c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b04c750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45f8012d20 tx=0x7f45f800f790 comp rx=0 tx=0).stop 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 shutdown_connections 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b058aa0 0x55ae8b057b30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b053790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b042770 0x55ae8b04c750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f45f3e00640 0 client.25336 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 --2- 172.21.15.37:0/3027564342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.572+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 >> 172.21.15.37:0/3027564342 conn(0x55ae8b03d2f0 msgr2=0x55ae8b03c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.573+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 shutdown_connections 2024-03-21T09:11:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.573+0000 7f4617c2d740 1 -- 172.21.15.37:0/3027564342 wait complete. 2024-03-21T09:11:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.573+0000 7f4617c2d740 20 client.25336 trim_cache size 0 max 16384 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 Processor -- start 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 -- start start 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b041d80 0x55ae8b043210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 0x55ae8b053750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b057b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b0582c0 con 0x55ae8b0478e0 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b058920 con 0x55ae8b042770 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.594+0000 7f4617c2d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b058f80 con 0x55ae8b041d80 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 0x55ae8b053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 0x55ae8b053750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54712/0 (socket says 172.21.15.37:54712) 2024-03-21T09:11:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 -- 172.21.15.37:0/1126632752 learned_addr learned my addr 172.21.15.37:0/1126632752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f460e800640 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b041d80 0x55ae8b043210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 -- 172.21.15.37:0/1126632752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b041d80 msgr2=0x55ae8b043210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b041d80 0x55ae8b043210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 -- 172.21.15.37:0/1126632752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 msgr2=0x55ae8b057b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f460e800640 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b057b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b057b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 -- 172.21.15.37:0/1126632752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8b043000 con 0x55ae8b042770 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f460e800640 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b057b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:11:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.595+0000 7f4605400640 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 0x55ae8b053750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f45f801ca60 tx=0x7f45f8005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f813e3dbe37207c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f45fea00640 1 -- 172.21.15.37:0/1126632752 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009ea0 con 0x55ae8b042770 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f45fea00640 1 -- 172.21.15.37:0/1126632752 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45f801cbd0 con 0x55ae8b042770 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f45fea00640 1 -- 172.21.15.37:0/1126632752 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8003850 con 0x55ae8b042770 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 -- 172.21.15.37:0/1126632752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 msgr2=0x55ae8b053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 0x55ae8b053750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f45f801ca60 tx=0x7f45f8005a70 comp rx=0 tx=0).stop 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 -- 172.21.15.37:0/1126632752 shutdown_connections 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b0478e0 0x55ae8b057b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b042770 0x55ae8b053750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1126632752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b041d80 0x55ae8b043210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.596+0000 7f4617c2d740 1 -- 172.21.15.37:0/1126632752 >> 172.21.15.37:0/1126632752 conn(0x55ae8b03d2f0 msgr2=0x55ae8b04ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.597+0000 7f4617c2d740 1 -- 172.21.15.37:0/1126632752 shutdown_connections 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.597+0000 7f4617c2d740 1 -- 172.21.15.37:0/1126632752 wait complete. 2024-03-21T09:11:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.597+0000 7f4617c2d740 1 -- start start 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.597+0000 7f4617c2d740 1 Processor -- start 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b042770 0x55ae8b04fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b0478e0 0x55ae8b052a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b059c40 con 0x55ae8b042770 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b0595e0 con 0x55ae8b041d80 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8b05a2a0 con 0x55ae8b0478e0 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4617c2d740 10 client.0 _ll_register_callbacks cb 0x7f45f8009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54724/0 (socket says 172.21.15.37:54724) 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 -- 172.21.15.37:0/1749837636 learned_addr learned my addr 172.21.15.37:0/1749837636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4605400640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b042770 0x55ae8b04fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b0478e0 0x55ae8b052a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 -- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b0478e0 msgr2=0x55ae8b052a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b0478e0 0x55ae8b052a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 -- 172.21.15.37:0/1749837636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b042770 msgr2=0x55ae8b04fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b042770 0x55ae8b04fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f460e800640 1 -- 172.21.15.37:0/1749837636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8b052ff0 con 0x55ae8b041d80 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.598+0000 7f4605400640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b042770 0x55ae8b04fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f460e800640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04f8e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4608000c00 tx=0x7f4608042040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804c1e650990c757 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f45f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f45f3e00640 1 -- 172.21.15.37:0/1749837636 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080040c0 con 0x55ae8b041d80 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f45f3e00640 1 -- 172.21.15.37:0/1749837636 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4608004260 con 0x55ae8b041d80 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45f801cbd0 con 0x55ae8b041d80 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f45f3e00640 1 -- 172.21.15.37:0/1749837636 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4608004460 con 0x55ae8b041d80 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f4617c2d740 10 client.25316 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.599+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ae8b0bc8c0 con 0x55ae8b041d80 2024-03-21T09:11:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f45f3e00640 1 -- 172.21.15.37:0/1749837636 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45f801cbd0 con 0x55ae8b041d80 2024-03-21T09:11:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f45f3e00640 1 -- 172.21.15.37:0/1749837636 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f46080038c0 con 0x55ae8b041d80 2024-03-21T09:11:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4617c2d740 10 client.25316 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4617c2d740 10 client.25316 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T09:11:40.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4617c2d740 20 client.25316 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b0c09c0 0x55ae8b0bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4617c2d740 4 client.25316 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae8afb41d0 con 0x55ae8b0c09c0 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4605400640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b0c09c0 0x55ae8b0bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f4605400640 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b0c09c0 0x55ae8b0bca90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=366484e8a9126172 server_cookie=7ed429323a6a8a70 in_seq=0 out_seq=0 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.600+0000 7f45f3e00640 10 client.25316 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f45f3e00640 1 -- 172.21.15.37:0/1749837636 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ae8afb41d0 con 0x55ae8b0c09c0 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f45f3e00640 10 client.25316 handle_command_reply: tid=0 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 client.25316 shutdown 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 msgr2=0x55ae8b04f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04f8e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4608000c00 tx=0x7f4608042040 comp rx=0 tx=0).stop 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 shutdown_connections 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ae8b0478e0 0x55ae8b052a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55ae8b0c09c0 0x55ae8b0bca90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae8b042770 0x55ae8b04fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 --2- 172.21.15.37:0/1749837636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8b041d80 0x55ae8b04f8e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 >> 172.21.15.37:0/1749837636 conn(0x55ae8b03d2f0 msgr2=0x55ae8b043210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f45f3e00640 0 client.25316 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 shutdown_connections 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 1 -- 172.21.15.37:0/1749837636 wait complete. 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.601+0000 7f4617c2d740 20 client.25316 trim_cache size 0 max 16384 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-03-21T09:11:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24553, 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 140.496348861, 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 94, 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 192.39018306599999 2024-03-21T09:11:40.601 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f45f40a2f20 msgr2=0x7f45f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f45f40a2f20 0x7f45f40a53e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46000cda60 tx=0x7f46000d3a60 comp rx=0 tx=0).stop 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 msgr2=0x7f4610199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610199980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f460000ebc0 tx=0x7f460000c6a0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 shutdown_connections 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f45f40a2f20 0x7f45f40a53e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46101052b0 0x7f46101a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46101048f0 0x7f4610199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 --2- 172.21.15.37:0/3349312204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610103700 0x7f4610199980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 >> 172.21.15.37:0/3349312204 conn(0x7f46100fec70 msgr2=0x7f4610107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 shutdown_connections 2024-03-21T09:11:40.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.602+0000 7f460f200640 1 -- 172.21.15.37:0/3349312204 wait complete. 2024-03-21T09:11:40.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T09:11:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 -- 172.21.15.37:0/2996704895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c105480 msgr2=0x7f4a0c10bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 --2- 172.21.15.37:0/2996704895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c105480 0x7f4a0c10bd10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009900 tx=0x7f4a0402bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:40.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 -- 172.21.15.37:0/2996704895 shutdown_connections 2024-03-21T09:11:40.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 --2- 172.21.15.37:0/2996704895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c105480 0x7f4a0c10bd10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 --2- 172.21.15.37:0/2996704895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c104ac0 0x7f4a0c104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 --2- 172.21.15.37:0/2996704895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 -- 172.21.15.37:0/2996704895 >> 172.21.15.37:0/2996704895 conn(0x7f4a0c0fee60 msgr2=0x7f4a0c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 -- 172.21.15.37:0/2996704895 shutdown_connections 2024-03-21T09:11:40.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.839+0000 7f4a10e00640 1 -- 172.21.15.37:0/2996704895 wait complete. 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.840+0000 7f4a10e00640 1 Processor -- start 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 -- start start 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c104ac0 0x7f4a0c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 0x7f4a0c1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c10f580 con 0x7f4a0c104ac0 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c10f400 con 0x7f4a0c105480 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a10e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c10f700 con 0x7f4a0c1038d0 2024-03-21T09:11:40.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a0a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 0x7f4a0c1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a0a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 0x7f4a0c1a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54752/0 (socket says 172.21.15.37:54752) 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a01400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c104ac0 0x7f4a0c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a0a800640 1 -- 172.21.15.37:0/426699793 learned_addr learned my addr 172.21.15.37:0/426699793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.841+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47906/0 (socket says 172.21.15.37:47906) 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a0a800640 1 -- 172.21.15.37:0/426699793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 msgr2=0x7f4a0c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a0a800640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a0a800640 1 -- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c104ac0 msgr2=0x7f4a0c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a0a800640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c104ac0 0x7f4a0c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a0a800640 1 -- 172.21.15.37:0/426699793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c1a1780 con 0x7f4a0c105480 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a09e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a0a800640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 0x7f4a0c1a1110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a0402c8e0 tx=0x7f4a0402c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a03400640 1 -- 172.21.15.37:0/426699793 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04043020 con 0x7f4a0c105480 2024-03-21T09:11:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a03400640 1 -- 172.21.15.37:0/426699793 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a040054e0 con 0x7f4a0c105480 2024-03-21T09:11:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c1a1a10 con 0x7f4a0c105480 2024-03-21T09:11:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.842+0000 7f4a03400640 1 -- 172.21.15.37:0/426699793 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04003d00 con 0x7f4a0c105480 2024-03-21T09:11:40.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.843+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c1a1f60 con 0x7f4a0c105480 2024-03-21T09:11:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.847+0000 7f4a13ba2740 1 Processor -- start 2024-03-21T09:11:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.847+0000 7f4a03400640 1 -- 172.21.15.37:0/426699793 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4a04046020 con 0x7f4a0c105480 2024-03-21T09:11:40.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.847+0000 7f4a13ba2740 1 -- start start 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebcefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebcf590 0x55e2bebd4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebd6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2beb33c90 con 0x55e2bebceba0 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2beadc980 con 0x55e2bebd4700 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2beb2fd30 con 0x55e2bebcf590 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a01400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebcefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a01400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebcefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46434/0 (socket says 172.21.15.37:46434) 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a01400640 1 -- 172.21.15.37:0/2213736382 learned_addr learned my addr 172.21.15.37:0/2213736382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a09e00640 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebcf590 0x55e2bebd4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.848+0000 7f4a01400640 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebd6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a09e00640 1 -- 172.21.15.37:0/2213736382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 msgr2=0x55e2bebd6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a09e00640 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebd6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a09e00640 1 -- 172.21.15.37:0/2213736382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 msgr2=0x55e2bebcefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a09e00640 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebcefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a09e00640 1 -- 172.21.15.37:0/2213736382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2beaf4510 con 0x55e2bebcf590 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a03400640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f00a2f20 0x7f49f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a09e00640 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebcf590 0x55e2bebd4090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c104920 tx=0x7f49fc011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34514aabb8b652af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a03400640 1 -- 172.21.15.37:0/426699793 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4a040f1820 con 0x7f4a0c105480 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f49fb400640 1 -- 172.21.15.37:0/2213736382 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003c30 con 0x55e2bebcf590 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f4a01400640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f00a2f20 0x7f49f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f49fb400640 1 -- 172.21.15.37:0/2213736382 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49fc00fe50 con 0x55e2bebcf590 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.849+0000 7f49fb400640 1 -- 172.21.15.37:0/2213736382 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0193e0 con 0x55e2bebcf590 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 -- 172.21.15.37:0/2213736382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebcf590 msgr2=0x55e2bebd4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a01400640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f00a2f20 0x7f49f00a53e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e2bebcfdf0 tx=0x7f49f40097a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebcf590 0x55e2bebd4090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c104920 tx=0x7f49fc011c70 comp rx=0 tx=0).stop 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 -- 172.21.15.37:0/2213736382 shutdown_connections 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebd6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebcf590 0x55e2bebd4090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/2213736382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebcefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 -- 172.21.15.37:0/2213736382 >> 172.21.15.37:0/2213736382 conn(0x55e2bebca110 msgr2=0x55e2bebca510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 -- 172.21.15.37:0/2213736382 shutdown_connections 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.850+0000 7f4a13ba2740 1 -- 172.21.15.37:0/2213736382 wait complete. 2024-03-21T09:11:40.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.851+0000 7f4a13ba2740 1 -- start start 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.851+0000 7f4a13ba2740 1 Processor -- start 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebe0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebd76b0 con 0x55e2bebcf590 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2beaefdd0 con 0x55e2bebd4700 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebd7830 con 0x55e2bebceba0 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a13ba2740 10 client.0 _ll_register_callbacks cb 0x55e2be847ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47924/0 (socket says 172.21.15.37:47924) 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a09e00640 1 -- 172.21.15.37:0/1427575224 learned_addr learned my addr 172.21.15.37:0/1427575224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a01400640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.852+0000 7f4a0a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebe0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a0a800640 1 -- 172.21.15.37:0/1427575224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 msgr2=0x55e2bebd90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a0a800640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a0a800640 1 -- 172.21.15.37:0/1427575224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 msgr2=0x55e2bebd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a0a800640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebd9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a0a800640 1 -- 172.21.15.37:0/1427575224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2be8482d0 con 0x55e2bebd4700 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a09e00640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f4a0a800640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebe0640 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a0404c030 tx=0x7f4a040bd7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b954f9060e00d907 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f49efe00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.853+0000 7f49efe00640 1 -- 172.21.15.37:0/1427575224 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04005690 con 0x55e2bebd4700 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f49efe00640 1 -- 172.21.15.37:0/1427575224 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4a040bde50 con 0x55e2bebd4700 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49fc00fe50 con 0x55e2bebd4700 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f49efe00640 1 -- 172.21.15.37:0/1427575224 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a040be2f0 con 0x55e2bebd4700 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 10 client.25328 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:40.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e2be9126b0 con 0x55e2bebd4700 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f49efe00640 1 -- 172.21.15.37:0/1427575224 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49fc00fe50 con 0x55e2bebd4700 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f49efe00640 1 -- 172.21.15.37:0/1427575224 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4a040c60e0 con 0x55e2bebd4700 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 10 client.25328 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 10 client.25328 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 20 client.25328 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bebe5920 0x55e2bebe47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 4 client.25328 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.854+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2bead5520 con 0x55e2bebe5920 2024-03-21T09:11:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.855+0000 7f4a09e00640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bebe5920 0x55e2bebe47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.855+0000 7f4a09e00640 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bebe5920 0x55e2bebe47a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=109c3eb8c1431f2b server_cookie=71f5320de97f5f2f in_seq=0 out_seq=0 2024-03-21T09:11:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.855+0000 7f49efe00640 10 client.25328 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f49efe00640 1 -- 172.21.15.37:0/1427575224 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e2bead5520 con 0x55e2bebe5920 2024-03-21T09:11:40.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f49efe00640 10 client.25328 handle_command_reply: tid=0 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 client.25328 shutdown 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 msgr2=0x55e2bebe0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebe0640 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a0404c030 tx=0x7f4a040bd7f0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 shutdown_connections 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bebe5920 0x55e2bebe47a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebe0640 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.856+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.857+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/1427575224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.857+0000 7f49efe00640 0 client.25328 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.857+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 >> 172.21.15.37:0/1427575224 conn(0x55e2bebca110 msgr2=0x55e2bebc9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.857+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 shutdown_connections 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.857+0000 7f4a13ba2740 1 -- 172.21.15.37:0/1427575224 wait complete. 2024-03-21T09:11:40.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.857+0000 7f4a13ba2740 20 client.25328 trim_cache size 0 max 16384 2024-03-21T09:11:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.894+0000 7f4a13ba2740 1 Processor -- start 2024-03-21T09:11:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.894+0000 7f4a13ba2740 1 -- start start 2024-03-21T09:11:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 0x55e2bebe0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebd4700 0x55e2bebd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebe5140 con 0x55e2bebceba0 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebe57a0 con 0x55e2bebcf590 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebe5e00 con 0x55e2bebd4700 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 0x55e2bebe0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 0x55e2bebe0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54764/0 (socket says 172.21.15.37:54764) 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 -- 172.21.15.37:0/3242039371 learned_addr learned my addr 172.21.15.37:0/3242039371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a0a800640 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebd4700 0x55e2bebd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 -- 172.21.15.37:0/3242039371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebd4700 msgr2=0x55e2bebd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebd4700 0x55e2bebd9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 -- 172.21.15.37:0/3242039371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 msgr2=0x55e2bebd90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.895+0000 7f4a01400640 1 -- 172.21.15.37:0/3242039371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2bebcf970 con 0x55e2bebcf590 2024-03-21T09:11:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.896+0000 7f4a01400640 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 0x55e2bebe0600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49f400fac0 tx=0x7f49f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be18f7adba1c595 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.896+0000 7f49faa00640 1 -- 172.21.15.37:0/3242039371 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40109f0 con 0x55e2bebcf590 2024-03-21T09:11:40.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.896+0000 7f49faa00640 1 -- 172.21.15.37:0/3242039371 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49f4010b90 con 0x55e2bebcf590 2024-03-21T09:11:40.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.896+0000 7f49faa00640 1 -- 172.21.15.37:0/3242039371 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f402b640 con 0x55e2bebcf590 2024-03-21T09:11:40.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 -- 172.21.15.37:0/3242039371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 msgr2=0x55e2bebe0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 0x55e2bebe0600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49f400fac0 tx=0x7f49f40046c0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 -- 172.21.15.37:0/3242039371 shutdown_connections 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebd4700 0x55e2bebd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebcf590 0x55e2bebe0600 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/3242039371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebceba0 0x55e2bebd90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 -- 172.21.15.37:0/3242039371 >> 172.21.15.37:0/3242039371 conn(0x55e2bebca110 msgr2=0x55e2bebca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 -- 172.21.15.37:0/3242039371 shutdown_connections 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.897+0000 7f4a13ba2740 1 -- 172.21.15.37:0/3242039371 wait complete. 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.898+0000 7f4a13ba2740 1 -- start start 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.898+0000 7f4a13ba2740 1 Processor -- start 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.898+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebdc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.898+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebdccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.898+0000 7f4a13ba2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebe6ac0 con 0x55e2bebcf590 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebe6460 con 0x55e2bebd4700 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a13ba2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2bebe7120 con 0x55e2bebceba0 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a13ba2740 10 client.0 _ll_register_callbacks cb 0x7f4a04005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a01400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebdccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebdc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebdc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47928/0 (socket says 172.21.15.37:47928) 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 -- 172.21.15.37:0/447181575 learned_addr learned my addr 172.21.15.37:0/447181575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a01400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebdccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46466/0 (socket says 172.21.15.37:46466) 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a0a800640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 -- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 msgr2=0x55e2bebdfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebdfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 -- 172.21.15.37:0/447181575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 msgr2=0x55e2bebdccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:40.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebdccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 -- 172.21.15.37:0/447181575 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2bebe0170 con 0x55e2bebceba0 2024-03-21T09:11:40.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a01400640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebdccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:40.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f4a09e00640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebdc790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49fc005c40 tx=0x7f49fc0092c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec581eb169695fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.899+0000 7f49efe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:11:40.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f49efe00640 1 -- 172.21.15.37:0/447181575 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc02f060 con 0x55e2bebceba0 2024-03-21T09:11:40.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f49efe00640 1 -- 172.21.15.37:0/447181575 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49fc004310 con 0x55e2bebceba0 2024-03-21T09:11:40.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49f4010b90 con 0x55e2bebceba0 2024-03-21T09:11:40.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f49efe00640 1 -- 172.21.15.37:0/447181575 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49fc036050 con 0x55e2bebceba0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f4a13ba2740 10 client.25360 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e2bec41330 con 0x55e2bebceba0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f49efe00640 1 -- 172.21.15.37:0/447181575 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49f4010b90 con 0x55e2bebceba0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f49efe00640 1 -- 172.21.15.37:0/447181575 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f49fc018070 con 0x55e2bebceba0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f4a13ba2740 10 client.25360 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f4a13ba2740 10 client.25360 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.900+0000 7f4a13ba2740 20 client.25360 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bec457d0 0x55e2bec41560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f4a13ba2740 4 client.25360 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e2beb411d0 con 0x55e2bec457d0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f4a01400640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bec457d0 0x55e2bec41560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f4a01400640 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bec457d0 0x55e2bec41560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1f62adb691a8047 server_cookie=266171f8757dc600 in_seq=0 out_seq=0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f49efe00640 10 client.25360 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f49efe00640 1 -- 172.21.15.37:0/447181575 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55e2beb411d0 con 0x55e2bec457d0 2024-03-21T09:11:40.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f49efe00640 10 client.25360 handle_command_reply: tid=0 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.901+0000 7f4a13ba2740 1 client.25360 shutdown 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 msgr2=0x55e2bebdc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebdc790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49fc005c40 tx=0x7f49fc0092c0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 shutdown_connections 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55e2bec457d0 0x55e2bec41560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2bebd4700 0x55e2bebdfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2bebcf590 0x55e2bebdccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 --2- 172.21.15.37:0/447181575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2bebceba0 0x55e2bebdc790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f49efe00640 0 client.25360 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 >> 172.21.15.37:0/447181575 conn(0x55e2bebca110 msgr2=0x55e2bebd90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 shutdown_connections 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 1 -- 172.21.15.37:0/447181575 wait complete. 2024-03-21T09:11:40.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.902+0000 7f4a13ba2740 20 client.25360 trim_cache size 0 max 16384 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24575, 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 141.81049575, 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 94, 2024-03-21T09:11:40.901 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-03-21T09:11:40.902 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 181.525173252 2024-03-21T09:11:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:11:40.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f00a2f20 msgr2=0x7f49f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f00a2f20 0x7f49f00a53e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e2bebcfdf0 tx=0x7f49f40097a0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 msgr2=0x7f4a0c1a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 0x7f4a0c1a1110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a0402c8e0 tx=0x7f4a0402c640 comp rx=0 tx=0).stop 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 shutdown_connections 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f49f00a2f20 0x7f49f00a53e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c105480 0x7f4a0c1a1110 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c104ac0 0x7f4a0c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 --2- 172.21.15.37:0/426699793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c1038d0 0x7f4a0c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 >> 172.21.15.37:0/426699793 conn(0x7f4a0c0fee60 msgr2=0x7f4a0c107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 shutdown_connections 2024-03-21T09:11:40.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:40.903+0000 7f4a10e00640 1 -- 172.21.15.37:0/426699793 wait complete. 2024-03-21T09:11:40.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-21T09:11:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 -- 172.21.15.37:0/3223499774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1014f0 msgr2=0x7f6cfc101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 --2- 172.21.15.37:0/3223499774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1014f0 0x7f6cfc101910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009900 tx=0x7f6cec02bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 -- 172.21.15.37:0/3223499774 shutdown_connections 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 --2- 172.21.15.37:0/3223499774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1030a0 0x7f6cfc109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 --2- 172.21.15.37:0/3223499774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 --2- 172.21.15.37:0/3223499774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1014f0 0x7f6cfc101910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.148+0000 7f6cfb200640 1 -- 172.21.15.37:0/3223499774 >> 172.21.15.37:0/3223499774 conn(0x7f6cfc0fca60 msgr2=0x7f6cfc0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.149+0000 7f6cfb200640 1 -- 172.21.15.37:0/3223499774 shutdown_connections 2024-03-21T09:11:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.149+0000 7f6cfb200640 1 -- 172.21.15.37:0/3223499774 wait complete. 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 Processor -- start 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 -- start start 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cfc1030a0 0x7f6cfc19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1a3530 0x7f6cfc1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc10d1d0 con 0x7f6cfc1030a0 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc10d050 con 0x7f6cfc1a3530 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfb200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc10d350 con 0x7f6cfc1026e0 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cfc1030a0 0x7f6cfc19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47950/0 (socket says 172.21.15.37:47950) 2024-03-21T09:11:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cf9e00640 1 -- 172.21.15.37:0/85146733 learned_addr learned my addr 172.21.15.37:0/85146733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.150+0000 7f6cfa800640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1a3530 0x7f6cfc1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf9e00640 1 -- 172.21.15.37:0/85146733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1a3530 msgr2=0x7f6cfc1a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf9e00640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1a3530 0x7f6cfc1a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf9e00640 1 -- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cfc1030a0 msgr2=0x7f6cfc19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf9e00640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cfc1030a0 0x7f6cfc19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf9e00640 1 -- 172.21.15.37:0/85146733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc1a5f90 con 0x7f6cfc1026e0 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf9e00640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc19df90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6cec036040 tx=0x7f6cec005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf2a00640 1 -- 172.21.15.37:0/85146733 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec041070 con 0x7f6cfc1026e0 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf2a00640 1 -- 172.21.15.37:0/85146733 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cec0054e0 con 0x7f6cfc1026e0 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.151+0000 7f6cf2a00640 1 -- 172.21.15.37:0/85146733 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec003e60 con 0x7f6cfc1026e0 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.152+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc1a6220 con 0x7f6cfc1026e0 2024-03-21T09:11:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.152+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc1a6690 con 0x7f6cfc1026e0 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6cf2a00640 1 -- 172.21.15.37:0/85146733 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6cec044070 con 0x7f6cfc1026e0 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 Processor -- start 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 -- start start 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfd91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdf3dc90 con 0x5560cdfde8e0 2024-03-21T09:11:41.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdee6980 con 0x5560cdfd8d80 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.155+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdf39d30 con 0x5560cdfd9770 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfd91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cf9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfd91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54798/0 (socket says 172.21.15.37:54798) 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 -- 172.21.15.37:0/1844221120 learned_addr learned my addr 172.21.15.37:0/1844221120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 -- 172.21.15.37:0/1844221120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 msgr2=0x5560cdfde270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfde270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 -- 172.21.15.37:0/1844221120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 msgr2=0x5560cdfe0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.156+0000 7f6cfa800640 1 -- 172.21.15.37:0/1844221120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560cdefe510 con 0x5560cdfd8d80 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6cfa800640 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfd91a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009fd0 tx=0x7f6cf40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57cce9ff3cfac304 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6cf2a00640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ce00a2fa0 0x7f6ce00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6ce7400640 1 -- 172.21.15.37:0/1844221120 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4004d20 con 0x5560cdfd8d80 2024-03-21T09:11:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6ce7400640 1 -- 172.21.15.37:0/1844221120 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cf4004050 con 0x5560cdfd8d80 2024-03-21T09:11:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6ce7400640 1 -- 172.21.15.37:0/1844221120 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4005260 con 0x5560cdfd8d80 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6cf2a00640 1 -- 172.21.15.37:0/85146733 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6cec0f1540 con 0x7f6cfc1026e0 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.157+0000 7f6cf9e00640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ce00a2fa0 0x7f6ce00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 -- 172.21.15.37:0/1844221120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 msgr2=0x5560cdfd91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfd91a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009fd0 tx=0x7f6cf40118d0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 -- 172.21.15.37:0/1844221120 shutdown_connections 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/1844221120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfd91a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 -- 172.21.15.37:0/1844221120 >> 172.21.15.37:0/1844221120 conn(0x5560cdfd42f0 msgr2=0x5560cdfd46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6cf9e00640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ce00a2fa0 0x7f6ce00a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0bd580 tx=0x7f6cec02bc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 -- 172.21.15.37:0/1844221120 shutdown_connections 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 -- 172.21.15.37:0/1844221120 wait complete. 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.158+0000 7f6d03fc1740 1 -- start start 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 Processor -- start 2024-03-21T09:11:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfe3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfd9770 0x5560cdfe3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfde8e0 0x5560cdfea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdfe1890 con 0x5560cdfd9770 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdef9dd0 con 0x5560cdfde8e0 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdfe1a10 con 0x5560cdfd8d80 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6d03fc1740 10 client.0 _ll_register_callbacks cb 0x5560cdc51ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfe3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfe3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47962/0 (socket says 172.21.15.37:47962) 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6cfa800640 1 -- 172.21.15.37:0/520561217 learned_addr learned my addr 172.21.15.37:0/520561217 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6cfa800640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfde8e0 0x5560cdfea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.159+0000 7f6cf9400640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfd9770 0x5560cdfe3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cfa800640 1 -- 172.21.15.37:0/520561217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfde8e0 msgr2=0x5560cdfea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cfa800640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfde8e0 0x5560cdfea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cfa800640 1 -- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfd9770 msgr2=0x5560cdfe3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cfa800640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfd9770 0x5560cdfe3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cfa800640 1 -- 172.21.15.37:0/520561217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560cdc522d0 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cfa800640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfe3320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009de0 tx=0x7f6cf4004890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a88187bd00d0d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cdbe00640 1 -- 172.21.15.37:0/520561217 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40046a0 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cf4004050 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.160+0000 7f6cdbe00640 1 -- 172.21.15.37:0/520561217 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6cf400f4e0 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6cdbe00640 1 -- 172.21.15.37:0/520561217 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cf4036050 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 10 client.25372 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5560cdd1c6b0 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6cdbe00640 1 -- 172.21.15.37:0/520561217 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cf4004050 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6cdbe00640 1 -- 172.21.15.37:0/520561217 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6cf4025040 con 0x5560cdfd8d80 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 10 client.25372 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 10 client.25372 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T09:11:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 20 client.25372 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560cdfef840 0x5560cdfeeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 4 client.25372 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:11:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.161+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560cdedf520 con 0x5560cdfef840 2024-03-21T09:11:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.162+0000 7f6cf9400640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560cdfef840 0x5560cdfeeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.162+0000 7f6cf9400640 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560cdfef840 0x5560cdfeeaf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69a0ee19e2980c5 server_cookie=a01cae48757bea3c in_seq=0 out_seq=0 2024-03-21T09:11:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.162+0000 7f6cdbe00640 10 client.25372 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.163+0000 7f6cdbe00640 1 -- 172.21.15.37:0/520561217 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5560cdedf520 con 0x5560cdfef840 2024-03-21T09:11:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.163+0000 7f6cdbe00640 10 client.25372 handle_command_reply: tid=0 2024-03-21T09:11:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.163+0000 7f6d03fc1740 1 client.25372 shutdown 2024-03-21T09:11:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 msgr2=0x5560cdfe3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfe3320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009de0 tx=0x7f6cf4004890 comp rx=0 tx=0).stop 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 shutdown_connections 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560cdfef840 0x5560cdfeeaf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfde8e0 0x5560cdfea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6cdbe00640 0 client.25372 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfd9770 0x5560cdfe3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/520561217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfe3320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 >> 172.21.15.37:0/520561217 conn(0x5560cdfd42f0 msgr2=0x5560cdfd34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 shutdown_connections 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 1 -- 172.21.15.37:0/520561217 wait complete. 2024-03-21T09:11:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.164+0000 7f6d03fc1740 20 client.25372 trim_cache size 0 max 16384 2024-03-21T09:11:41.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:40 smithi037 ceph-mon[31830]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:41.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:40 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4134993080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 Processor -- start 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 -- start start 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfe1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfd46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdfef060 con 0x5560cdfde8e0 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdfef6c0 con 0x5560cdfd8d80 2024-03-21T09:11:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdfefd20 con 0x5560cdfd9770 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfe1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfea7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47974/0 (socket says 172.21.15.37:47974) 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6cf9400640 1 -- 172.21.15.37:0/3349841879 learned_addr learned my addr 172.21.15.37:0/3349841879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.183+0000 7f6cfa800640 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfd46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cf9400640 1 -- 172.21.15.37:0/3349841879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 msgr2=0x5560cdfe1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cf9400640 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfe1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cf9400640 1 -- 172.21.15.37:0/3349841879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 msgr2=0x5560cdfd46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cf9400640 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfd46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cf9400640 1 -- 172.21.15.37:0/3349841879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560cdfc6170 con 0x5560cdfd9770 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cfa800640 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfe1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6cf9400640 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfea7c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8000c00 tx=0x7f6ce800a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c75426fdae63bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6ce6a00640 1 -- 172.21.15.37:0/3349841879 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8020230 con 0x5560cdfd9770 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6ce6a00640 1 -- 172.21.15.37:0/3349841879 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ce8024020 con 0x5560cdfd9770 2024-03-21T09:11:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.184+0000 7f6ce6a00640 1 -- 172.21.15.37:0/3349841879 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce801da70 con 0x5560cdfd9770 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 -- 172.21.15.37:0/3349841879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 msgr2=0x5560cdfea7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfea7c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8000c00 tx=0x7f6ce800a2d0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 -- 172.21.15.37:0/3349841879 shutdown_connections 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfd46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd9770 0x5560cdfea7c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/3349841879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd8d80 0x5560cdfe1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 -- 172.21.15.37:0/3349841879 >> 172.21.15.37:0/3349841879 conn(0x5560cdfd42f0 msgr2=0x5560cdfda210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 -- 172.21.15.37:0/3349841879 shutdown_connections 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.185+0000 7f6d03fc1740 1 -- 172.21.15.37:0/3349841879 wait complete. 2024-03-21T09:11:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 -- start start 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 Processor -- start 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 0x5560cdfea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdff09e0 con 0x5560cdfde8e0 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdff0380 con 0x5560cdfd9770 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560cdff1040 con 0x5560cdfd8d80 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.186+0000 7f6d03fc1740 10 client.0 _ll_register_callbacks cb 0x7f6ce800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 0x5560cdfea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 0x5560cdfea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54834/0 (socket says 172.21.15.37:54834) 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 -- 172.21.15.37:0/2565223681 learned_addr learned my addr 172.21.15.37:0/2565223681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cfa800640 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 -- 172.21.15.37:0/2565223681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 msgr2=0x5560cdfea250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfea250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 -- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 msgr2=0x5560cdfe6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 -- 172.21.15.37:0/2565223681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560cdfe75f0 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cf9400640 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 0x5560cdfea790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80202b0 tx=0x7f6ce80202e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fdfdd11c5113a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.187+0000 7f6cdbe00640 1 -- 172.21.15.37:0/2565223681 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce801da80 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6cdbe00640 1 -- 172.21.15.37:0/2565223681 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ce801dc20 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ce8024020 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6cdbe00640 1 -- 172.21.15.37:0/2565223681 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ce803a050 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 10 client.25352 fetch_fsmap learned FSMap version 23 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5560ce04b6e0 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6cdbe00640 1 -- 172.21.15.37:0/2565223681 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ce8024020 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6cdbe00640 1 -- 172.21.15.37:0/2565223681 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6ce8004c90 con 0x5560cdfd9770 2024-03-21T09:11:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 10 client.25352 fetch_fsmap finished waiting for FSMap version 23 2024-03-21T09:11:41.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 10 client.25352 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T09:11:41.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 20 client.25352 populate_metadata read hostname 'smithi037' 2024-03-21T09:11:41.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560ce04fa70 0x5560ce04b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 4 client.25352 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T09:11:41.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.188+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5560cdf4b1d0 con 0x5560ce04fa70 2024-03-21T09:11:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.189+0000 7f6cfa800640 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560ce04fa70 0x5560ce04b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.189+0000 7f6cfa800640 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560ce04fa70 0x5560ce04b9c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857b07218a3888a5 server_cookie=eba125c792ef4ff in_seq=0 out_seq=0 2024-03-21T09:11:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.189+0000 7f6cdbe00640 10 client.25352 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6cdbe00640 1 -- 172.21.15.37:0/2565223681 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5560cdf4b1d0 con 0x5560ce04fa70 2024-03-21T09:11:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6cdbe00640 10 client.25352 handle_command_reply: tid=0 2024-03-21T09:11:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 client.25352 shutdown 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 msgr2=0x5560cdfea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 0x5560cdfea790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80202b0 tx=0x7f6ce80202e0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 shutdown_connections 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5560cdfde8e0 0x5560cdfe6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5560ce04fa70 0x5560ce04b9c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5560cdfd9770 0x5560cdfea790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 --2- 172.21.15.37:0/2565223681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5560cdfd8d80 0x5560cdfea250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6cdbe00640 0 client.25352 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 >> 172.21.15.37:0/2565223681 conn(0x5560cdfd42f0 msgr2=0x5560cdfe1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 shutdown_connections 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 1 -- 172.21.15.37:0/2565223681 wait complete. 2024-03-21T09:11:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.190+0000 7f6d03fc1740 20 client.25352 trim_cache size 0 max 16384 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "2dc01c4e-e761-11ee-95cd-87774f69a715", 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24589, 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 174.29987969999999, 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 94, 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 184.71257153600001 2024-03-21T09:11:41.188 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ce00a2fa0 msgr2=0x7f6ce00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ce00a2fa0 0x7f6ce00a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0bd580 tx=0x7f6cec02bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 msgr2=0x7f6cfc19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc19df90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6cec036040 tx=0x7f6cec005dd0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 shutdown_connections 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6ce00a2fa0 0x7f6ce00a5460 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cfc1a3530 0x7f6cfc1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cfc1030a0 0x7f6cfc19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 --2- 172.21.15.37:0/85146733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cfc1026e0 0x7f6cfc19df90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 >> 172.21.15.37:0/85146733 conn(0x7f6cfc0fca60 msgr2=0x7f6cfc1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 shutdown_connections 2024-03-21T09:11:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:41.191+0000 7f6cfb200640 1 -- 172.21.15.37:0/85146733 wait complete. 2024-03-21T09:11:41.190 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:40 smithi053 ceph-mon[36169]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:41.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:40 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4134993080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:41.203 INFO:teuthology.run_tasks:Running task kclient... 2024-03-21T09:11:41.216 INFO:tasks.kclient:Mounting kernel clients... 2024-03-21T09:11:41.217 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-21T09:11:41.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:41.217 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-21T09:11:41.268 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-21T09:11:41.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:41.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T09:11:41.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:40 smithi071 ceph-mon[36597]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-21T09:11:41.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:40 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4134993080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:41.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:41.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-03-21T09:11:41.478 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-21T09:11:41.479 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T09:11:41.479 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T09:11:41.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T09:11:41.938 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:43.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:42 smithi053 ceph-mon[36169]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:43.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:42 smithi071 ceph-mon[36597]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:43.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:42 smithi037 ceph-mon[31830]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:11:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 -- 172.21.15.37:0/4133120771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc102440 msgr2=0x7ff8dc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/4133120771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc102440 0x7ff8dc102820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00099c0 tx=0x7ff8d0009cd0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 -- 172.21.15.37:0/4133120771 shutdown_connections 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/4133120771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc107510 0x7ff8dc10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/4133120771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc102440 0x7ff8dc102820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/4133120771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102df0 0x7ff8dc106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 -- 172.21.15.37:0/4133120771 >> 172.21.15.37:0/4133120771 conn(0x7ff8dc0792f0 msgr2=0x7ff8dc079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 -- 172.21.15.37:0/4133120771 shutdown_connections 2024-03-21T09:11:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.297+0000 7ff8e3a00700 1 -- 172.21.15.37:0/4133120771 wait complete. 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.299+0000 7ff8e3a00700 1 Processor -- start 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.299+0000 7ff8e3a00700 1 -- start start 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.299+0000 7ff8e3a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 0x7ff8dc1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e3a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc102df0 0x7ff8dc1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e3a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc107510 0x7ff8dc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e3a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc112540 con 0x7ff8dc102df0 2024-03-21T09:11:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e3a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc1123c0 con 0x7ff8dc102440 2024-03-21T09:11:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e3a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc1126c0 con 0x7ff8dc107510 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 0x7ff8dc1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 0x7ff8dc1a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53332/0 (socket says 172.21.15.37:53332) 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 -- 172.21.15.37:0/3287326316 learned_addr learned my addr 172.21.15.37:0/3287326316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc107510 0x7ff8dc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8dbe00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc102df0 0x7ff8dc1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 -- 172.21.15.37:0/3287326316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc107510 msgr2=0x7ff8dc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc107510 0x7ff8dc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 -- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc102df0 msgr2=0x7ff8dc1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc102df0 0x7ff8dc1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8e1000700 1 -- 172.21.15.37:0/3287326316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc19b1a0 con 0x7ff8dc102440 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.300+0000 7ff8dbe00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc102df0 0x7ff8dc1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.301+0000 7ff8e1000700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 0x7ff8dc1a0880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0005910 tx=0x7ff8d000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.301+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0027070 con 0x7ff8dc102440 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.301+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0003680 con 0x7ff8dc102440 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.301+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc19b430 con 0x7ff8dc102440 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.301+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00173f0 con 0x7ff8dc102440 2024-03-21T09:11:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.302+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc1a7750 con 0x7ff8dc102440 2024-03-21T09:11:44.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.304+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc068e50 con 0x7ff8dc102440 2024-03-21T09:11:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.306+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff8d002a070 con 0x7ff8dc102440 2024-03-21T09:11:44.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.307+0000 7ff8d9600700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff8c40a3030 0x7ff8c40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.307+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8d00cff80 con 0x7ff8dc102440 2024-03-21T09:11:44.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.307+0000 7ff8dbe00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff8c40a3030 0x7ff8c40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.308+0000 7ff8dbe00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff8c40a3030 0x7ff8c40a54f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc076270 tx=0x7ff8c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.311+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8d0013070 con 0x7ff8dc102440 2024-03-21T09:11:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.593+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff8dc1a7920 con 0x7ff8dc102440 2024-03-21T09:11:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.594+0000 7ff8d9600700 1 -- 172.21.15.37:0/3287326316 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff8d009ca50 con 0x7ff8dc102440 2024-03-21T09:11:44.588 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff8c40a3030 msgr2=0x7ff8c40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff8c40a3030 0x7ff8c40a54f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc076270 tx=0x7ff8c8009260 comp rx=0 tx=0).stop 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 msgr2=0x7ff8dc1a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 0x7ff8dc1a0880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0005910 tx=0x7ff8d000fa00 comp rx=0 tx=0).stop 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 shutdown_connections 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc107510 0x7ff8dc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7ff8c40a3030 0x7ff8c40a54f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8dc102440 0x7ff8dc1a0880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 --2- 172.21.15.37:0/3287326316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8dc102df0 0x7ff8dc1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 >> 172.21.15.37:0/3287326316 conn(0x7ff8dc0792f0 msgr2=0x7ff8dc075600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 shutdown_connections 2024-03-21T09:11:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:44.597+0000 7ff8e3a00700 1 -- 172.21.15.37:0/3287326316 wait complete. 2024-03-21T09:11:45.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:44 smithi053 ceph-mon[36169]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:45.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:44 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3287326316' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T09:11:45.240 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:44 smithi037 ceph-mon[31830]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:45.240 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:44 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3287326316' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T09:11:45.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:44 smithi071 ceph-mon[36597]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:11:45.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:44 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3287326316' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T09:11:45.602 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T09:11:45.602 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T09:11:45.602 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-03-21T09:11:45.602 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-21T09:11:45.603 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T09:11:45.603 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T09:11:45.603 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T09:11:45.603 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T09:11:45.603 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-21T09:11:45.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:45.603 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:11:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41855sec preferred_lft 41855sec 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-03-21T09:11:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-03-21T09:11:45.665 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T09:11:45.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T09:11:45.665 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:11:45.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:45.903 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T09:11:45.979 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-03-21T09:11:45.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:45.983 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-03-21T09:11:46.039 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-03-21T09:11:46.039 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T09:11:46.039 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T09:11:46.039 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T09:11:46.039 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T09:11:46.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:46.040 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:11:46.040 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:11:46.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T09:11:46.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T09:11:46.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T09:11:46.041 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:11:46.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:46.230 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-21T09:11:46.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:46.279 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-03-21T09:11:46.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:46.329 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:11:46.329 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:11:46.329 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T09:11:46.329 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-21T09:11:46.329 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:11:46.468 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-21T09:11:46.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-21T09:11:46.469 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:11:46.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:46.755 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-21T09:11:46.755 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-21T09:11:46.755 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-03-21T09:11:46.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T09:11:46.756 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-21T09:11:46.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T09:11:46.937 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-21T09:11:46.983 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-21T09:11:46.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T09:11:46.984 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-21T09:11:47.078 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-21T09:11:47.078 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-21T09:11:47.078 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/cb278297-306c-49a2-90ad-118e60fdafc6 2024-03-21T09:11:47.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:47.079 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/cb278297-306c-49a2-90ad-118e60fdafc6 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-21T09:11:47.191 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:46 smithi053 ceph-mon[36169]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:47.217 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-21T09:11:47.217 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-21T09:11:47.217 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-03-21T09:11:47.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:46 smithi037 ceph-mon[31830]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:47.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:46 smithi071 ceph-mon[36597]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-21T09:11:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-21T09:11:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-21T09:11:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-03-21T09:11:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.53:6789,172.21.15.71:6789" 2024-03-21T09:11:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.53:6789,172.21.15.71:6789:/volumes/qa/sv_1/cb278297-306c-49a2-90ad-118e60fdafc6 2024-03-21T09:11:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=2dc01c4e-e761-11ee-95cd-87774f69a715" will pass to kernel 2024-03-21T09:11:47.892 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-21T09:11:47.893 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-03-21T09:11:47.893 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-21T09:11:47.893 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-03-21T09:11:47.893 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-21T09:11:47.900 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-21T09:11:47.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:47.900 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-21T09:11:47.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:47.996 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T09:11:48.051 INFO:teuthology.orchestra.run.smithi037.stdout:client15240 2024-03-21T09:11:48.052 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:11:48.052 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:11:48.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:48.096 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T09:11:48.194 INFO:teuthology.orchestra.run.smithi037.stdout:client15240 2024-03-21T09:11:48.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:48.194 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T09:11:48.245 INFO:teuthology.orchestra.run.smithi037.stdout:client15240 2024-03-21T09:11:48.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:48.245 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715.client15240/status') 2024-03-21T09:11:48.334 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.15240 (3)192.168.0.1:0/2859613724 2024-03-21T09:11:48.335 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-03-21T09:11:48.335 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-03-21T09:11:48.335 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-03-21T09:11:48.335 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 7.6195e-05 s, 919 kB/s 2024-03-21T09:11:48.339 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T09:11:48.349 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T09:11:48.350 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:11:48.350 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-21T09:11:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 -- 172.21.15.37:0/708347068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c1021b0 msgr2=0x7f8c5c1045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 --2- 172.21.15.37:0/708347068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c1021b0 0x7f8c5c1045a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 -- 172.21.15.37:0/708347068 shutdown_connections 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 --2- 172.21.15.37:0/708347068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 --2- 172.21.15.37:0/708347068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5c104ae0 0x7f8c5c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 --2- 172.21.15.37:0/708347068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c1021b0 0x7f8c5c1045a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.564+0000 7f8c61e00640 1 -- 172.21.15.37:0/708347068 >> 172.21.15.37:0/708347068 conn(0x7f8c5c0fbb40 msgr2=0x7f8c5c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.565+0000 7f8c61e00640 1 -- 172.21.15.37:0/708347068 shutdown_connections 2024-03-21T09:11:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.565+0000 7f8c61e00640 1 -- 172.21.15.37:0/708347068 wait complete. 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 Processor -- start 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 -- start start 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5c1021b0 0x7f8c5c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c104ae0 0x7f8c5c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c10d390 con 0x7f8c5c1021b0 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c10d210 con 0x7f8c5c107410 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c61e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c10d510 con 0x7f8c5c104ae0 2024-03-21T09:11:48.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c5b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c5a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c104ae0 0x7f8c5c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c5aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5c1021b0 0x7f8c5c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c5b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c1a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53346/0 (socket says 172.21.15.37:53346) 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.566+0000 7f8c5b400640 1 -- 172.21.15.37:0/1688870890 learned_addr learned my addr 172.21.15.37:0/1688870890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c5b400640 1 -- 172.21.15.37:0/1688870890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c104ae0 msgr2=0x7f8c5c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c5b400640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c104ae0 0x7f8c5c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c5b400640 1 -- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5c1021b0 msgr2=0x7f8c5c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c5b400640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5c1021b0 0x7f8c5c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c5b400640 1 -- 172.21.15.37:0/1688870890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c1a1890 con 0x7f8c5c107410 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c5b400640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c1a11f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400eb10 tx=0x7f8c5400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004b20 con 0x7f8c5c107410 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c1a1b80 con 0x7f8c5c107410 2024-03-21T09:11:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c54004cc0 con 0x7f8c5c107410 2024-03-21T09:11:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.567+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54010760 con 0x7f8c5c107410 2024-03-21T09:11:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.568+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c1a20a0 con 0x7f8c5c107410 2024-03-21T09:11:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.569+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c108f00 con 0x7f8c5c107410 2024-03-21T09:11:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.571+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8c54005020 con 0x7f8c5c107410 2024-03-21T09:11:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.572+0000 7f8c60a00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8c400a2f10 0x7f8c400a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.572+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8c540cfdc0 con 0x7f8c5c107410 2024-03-21T09:11:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.572+0000 7f8c5aa00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8c400a2f10 0x7f8c400a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.573+0000 7f8c5aa00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8c400a2f10 0x7f8c400a53d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02c240 tx=0x7f8c4c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:48.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.576+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c5409c800 con 0x7f8c5c107410 2024-03-21T09:11:48.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.824+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f8c5c0622a0 con 0x7f8c5c107410 2024-03-21T09:11:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.963+0000 7f8c60a00640 1 -- 172.21.15.37:0/1688870890 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c5409c100 con 0x7f8c5c107410 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8c400a2f10 msgr2=0x7f8c400a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8c400a2f10 0x7f8c400a53d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02c240 tx=0x7f8c4c02df90 comp rx=0 tx=0).stop 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 msgr2=0x7f8c5c1a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c1a11f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400eb10 tx=0x7f8c5400c2d0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 shutdown_connections 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f8c400a2f10 0x7f8c400a53d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c107410 0x7f8c5c1a11f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c5c104ae0 0x7f8c5c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 --2- 172.21.15.37:0/1688870890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5c1021b0 0x7f8c5c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:48.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 >> 172.21.15.37:0/1688870890 conn(0x7f8c5c0fbb40 msgr2=0x7f8c5c1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:48.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 shutdown_connections 2024-03-21T09:11:48.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:48.967+0000 7f8c61e00640 1 -- 172.21.15.37:0/1688870890 wait complete. 2024-03-21T09:11:48.982 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-21T09:11:49.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 -- 172.21.15.37:0/2931459704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 msgr2=0x7fa6ac109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 --2- 172.21.15.37:0/2931459704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 0x7fa6ac109960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009be0 tx=0x7fa6a402bef0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 -- 172.21.15.37:0/2931459704 shutdown_connections 2024-03-21T09:11:49.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 --2- 172.21.15.37:0/2931459704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 0x7fa6ac109960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 --2- 172.21.15.37:0/2931459704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac102710 0x7fa6ac102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 --2- 172.21.15.37:0/2931459704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac101520 0x7fa6ac101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 -- 172.21.15.37:0/2931459704 >> 172.21.15.37:0/2931459704 conn(0x7fa6ac0fcab0 msgr2=0x7fa6ac0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:49.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 -- 172.21.15.37:0/2931459704 shutdown_connections 2024-03-21T09:11:49.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.194+0000 7fa6abe00640 1 -- 172.21.15.37:0/2931459704 wait complete. 2024-03-21T09:11:49.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 Processor -- start 2024-03-21T09:11:49.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 -- start start 2024-03-21T09:11:49.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 0x7fa6ac1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac102710 0x7fa6ac199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 0x7fa6ac1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10d200 con 0x7fa6ac102710 2024-03-21T09:11:49.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10d080 con 0x7fa6ac101520 2024-03-21T09:11:49.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.196+0000 7fa6abe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10d380 con 0x7fa6ac1030d0 2024-03-21T09:11:49.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 0x7fa6ac1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 0x7fa6ac1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53366/0 (socket says 172.21.15.37:53366) 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 -- 172.21.15.37:0/1784597981 learned_addr learned my addr 172.21.15.37:0/1784597981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aa000640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac102710 0x7fa6ac199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6ab400640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 0x7fa6ac1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 -- 172.21.15.37:0/1784597981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 msgr2=0x7fa6ac1a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 0x7fa6ac1a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 -- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac102710 msgr2=0x7fa6ac199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac102710 0x7fa6ac199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 -- 172.21.15.37:0/1784597981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac1a15e0 con 0x7fa6ac101520 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.197+0000 7fa6aaa00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 0x7fa6ac1999c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9f0 tx=0x7fa69c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.198+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00ccc0 con 0x7fa6ac101520 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.198+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac1a18d0 con 0x7fa6ac101520 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.198+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa69c0035f0 con 0x7fa6ac101520 2024-03-21T09:11:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.198+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00ce60 con 0x7fa6ac101520 2024-03-21T09:11:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.198+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac1a1e20 con 0x7fa6ac101520 2024-03-21T09:11:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.199+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac1a20b0 con 0x7fa6ac101520 2024-03-21T09:11:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.201+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa69c0037a0 con 0x7fa6ac101520 2024-03-21T09:11:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.202+0000 7fa6a3400640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa6900a2f00 0x7fa6900a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.202+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa69c0cfa20 con 0x7fa6ac101520 2024-03-21T09:11:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.203+0000 7fa6aa000640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa6900a2f00 0x7fa6900a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.203+0000 7fa6aa000640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa6900a2f00 0x7fa6900a53c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac19aed0 tx=0x7fa698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:49.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.209+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa69c09c460 con 0x7fa6ac101520 2024-03-21T09:11:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:48 smithi071 ceph-mon[36597]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:48 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1688870890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T09:11:49.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:48 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T09:11:49.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:48 smithi037 ceph-mon[31830]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:49.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:48 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1688870890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T09:11:49.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:48 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T09:11:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.441+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6ac074ff0 con 0x7fa6900a2f00 2024-03-21T09:11:49.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:48 smithi053 ceph-mon[36169]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-21T09:11:49.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:48 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1688870890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T09:11:49.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:48 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T09:11:49.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.465+0000 7fa6a3400640 1 -- 172.21.15.37:0/1784597981 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fa6ac074ff0 con 0x7fa6900a2f00 2024-03-21T09:11:49.459 INFO:teuthology.orchestra.run.smithi037.stdout:{} 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa6900a2f00 msgr2=0x7fa6900a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa6900a2f00 0x7fa6900a53c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac19aed0 tx=0x7fa698005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 msgr2=0x7fa6ac1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 0x7fa6ac1999c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9f0 tx=0x7fa69c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 shutdown_connections 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa6900a2f00 0x7fa6900a53c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6ac1030d0 0x7fa6ac1a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac102710 0x7fa6ac199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 --2- 172.21.15.37:0/1784597981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6ac101520 0x7fa6ac1999c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 >> 172.21.15.37:0/1784597981 conn(0x7fa6ac0fcab0 msgr2=0x7fa6ac0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 shutdown_connections 2024-03-21T09:11:49.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.468+0000 7fa6abe00640 1 -- 172.21.15.37:0/1784597981 wait complete. 2024-03-21T09:11:49.480 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-21T09:11:49.491 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-21T09:11:49.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 -- 172.21.15.37:0/1760548078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 msgr2=0x7fceb4101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 --2- 172.21.15.37:0/1760548078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb4101b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fceac009ca0 tx=0x7fceac02c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 -- 172.21.15.37:0/1760548078 shutdown_connections 2024-03-21T09:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 --2- 172.21.15.37:0/1760548078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fceb41032e0 0x7fceb4109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 --2- 172.21.15.37:0/1760548078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb4102920 0x7fceb4102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 --2- 172.21.15.37:0/1760548078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb4101b50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 -- 172.21.15.37:0/1760548078 >> 172.21.15.37:0/1760548078 conn(0x7fceb40fccc0 msgr2=0x7fceb40ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.752+0000 7fceb8a00640 1 -- 172.21.15.37:0/1760548078 shutdown_connections 2024-03-21T09:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.753+0000 7fceb8a00640 1 -- 172.21.15.37:0/1760548078 wait complete. 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.753+0000 7fceb8a00640 1 Processor -- start 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 -- start start 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fceb4102920 0x7fceb4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb41032e0 0x7fceb419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb410d310 con 0x7fceb4102920 2024-03-21T09:11:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb410d190 con 0x7fceb41032e0 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb410d490 con 0x7fceb4101730 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35968/0 (socket says 172.21.15.37:35968) 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb1e00640 1 -- 172.21.15.37:0/2493192595 learned_addr learned my addr 172.21.15.37:0/2493192595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb2800640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb41032e0 0x7fceb419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb1400640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fceb4102920 0x7fceb4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb1e00640 1 -- 172.21.15.37:0/2493192595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb41032e0 msgr2=0x7fceb419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.754+0000 7fceb1e00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb41032e0 0x7fceb419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceb1e00640 1 -- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fceb4102920 msgr2=0x7fceb4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceb1e00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fceb4102920 0x7fceb4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceb1e00640 1 -- 172.21.15.37:0/2493192595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb419f600 con 0x7fceb4101730 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceb1e00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb41979e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fceac0324c0 tx=0x7fceac032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac041020 con 0x7fceb4101730 2024-03-21T09:11:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fceac034cd0 con 0x7fceb4101730 2024-03-21T09:11:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb419f890 con 0x7fceb4101730 2024-03-21T09:11:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac039040 con 0x7fceb4101730 2024-03-21T09:11:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.755+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb419fd80 con 0x7fceb4101730 2024-03-21T09:11:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.756+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb4108aa0 con 0x7fceb4101730 2024-03-21T09:11:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.758+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fceac044020 con 0x7fceb4101730 2024-03-21T09:11:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.759+0000 7fceaaa00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fce980a2f20 0x7fce980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.760+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fceac0f17a0 con 0x7fceb4101730 2024-03-21T09:11:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.760+0000 7fceb1400640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fce980a2f20 0x7fce980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.761+0000 7fceb1400640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fce980a2f20 0x7fce980a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fceb4198ef0 tx=0x7fce9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.764+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fceac04b020 con 0x7fceb4101730 2024-03-21T09:11:49.922 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:11:49.449+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:11:49.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:11:49.449+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:11:49.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:11:49.449+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:11:49.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:11:49.449+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:11:49.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:11:49.449+0000 7f454ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:11:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.973+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fceb40622a0 con 0x7fceb4101730 2024-03-21T09:11:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.978+0000 7fceaaa00640 1 -- 172.21.15.37:0/2493192595 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7fceac0be270 con 0x7fceb4101730 2024-03-21T09:11:49.972 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:49.972 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:49.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fce980a2f20 msgr2=0x7fce980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fce980a2f20 0x7fce980a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fceb4198ef0 tx=0x7fce9c005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:49.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 msgr2=0x7fceb41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:49.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb41979e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fceac0324c0 tx=0x7fceac032f40 comp rx=0 tx=0).stop 2024-03-21T09:11:49.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 shutdown_connections 2024-03-21T09:11:49.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fce980a2f20 0x7fce980a53e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb41032e0 0x7fceb419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fceb4102920 0x7fceb4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 --2- 172.21.15.37:0/2493192595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb4101730 0x7fceb41979e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:49.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 >> 172.21.15.37:0/2493192595 conn(0x7fceb40fccc0 msgr2=0x7fceb4105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:49.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 shutdown_connections 2024-03-21T09:11:49.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:49.980+0000 7fceb8a00640 1 -- 172.21.15.37:0/2493192595 wait complete. 2024-03-21T09:11:49.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:11:49.995 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 -- 172.21.15.37:0/3637170378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc103700 msgr2=0x7f03dc103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 --2- 172.21.15.37:0/3637170378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc103700 0x7f03dc103b20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009900 tx=0x7f03cc02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 -- 172.21.15.37:0/3637170378 shutdown_connections 2024-03-21T09:11:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 --2- 172.21.15.37:0/3637170378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 0x7f03dc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 --2- 172.21.15.37:0/3637170378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc1048f0 0x7f03dc104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 --2- 172.21.15.37:0/3637170378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc103700 0x7f03dc103b20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 -- 172.21.15.37:0/3637170378 >> 172.21.15.37:0/3637170378 conn(0x7f03dc0fec70 msgr2=0x7f03dc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 -- 172.21.15.37:0/3637170378 shutdown_connections 2024-03-21T09:11:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.199+0000 7f03e1200640 1 -- 172.21.15.37:0/3637170378 wait complete. 2024-03-21T09:11:50.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 Processor -- start 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 -- start start 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc103700 0x7f03dc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 0x7f03dc199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 0x7f03dc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc10f3d0 con 0x7f03dc103700 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc10f250 con 0x7f03dc1052b0 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.201+0000 7f03e1200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc10f550 con 0x7f03dc1048f0 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc103700 0x7f03dc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 0x7f03dc199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 0x7f03dc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 0x7f03dc199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35988/0 (socket says 172.21.15.37:35988) 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 -- 172.21.15.37:0/629055309 learned_addr learned my addr 172.21.15.37:0/629055309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc103700 0x7f03dc199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50278/0 (socket says 172.21.15.37:50278) 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 -- 172.21.15.37:0/629055309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 msgr2=0x7f03dc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 0x7f03dc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 -- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc103700 msgr2=0x7f03dc199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc103700 0x7f03dc199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 -- 172.21.15.37:0/629055309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc1a1640 con 0x7f03dc1048f0 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03dbe00640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 0x7f03dc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03daa00640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 0x7f03dc199f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d950 tx=0x7f03c800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.202+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8017070 con 0x7f03dc1048f0 2024-03-21T09:11:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.203+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c8009c30 con 0x7f03dc1048f0 2024-03-21T09:11:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.203+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc1a1930 con 0x7f03dc1048f0 2024-03-21T09:11:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.203+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800f6b0 con 0x7f03dc1048f0 2024-03-21T09:11:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.203+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc1a1e80 con 0x7f03dc1048f0 2024-03-21T09:11:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.204+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc10aa20 con 0x7f03dc1048f0 2024-03-21T09:11:50.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.206+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f03c80109d0 con 0x7f03dc1048f0 2024-03-21T09:11:50.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.208+0000 7f03d3e00640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f03c00a2f20 0x7f03c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.208+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f03c80cf350 con 0x7f03dc1048f0 2024-03-21T09:11:50.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.208+0000 7f03db400640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f03c00a2f20 0x7f03c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.209+0000 7f03db400640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f03c00a2f20 0x7f03c00a53e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03cc02c170 tx=0x7f03cc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:50.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.209+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f03c8009de0 con 0x7f03dc1048f0 2024-03-21T09:11:50.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:49 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T09:11:50.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:49 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:11:50.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:49 smithi071 ceph-mon[36597]: from='client.25370 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:50.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.412+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03dc02d2c0 con 0x7f03dc1048f0 2024-03-21T09:11:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.412+0000 7f03d3e00640 1 -- 172.21.15.37:0/629055309 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03c809b740 con 0x7f03dc1048f0 2024-03-21T09:11:50.406 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.415+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f03c00a2f20 msgr2=0x7f03c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.415+0000 7f03e1200640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f03c00a2f20 0x7f03c00a53e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03cc02c170 tx=0x7f03cc02df90 comp rx=0 tx=0).stop 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.415+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 msgr2=0x7f03dc199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.415+0000 7f03e1200640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 0x7f03dc199f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d950 tx=0x7f03c800de10 comp rx=0 tx=0).stop 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.415+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 shutdown_connections 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f03c00a2f20 0x7f03c00a53e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03dc1052b0 0x7f03dc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc1048f0 0x7f03dc199f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 --2- 172.21.15.37:0/629055309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03dc103700 0x7f03dc199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 >> 172.21.15.37:0/629055309 conn(0x7f03dc0fec70 msgr2=0x7f03dc107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 shutdown_connections 2024-03-21T09:11:50.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.416+0000 7f03e1200640 1 -- 172.21.15.37:0/629055309 wait complete. 2024-03-21T09:11:50.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T09:11:50.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:11:50.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:49 smithi037 ceph-mon[31830]: from='client.25370 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:50.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:50.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:49 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T09:11:50.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:49 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:11:50.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:49 smithi053 ceph-mon[36169]: from='client.25370 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.660+0000 7fa838600640 1 -- 172.21.15.37:0/3900584480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 msgr2=0x7fa830102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.660+0000 7fa838600640 1 --2- 172.21.15.37:0/3900584480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 0x7fa830102d20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009900 tx=0x7fa81c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 -- 172.21.15.37:0/3900584480 shutdown_connections 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 --2- 172.21.15.37:0/3900584480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa830103260 0x7fa830109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 --2- 172.21.15.37:0/3900584480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 0x7fa830102d20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 --2- 172.21.15.37:0/3900584480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8301016b0 0x7fa830101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 -- 172.21.15.37:0/3900584480 >> 172.21.15.37:0/3900584480 conn(0x7fa8300fcc20 msgr2=0x7fa8300ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 -- 172.21.15.37:0/3900584480 shutdown_connections 2024-03-21T09:11:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.661+0000 7fa838600640 1 -- 172.21.15.37:0/3900584480 wait complete. 2024-03-21T09:11:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 Processor -- start 2024-03-21T09:11:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 -- start start 2024-03-21T09:11:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8301016b0 0x7fa830075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 0x7fa830073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 0x7fa830073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83010d360 con 0x7fa8301016b0 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83010d1e0 con 0x7fa830103260 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa838600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83010d4e0 con 0x7fa8301028a0 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa836600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 0x7fa830073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa835200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 0x7fa830073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa835c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8301016b0 0x7fa830075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.662+0000 7fa836600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 0x7fa830073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53438/0 (socket says 172.21.15.37:53438) 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 -- 172.21.15.37:0/1438086023 learned_addr learned my addr 172.21.15.37:0/1438086023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 -- 172.21.15.37:0/1438086023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 msgr2=0x7fa830073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 0x7fa830073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 -- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8301016b0 msgr2=0x7fa830075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8301016b0 0x7fa830075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 -- 172.21.15.37:0/1438086023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830074860 con 0x7fa830103260 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa836600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 0x7fa830073fd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa82800eb10 tx=0x7fa82800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828004b20 con 0x7fa830103260 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa828004cc0 con 0x7fa830103260 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.663+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8301a1ad0 con 0x7fa830103260 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.664+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828010760 con 0x7fa830103260 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.664+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8301a1fa0 con 0x7fa830103260 2024-03-21T09:11:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.665+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa830068970 con 0x7fa830103260 2024-03-21T09:11:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.667+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa828005020 con 0x7fa830103260 2024-03-21T09:11:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.668+0000 7fa82e000640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8180a2f20 0x7fa8180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.668+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa8280cfda0 con 0x7fa830103260 2024-03-21T09:11:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.668+0000 7fa835c00640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8180a2f20 0x7fa8180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.669+0000 7fa835c00640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8180a2f20 0x7fa8180a53e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa8240095f0 tx=0x7fa824005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:50.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.677+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa82809c7e0 con 0x7fa830103260 2024-03-21T09:11:50.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.901+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8300622a0 con 0x7fa830103260 2024-03-21T09:11:50.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.902+0000 7fa82e000640 1 -- 172.21.15.37:0/1438086023 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa82809c0e0 con 0x7fa830103260 2024-03-21T09:11:50.895 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8180a2f20 msgr2=0x7fa8180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8180a2f20 0x7fa8180a53e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa8240095f0 tx=0x7fa824005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 msgr2=0x7fa830073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 0x7fa830073fd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa82800eb10 tx=0x7fa82800c2d0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 shutdown_connections 2024-03-21T09:11:50.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa8180a2f20 0x7fa8180a53e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa830103260 0x7fa830073fd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8301028a0 0x7fa830073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 --2- 172.21.15.37:0/1438086023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8301016b0 0x7fa830075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 >> 172.21.15.37:0/1438086023 conn(0x7fa8300fcc20 msgr2=0x7fa830105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 shutdown_connections 2024-03-21T09:11:50.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:50.905+0000 7fa838600640 1 -- 172.21.15.37:0/1438086023 wait complete. 2024-03-21T09:11:50.918 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.124+0000 7f274a400640 1 -- 172.21.15.37:0/3544921831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27441014e0 msgr2=0x7f2744101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.124+0000 7f274a400640 1 --2- 172.21.15.37:0/3544921831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27441014e0 0x7f2744101900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2738009900 tx=0x7f273802bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 -- 172.21.15.37:0/3544921831 shutdown_connections 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 --2- 172.21.15.37:0/3544921831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2744103090 0x7f2744109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 --2- 172.21.15.37:0/3544921831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 0x7f2744102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 --2- 172.21.15.37:0/3544921831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27441014e0 0x7f2744101900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 -- 172.21.15.37:0/3544921831 >> 172.21.15.37:0/3544921831 conn(0x7f27440fca50 msgr2=0x7f27440fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 -- 172.21.15.37:0/3544921831 shutdown_connections 2024-03-21T09:11:51.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.125+0000 7f274a400640 1 -- 172.21.15.37:0/3544921831 wait complete. 2024-03-21T09:11:51.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 Processor -- start 2024-03-21T09:11:51.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 -- start start 2024-03-21T09:11:51.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27441014e0 0x7f2744075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 0x7f2744073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 0x7f2744073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274410d1c0 con 0x7f27441014e0 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274410d040 con 0x7f2744103090 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.126+0000 7f274a400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274410d340 con 0x7f27441026d0 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 0x7f2744073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27441014e0 0x7f2744075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2743e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 0x7f2744073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 0x7f2744073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53466/0 (socket says 172.21.15.37:53466) 2024-03-21T09:11:51.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 -- 172.21.15.37:0/4203277452 learned_addr learned my addr 172.21.15.37:0/4203277452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2743e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 0x7f2744073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:36020/0 (socket says 172.21.15.37:36020) 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 -- 172.21.15.37:0/4203277452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 msgr2=0x7f2744073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 0x7f2744073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 -- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27441014e0 msgr2=0x7f2744075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27441014e0 0x7f2744075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 -- 172.21.15.37:0/4203277452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744074860 con 0x7f2744103090 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2749a00640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 0x7f2744073fd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f273c00eaa0 tx=0x7f273c00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.127+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00cdb0 con 0x7f2744103090 2024-03-21T09:11:51.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.128+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f273c005020 con 0x7f2744103090 2024-03-21T09:11:51.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.128+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27441a1970 con 0x7f2744103090 2024-03-21T09:11:51.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.128+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c0116b0 con 0x7f2744103090 2024-03-21T09:11:51.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.128+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27441a1ec0 con 0x7f2744103090 2024-03-21T09:11:51.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.129+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2744108800 con 0x7f2744103090 2024-03-21T09:11:51.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.130+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f273c005560 con 0x7f2744103090 2024-03-21T09:11:51.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.133+0000 7f2741600640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f272c0a3030 0x7f272c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.133+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f273c0cfb00 con 0x7f2744103090 2024-03-21T09:11:51.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.133+0000 7f2749000640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f272c0a3030 0x7f272c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.133+0000 7f2749000640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f272c0a3030 0x7f272c0a54f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f273802c170 tx=0x7f273802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:51.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.136+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f273c016030 con 0x7f2744103090 2024-03-21T09:11:51.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:50 smithi037 ceph-mon[31830]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T09:11:51.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:50 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2493192595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:51.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:50 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/629055309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:51.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:50 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1438086023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:51.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:50 smithi071 ceph-mon[36597]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T09:11:51.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:50 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2493192595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:51.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:50 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/629055309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:51.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:50 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1438086023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.407+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f274402d220 con 0x7f2744103090 2024-03-21T09:11:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.408+0000 7f2741600640 1 -- 172.21.15.37:0/4203277452 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f273c09bd90 con 0x7f2744103090 2024-03-21T09:11:51.402 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f272c0a3030 msgr2=0x7f272c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f272c0a3030 0x7f272c0a54f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f273802c170 tx=0x7f273802df90 comp rx=0 tx=0).stop 2024-03-21T09:11:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 msgr2=0x7f2744073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 0x7f2744073fd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f273c00eaa0 tx=0x7f273c00ef60 comp rx=0 tx=0).stop 2024-03-21T09:11:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 shutdown_connections 2024-03-21T09:11:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f272c0a3030 0x7f272c0a54f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2744103090 0x7f2744073fd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27441026d0 0x7f2744073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 --2- 172.21.15.37:0/4203277452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27441014e0 0x7f2744075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.410+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 >> 172.21.15.37:0/4203277452 conn(0x7f27440fca50 msgr2=0x7f27441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:51.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.411+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 shutdown_connections 2024-03-21T09:11:51.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.411+0000 7f274a400640 1 -- 172.21.15.37:0/4203277452 wait complete. 2024-03-21T09:11:51.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:50 smithi053 ceph-mon[36169]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T09:11:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:50 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2493192595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:50 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/629055309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:51.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:50 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1438086023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 -- 172.21.15.37:0/403299857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 msgr2=0x7f95b4100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 --2- 172.21.15.37:0/403299857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b4100810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009900 tx=0x7f95a802bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 -- 172.21.15.37:0/403299857 shutdown_connections 2024-03-21T09:11:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 --2- 172.21.15.37:0/403299857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b4101fa0 0x7f95b4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 --2- 172.21.15.37:0/403299857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 0x7f95b4101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 --2- 172.21.15.37:0/403299857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b4100810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.664+0000 7f95ba400640 1 -- 172.21.15.37:0/403299857 >> 172.21.15.37:0/403299857 conn(0x7f95b40fb980 msgr2=0x7f95b40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:51.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.665+0000 7f95ba400640 1 -- 172.21.15.37:0/403299857 shutdown_connections 2024-03-21T09:11:51.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.665+0000 7f95ba400640 1 -- 172.21.15.37:0/403299857 wait complete. 2024-03-21T09:11:51.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.665+0000 7f95ba400640 1 Processor -- start 2024-03-21T09:11:51.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 -- start start 2024-03-21T09:11:51.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b41978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 0x7f95b4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b4101fa0 0x7f95b419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b410c0d0 con 0x7f95b41015e0 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b410bf50 con 0x7f95b4101fa0 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95ba400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b410c250 con 0x7f95b41003f0 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 0x7f95b4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95b9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b41978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95b9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b4101fa0 0x7f95b419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95b9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b41978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:36046/0 (socket says 172.21.15.37:36046) 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95b9000640 1 -- 172.21.15.37:0/1217250946 learned_addr learned my addr 172.21.15.37:0/1217250946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.666+0000 7f95b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 0x7f95b4197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50330/0 (socket says 172.21.15.37:50330) 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b9000640 1 -- 172.21.15.37:0/1217250946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b4101fa0 msgr2=0x7f95b419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b9000640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b4101fa0 0x7f95b419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b9000640 1 -- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 msgr2=0x7f95b4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b9000640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 0x7f95b4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b9000640 1 -- 172.21.15.37:0/1217250946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b419f4a0 con 0x7f95b41003f0 2024-03-21T09:11:51.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b9000640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b41978d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005ab0 tx=0x7f95a802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:51.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8043020 con 0x7f95b41003f0 2024-03-21T09:11:51.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95a80054e0 con 0x7f95b41003f0 2024-03-21T09:11:51.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8003ce0 con 0x7f95b41003f0 2024-03-21T09:11:51.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.667+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b419f730 con 0x7f95b41003f0 2024-03-21T09:11:51.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.668+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b419fc80 con 0x7f95b41003f0 2024-03-21T09:11:51.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.669+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b419fe80 con 0x7f95b41003f0 2024-03-21T09:11:51.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.670+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f95a8046020 con 0x7f95b41003f0 2024-03-21T09:11:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.672+0000 7f95b1600640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f959c0a2fb0 0x7f959c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.672+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f95a80f1980 con 0x7f95b41003f0 2024-03-21T09:11:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.672+0000 7f95b3e00640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f959c0a2fb0 0x7f959c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.673+0000 7f95b3e00640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f959c0a2fb0 0x7f959c0a5470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95b4198d90 tx=0x7f95a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.675+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f95a804d020 con 0x7f95b41003f0 2024-03-21T09:11:51.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.879+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f95b41a0050 con 0x7f95b41003f0 2024-03-21T09:11:51.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.879+0000 7f95b1600640 1 -- 172.21.15.37:0/1217250946 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f95a80be370 con 0x7f95b41003f0 2024-03-21T09:11:51.874 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.882+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f959c0a2fb0 msgr2=0x7f959c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.882+0000 7f95ba400640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f959c0a2fb0 0x7f959c0a5470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95b4198d90 tx=0x7f95a000b410 comp rx=0 tx=0).stop 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.882+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 msgr2=0x7f95b41978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.882+0000 7f95ba400640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b41978d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005ab0 tx=0x7f95a802c6c0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 shutdown_connections 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f959c0a2fb0 0x7f959c0a5470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b4101fa0 0x7f95b419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b41015e0 0x7f95b4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 --2- 172.21.15.37:0/1217250946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b41003f0 0x7f95b41978d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 >> 172.21.15.37:0/1217250946 conn(0x7f95b40fb980 msgr2=0x7f95b4069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 shutdown_connections 2024-03-21T09:11:51.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:51.883+0000 7f95ba400640 1 -- 172.21.15.37:0/1217250946 wait complete. 2024-03-21T09:11:51.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 -- 172.21.15.37:0/2745628926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 msgr2=0x7f96b0104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 --2- 172.21.15.37:0/2745628926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0104660 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009900 tx=0x7f96a002bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 -- 172.21.15.37:0/2745628926 shutdown_connections 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 --2- 172.21.15.37:0/2745628926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96b01074d0 0x7f96b01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 --2- 172.21.15.37:0/2745628926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96b0104ba0 0x7f96b0106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 --2- 172.21.15.37:0/2745628926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0104660 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 -- 172.21.15.37:0/2745628926 >> 172.21.15.37:0/2745628926 conn(0x7f96b00fbb80 msgr2=0x7f96b00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 -- 172.21.15.37:0/2745628926 shutdown_connections 2024-03-21T09:11:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.122+0000 7f96b4e00640 1 -- 172.21.15.37:0/2745628926 wait complete. 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.123+0000 7f96b4e00640 1 Processor -- start 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.123+0000 7f96b4e00640 1 -- start start 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96b4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96b4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96b0104ba0 0x7f96b0100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96b4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96b01074d0 0x7f96b0100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96b4e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b010d390 con 0x7f96b01074d0 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96b4e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b010d210 con 0x7f96b0104ba0 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96b4e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b010d510 con 0x7f96b0102270 2024-03-21T09:11:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ad400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96b0104ba0 0x7f96b0100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:36072/0 (socket says 172.21.15.37:36072) 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 -- 172.21.15.37:0/1556069046 learned_addr learned my addr 172.21.15.37:0/1556069046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ae800640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96b01074d0 0x7f96b0100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 -- 172.21.15.37:0/1556069046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96b0104ba0 msgr2=0x7f96b0100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96b0104ba0 0x7f96b0100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 -- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96b01074d0 msgr2=0x7f96b0100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.124+0000 7f96ade00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96b01074d0 0x7f96b0100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96ade00640 1 -- 172.21.15.37:0/1556069046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0100ca0 con 0x7f96b0102270 2024-03-21T09:11:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96ade00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0101be0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96a002c8f0 tx=0x7f96a0005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0041070 con 0x7f96b0102270 2024-03-21T09:11:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a00054e0 con 0x7f96b0102270 2024-03-21T09:11:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0034440 con 0x7f96b0102270 2024-03-21T09:11:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b0100f30 con 0x7f96b0102270 2024-03-21T09:11:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.125+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b01a6400 con 0x7f96b0102270 2024-03-21T09:11:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.126+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b0068970 con 0x7f96b0102270 2024-03-21T09:11:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.128+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f96a0044070 con 0x7f96b0102270 2024-03-21T09:11:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.129+0000 7f96a6a00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f96940a3030 0x7f96940a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.129+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f96a00f1420 con 0x7f96b0102270 2024-03-21T09:11:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.130+0000 7f96ad400640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f96940a3030 0x7f96940a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.130+0000 7f96ad400640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f96940a3030 0x7f96940a54f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f96b0101950 tx=0x7f969c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:52.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.133+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96a00bd7c0 con 0x7f96b0102270 2024-03-21T09:11:52.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:51 smithi037 ceph-mon[31830]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-21T09:11:52.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:51 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4203277452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:52.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:51 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1217250946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:52.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.340+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96b002d3e0 con 0x7f96b0102270 2024-03-21T09:11:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.342+0000 7f96a6a00640 1 -- 172.21.15.37:0/1556069046 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7f96a00c1860 con 0x7f96b0102270 2024-03-21T09:11:52.336 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:52.336 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:52.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f96940a3030 msgr2=0x7f96940a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f96940a3030 0x7f96940a54f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f96b0101950 tx=0x7f969c00b410 comp rx=0 tx=0).stop 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 msgr2=0x7f96b0101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0101be0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96a002c8f0 tx=0x7f96a0005e40 comp rx=0 tx=0).stop 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 shutdown_connections 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f96940a3030 0x7f96940a54f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96b01074d0 0x7f96b0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96b0104ba0 0x7f96b0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 --2- 172.21.15.37:0/1556069046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96b0102270 0x7f96b0101be0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.345+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 >> 172.21.15.37:0/1556069046 conn(0x7f96b00fbb80 msgr2=0x7f96b0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.346+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 shutdown_connections 2024-03-21T09:11:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.346+0000 7f96b4e00640 1 -- 172.21.15.37:0/1556069046 wait complete. 2024-03-21T09:11:52.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:51 smithi071 ceph-mon[36597]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-21T09:11:52.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:51 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4203277452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:52.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:51 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1217250946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:52.358 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:11:52.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:51 smithi053 ceph-mon[36169]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-21T09:11:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:51 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4203277452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:51 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1217250946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.579+0000 7fdc17800640 1 -- 172.21.15.37:0/425664002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 msgr2=0x7fdc10103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.579+0000 7fdc17800640 1 --2- 172.21.15.37:0/425664002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10103ce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc0402bc40 comp rx=0 tx=0).stop 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 -- 172.21.15.37:0/425664002 shutdown_connections 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 --2- 172.21.15.37:0/425664002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc10105470 0x7fdc1010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 --2- 172.21.15.37:0/425664002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 0x7fdc10104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 --2- 172.21.15.37:0/425664002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10103ce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 -- 172.21.15.37:0/425664002 >> 172.21.15.37:0/425664002 conn(0x7fdc100fee50 msgr2=0x7fdc10101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 -- 172.21.15.37:0/425664002 shutdown_connections 2024-03-21T09:11:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.580+0000 7fdc17800640 1 -- 172.21.15.37:0/425664002 wait complete. 2024-03-21T09:11:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 Processor -- start 2024-03-21T09:11:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 -- start start 2024-03-21T09:11:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 0x7fdc1019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc10105470 0x7fdc101a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1010f580 con 0x7fdc10105470 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1010f400 con 0x7fdc10104ab0 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc17800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1010f700 con 0x7fdc101038c0 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc14e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc0fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 0x7fdc1019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc14e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:36096/0 (socket says 172.21.15.37:36096) 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.581+0000 7fdc14e00640 1 -- 172.21.15.37:0/1717047953 learned_addr learned my addr 172.21.15.37:0/1717047953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc15800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc10105470 0x7fdc101a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0fe00640 1 -- 172.21.15.37:0/1717047953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 msgr2=0x7fdc10199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0fe00640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0fe00640 1 -- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc10105470 msgr2=0x7fdc101a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0fe00640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc10105470 0x7fdc101a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0fe00640 1 -- 172.21.15.37:0/1717047953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc101a1840 con 0x7fdc10104ab0 2024-03-21T09:11:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0fe00640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 0x7fdc1019a170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e9f0 tx=0x7fdbfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00ccc0 con 0x7fdc10104ab0 2024-03-21T09:11:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbfc005020 con 0x7fdc10104ab0 2024-03-21T09:11:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc101a1b30 con 0x7fdc10104ab0 2024-03-21T09:11:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.582+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0049e0 con 0x7fdc10104ab0 2024-03-21T09:11:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.583+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc101a2050 con 0x7fdc10104ab0 2024-03-21T09:11:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.584+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1010ac80 con 0x7fdc10104ab0 2024-03-21T09:11:52.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.589+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fdbfc005560 con 0x7fdc10104ab0 2024-03-21T09:11:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.591+0000 7fdc0d600640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdbf80a3030 0x7fdbf80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.591+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdbfc0cf950 con 0x7fdc10104ab0 2024-03-21T09:11:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.592+0000 7fdc14e00640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdbf80a3030 0x7fdbf80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.593+0000 7fdc14e00640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdbf80a3030 0x7fdbf80a54f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdc04005ab0 tx=0x7fdc0402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.594+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdbfc0051d0 con 0x7fdc10104ab0 2024-03-21T09:11:52.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.796+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc1002d2c0 con 0x7fdc10104ab0 2024-03-21T09:11:52.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.797+0000 7fdc0d600640 1 -- 172.21.15.37:0/1717047953 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdbfc09c290 con 0x7fdc10104ab0 2024-03-21T09:11:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:11:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.799+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdbf80a3030 msgr2=0x7fdbf80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.799+0000 7fdc17800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdbf80a3030 0x7fdbf80a54f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdc04005ab0 tx=0x7fdc0402cf90 comp rx=0 tx=0).stop 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.799+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 msgr2=0x7fdc1019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.799+0000 7fdc17800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 0x7fdc1019a170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e9f0 tx=0x7fdbfc00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 shutdown_connections 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fdbf80a3030 0x7fdbf80a54f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc10105470 0x7fdc101a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc10104ab0 0x7fdc1019a170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 --2- 172.21.15.37:0/1717047953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc101038c0 0x7fdc10199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 >> 172.21.15.37:0/1717047953 conn(0x7fdc100fee50 msgr2=0x7fdc10107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 shutdown_connections 2024-03-21T09:11:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:52.800+0000 7fdc17800640 1 -- 172.21.15.37:0/1717047953 wait complete. 2024-03-21T09:11:52.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 -- 172.21.15.37:0/1452471467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc41015e0 msgr2=0x7f5cc4101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/1452471467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc41015e0 0x7f5cc4101a60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009900 tx=0x7f5cb002bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 -- 172.21.15.37:0/1452471467 shutdown_connections 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/1452471467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc4101fa0 0x7f5cc4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/1452471467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc41015e0 0x7f5cc4101a60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/1452471467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 0x7f5cc4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 -- 172.21.15.37:0/1452471467 >> 172.21.15.37:0/1452471467 conn(0x7f5cc40fb980 msgr2=0x7f5cc40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 -- 172.21.15.37:0/1452471467 shutdown_connections 2024-03-21T09:11:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.044+0000 7f5cc3e00640 1 -- 172.21.15.37:0/1452471467 wait complete. 2024-03-21T09:11:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 Processor -- start 2024-03-21T09:11:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 -- start start 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 0x7f5cc4104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 0x7f5cc41029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc4101fa0 0x7f5cc4102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410c0d0 con 0x7f5cc41003f0 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410bf50 con 0x7f5cc41015e0 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.045+0000 7f5cc3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410c250 con 0x7f5cc4101fa0 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 0x7f5cc41029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 0x7f5cc4104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 0x7f5cc41029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53534/0 (socket says 172.21.15.37:53534) 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 0x7f5cc4104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50380/0 (socket says 172.21.15.37:50380) 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc4101fa0 0x7f5cc4102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc2000640 1 -- 172.21.15.37:0/3151534782 learned_addr learned my addr 172.21.15.37:0/3151534782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc3400640 1 -- 172.21.15.37:0/3151534782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 msgr2=0x7f5cc41029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc3400640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 0x7f5cc41029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc3400640 1 -- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 msgr2=0x7f5cc4104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc3400640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 0x7f5cc4104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc3400640 1 -- 172.21.15.37:0/3151534782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4103790 con 0x7f5cc4101fa0 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.046+0000 7f5cc2000640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 0x7f5cc41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.047+0000 7f5cc3400640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc4101fa0 0x7f5cc4102f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009fd0 tx=0x7f5cbc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.047+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00ce20 con 0x7f5cc4101fa0 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.047+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc005020 con 0x7f5cc4101fa0 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.047+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc010720 con 0x7f5cc4101fa0 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.047+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc41a19a0 con 0x7f5cc4101fa0 2024-03-21T09:11:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.047+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc41a1ef0 con 0x7f5cc4101fa0 2024-03-21T09:11:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.048+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc41a2120 con 0x7f5cc4101fa0 2024-03-21T09:11:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.050+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f5cbc005540 con 0x7f5cc4101fa0 2024-03-21T09:11:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.051+0000 7f5cbb400640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5ca80a2fb0 0x7f5ca80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.051+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5cbc0cff90 con 0x7f5cc4101fa0 2024-03-21T09:11:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.051+0000 7f5cc2a00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5ca80a2fb0 0x7f5ca80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.052+0000 7f5cc2a00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5ca80a2fb0 0x7f5ca80a5470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005fd0 tx=0x7f5cb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:53.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.055+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5cbc09c870 con 0x7f5cc4101fa0 2024-03-21T09:11:53.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:53 smithi037 ceph-mon[31830]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-21T09:11:53.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:53 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1556069046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:53.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:53 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1717047953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:53.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.261+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cc41a22f0 con 0x7f5cc4101fa0 2024-03-21T09:11:53.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.262+0000 7f5cbb400640 1 -- 172.21.15.37:0/3151534782 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cbc09c170 con 0x7f5cc4101fa0 2024-03-21T09:11:53.256 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:53.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.264+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5ca80a2fb0 msgr2=0x7f5ca80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.264+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5ca80a2fb0 0x7f5ca80a5470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005fd0 tx=0x7f5cb400b410 comp rx=0 tx=0).stop 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.264+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc4101fa0 msgr2=0x7f5cc4102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.264+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc4101fa0 0x7f5cc4102f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009fd0 tx=0x7f5cbc00ef10 comp rx=0 tx=0).stop 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 shutdown_connections 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f5ca80a2fb0 0x7f5ca80a5470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cc4101fa0 0x7f5cc4102f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cc41015e0 0x7f5cc41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 --2- 172.21.15.37:0/3151534782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc41003f0 0x7f5cc4104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 >> 172.21.15.37:0/3151534782 conn(0x7f5cc40fb980 msgr2=0x7f5cc40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 shutdown_connections 2024-03-21T09:11:53.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.265+0000 7f5cc3e00640 1 -- 172.21.15.37:0/3151534782 wait complete. 2024-03-21T09:11:53.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:53.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:53 smithi071 ceph-mon[36597]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-21T09:11:53.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:53 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1556069046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:53.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:53 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1717047953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:53.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:53 smithi053 ceph-mon[36169]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-21T09:11:53.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:53 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1556069046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:53.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:53 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1717047953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:53.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 -- 172.21.15.37:0/2968545752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1052b0 msgr2=0x7f400c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 --2- 172.21.15.37:0/2968545752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1052b0 0x7f400c10bb40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4004009ca0 tx=0x7f400402c100 comp rx=0 tx=0).stop 2024-03-21T09:11:53.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 -- 172.21.15.37:0/2968545752 shutdown_connections 2024-03-21T09:11:53.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 --2- 172.21.15.37:0/2968545752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1052b0 0x7f400c10bb40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 --2- 172.21.15.37:0/2968545752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f400c1048f0 0x7f400c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 --2- 172.21.15.37:0/2968545752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f400c103700 0x7f400c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 -- 172.21.15.37:0/2968545752 >> 172.21.15.37:0/2968545752 conn(0x7f400c0fec70 msgr2=0x7f400c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 -- 172.21.15.37:0/2968545752 shutdown_connections 2024-03-21T09:11:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.533+0000 7f4012a00640 1 -- 172.21.15.37:0/2968545752 wait complete. 2024-03-21T09:11:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.534+0000 7f4012a00640 1 Processor -- start 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.534+0000 7f4012a00640 1 -- start start 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f400c103700 0x7f400c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 0x7f400c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f400c1052b0 0x7f400c1a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c10f380 con 0x7f400c103700 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c10f200 con 0x7f400c1052b0 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c10f500 con 0x7f400c1048f0 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 0x7f400c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4011600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f400c103700 0x7f400c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 0x7f400c199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:36144/0 (socket says 172.21.15.37:36144) 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 -- 172.21.15.37:0/169648339 learned_addr learned my addr 172.21.15.37:0/169648339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4012000640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f400c1052b0 0x7f400c1a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 -- 172.21.15.37:0/169648339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f400c1052b0 msgr2=0x7f400c1a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f400c1052b0 0x7f400c1a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 -- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f400c103700 msgr2=0x7f400c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f400c103700 0x7f400c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.535+0000 7f4010c00640 1 -- 172.21.15.37:0/169648339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c1a15b0 con 0x7f400c1048f0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.536+0000 7f4010c00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 0x7f400c199ed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800e9f0 tx=0x7f3ff800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.536+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800cd80 con 0x7f400c1048f0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.536+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ff8005500 con 0x7f400c1048f0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.536+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c1a18a0 con 0x7f400c1048f0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.536+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8010690 con 0x7f400c1048f0 2024-03-21T09:11:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.536+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1a1df0 con 0x7f400c1048f0 2024-03-21T09:11:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.538+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c10aa20 con 0x7f400c1048f0 2024-03-21T09:11:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.540+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3ff80046c0 con 0x7f400c1048f0 2024-03-21T09:11:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.541+0000 7f400a000640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ff40a3030 0x7f3ff40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.541+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3ff80cfce0 con 0x7f400c1048f0 2024-03-21T09:11:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.541+0000 7f4011600640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ff40a3030 0x7f3ff40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.542+0000 7f4011600640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ff40a3030 0x7f3ff40a54f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f400c104750 tx=0x7f4000005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.545+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ff809c5c0 con 0x7f400c1048f0 2024-03-21T09:11:53.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.749+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f400c02d070 con 0x7f400c1048f0 2024-03-21T09:11:53.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.750+0000 7f400a000640 1 -- 172.21.15.37:0/169648339 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ff809bec0 con 0x7f400c1048f0 2024-03-21T09:11:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.752+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ff40a3030 msgr2=0x7f3ff40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.752+0000 7f4012a00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ff40a3030 0x7f3ff40a54f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f400c104750 tx=0x7f4000005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.752+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 msgr2=0x7f400c199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.752+0000 7f4012a00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 0x7f400c199ed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800e9f0 tx=0x7f3ff800edf0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 shutdown_connections 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f3ff40a3030 0x7f3ff40a54f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f400c1052b0 0x7f400c1a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f400c1048f0 0x7f400c199ed0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 --2- 172.21.15.37:0/169648339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f400c103700 0x7f400c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 >> 172.21.15.37:0/169648339 conn(0x7f400c0fec70 msgr2=0x7f400c107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 shutdown_connections 2024-03-21T09:11:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:53.753+0000 7f4012a00640 1 -- 172.21.15.37:0/169648339 wait complete. 2024-03-21T09:11:53.766 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.000+0000 7f7f73e00640 1 -- 172.21.15.37:0/3776366330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 msgr2=0x7f7f74100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.000+0000 7f7f73e00640 1 --2- 172.21.15.37:0/3776366330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74100b60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009900 tx=0x7f7f5c02bc50 comp rx=0 tx=0).stop 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 -- 172.21.15.37:0/3776366330 shutdown_connections 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 --2- 172.21.15.37:0/3776366330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f74101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 --2- 172.21.15.37:0/3776366330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74100b60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 --2- 172.21.15.37:0/3776366330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f740ffda0 0x7f7f741001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 -- 172.21.15.37:0/3776366330 >> 172.21.15.37:0/3776366330 conn(0x7f7f740fb980 msgr2=0x7f7f740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 -- 172.21.15.37:0/3776366330 shutdown_connections 2024-03-21T09:11:53.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.001+0000 7f7f73e00640 1 -- 172.21.15.37:0/3776366330 wait complete. 2024-03-21T09:11:53.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 Processor -- start 2024-03-21T09:11:53.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 -- start start 2024-03-21T09:11:53.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f740ffda0 0x7f7f74199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f741a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7410f2d0 con 0x7f7f740ffda0 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7410f150 con 0x7f7f741010a0 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.002+0000 7f7f73e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7410f450 con 0x7f7f74100700 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f72000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f72a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f740ffda0 0x7f7f74199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f741a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f72000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57094/0 (socket says 172.21.15.37:57094) 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f72000640 1 -- 172.21.15.37:0/2088413331 learned_addr learned my addr 172.21.15.37:0/2088413331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f741a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50352/0 (socket says 172.21.15.37:50352) 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 -- 172.21.15.37:0/2088413331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 msgr2=0x7f7f74199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 -- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f740ffda0 msgr2=0x7f7f74199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f740ffda0 0x7f7f74199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 -- 172.21.15.37:0/2088413331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f741a1660 con 0x7f7f741010a0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.003+0000 7f7f73400640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f741a0ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e970 tx=0x7f7f6c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.004+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00ccc0 con 0x7f7f741010a0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.004+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f6c0035f0 con 0x7f7f741010a0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.004+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00ce60 con 0x7f7f741010a0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.004+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f741a1950 con 0x7f7f741010a0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.004+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f741a1ea0 con 0x7f7f741010a0 2024-03-21T09:11:53.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.005+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7410aaf0 con 0x7f7f741010a0 2024-03-21T09:11:54.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.007+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7f6c0037a0 con 0x7f7f741010a0 2024-03-21T09:11:54.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.008+0000 7f7f6b400640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f580a3030 0x7f7f580a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.009+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f6c0cfb10 con 0x7f7f741010a0 2024-03-21T09:11:54.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.009+0000 7f7f72a00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f580a3030 0x7f7f580a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.010+0000 7f7f72a00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f580a3030 0x7f7f580a54f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f64009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:54.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.013+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f6c09c3f0 con 0x7f7f741010a0 2024-03-21T09:11:54.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:54 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3151534782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:54.173 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:54 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/169648339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.217+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7f7402d220 con 0x7f7f741010a0 2024-03-21T09:11:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.217+0000 7f7f6b400640 1 -- 172.21.15.37:0/2088413331 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7f6c09bcf0 con 0x7f7f741010a0 2024-03-21T09:11:54.211 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f580a3030 msgr2=0x7f7f580a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f580a3030 0x7f7f580a54f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f64009500 comp rx=0 tx=0).stop 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 msgr2=0x7f7f741a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f741a0ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e970 tx=0x7f7f6c00ee30 comp rx=0 tx=0).stop 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 shutdown_connections 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f580a3030 0x7f7f580a54f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f741010a0 0x7f7f741a0ff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f74100700 0x7f7f74199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2088413331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f740ffda0 0x7f7f74199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.220+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 >> 172.21.15.37:0/2088413331 conn(0x7f7f740fb980 msgr2=0x7f7f740fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:54.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.221+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 shutdown_connections 2024-03-21T09:11:54.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.221+0000 7f7f73e00640 1 -- 172.21.15.37:0/2088413331 wait complete. 2024-03-21T09:11:54.237 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-21T09:11:54.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:54.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:54 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3151534782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:54.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:54 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/169648339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:54 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3151534782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:54.469 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:54 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/169648339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 -- 172.21.15.37:0/3225496672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 msgr2=0x7f7f7c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 --2- 172.21.15.37:0/3225496672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c074500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009900 tx=0x7f7f6402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 -- 172.21.15.37:0/3225496672 shutdown_connections 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 --2- 172.21.15.37:0/3225496672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f7c109450 0x7f7f7c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 --2- 172.21.15.37:0/3225496672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c074500 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 --2- 172.21.15.37:0/3225496672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c075680 0x7f7f7c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 -- 172.21.15.37:0/3225496672 >> 172.21.15.37:0/3225496672 conn(0x7f7f7c0fcb30 msgr2=0x7f7f7c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 -- 172.21.15.37:0/3225496672 shutdown_connections 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.456+0000 7f7f7be00640 1 -- 172.21.15.37:0/3225496672 wait complete. 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.457+0000 7f7f7be00640 1 Processor -- start 2024-03-21T09:11:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.457+0000 7f7f7be00640 1 -- start start 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.457+0000 7f7f7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.457+0000 7f7f7be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f7c075680 0x7f7f7c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.457+0000 7f7f7be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c109450 0x7f7f7c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c10f440 con 0x7f7f7c109450 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c10f2c0 con 0x7f7f7c075680 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c10f5c0 con 0x7f7f7c0740a0 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57106/0 (socket says 172.21.15.37:57106) 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 -- 172.21.15.37:0/2234939561 learned_addr learned my addr 172.21.15.37:0/2234939561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f72000640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f7c075680 0x7f7f7c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7b400640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c109450 0x7f7f7c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 -- 172.21.15.37:0/2234939561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f7c075680 msgr2=0x7f7f7c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f7c075680 0x7f7f7c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 -- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c109450 msgr2=0x7f7f7c1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c109450 0x7f7f7c1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.458+0000 7f7f7aa00640 1 -- 172.21.15.37:0/2234939561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c1a1680 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.459+0000 7f7f7aa00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c199a60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00db80 tx=0x7f7f6c00b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.459+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c017070 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.459+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f6c003b70 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.459+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0106c0 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.459+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c1a1970 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.459+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c1a1e90 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.460+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c108210 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.462+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7f6c004020 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.463+0000 7f7f73e00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f600a2fa0 0x7f7f600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.463+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f6c0cf6e0 con 0x7f7f7c0740a0 2024-03-21T09:11:54.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.464+0000 7f7f72000640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f600a2fa0 0x7f7f600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.464+0000 7f7f72000640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f600a2fa0 0x7f7f600a5460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f6402c850 tx=0x7f7f64000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:54.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.467+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f6c020070 con 0x7f7f7c0740a0 2024-03-21T09:11:54.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.712+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f7c02d070 con 0x7f7f7c0740a0 2024-03-21T09:11:54.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.713+0000 7f7f73e00640 1 -- 172.21.15.37:0/2234939561 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f7f6c09bfc0 con 0x7f7f7c0740a0 2024-03-21T09:11:54.707 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:54.707 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:11:48.827236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:11:54.707 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f600a2fa0 msgr2=0x7f7f600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f600a2fa0 0x7f7f600a5460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f6402c850 tx=0x7f7f64000bc0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 msgr2=0x7f7f7c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c199a60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00db80 tx=0x7f7f6c00b630 comp rx=0 tx=0).stop 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 shutdown_connections 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f7f600a2fa0 0x7f7f600a5460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f7c109450 0x7f7f7c1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f7c075680 0x7f7f7c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 --2- 172.21.15.37:0/2234939561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7c0740a0 0x7f7f7c199a60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 >> 172.21.15.37:0/2234939561 conn(0x7f7f7c0fcb30 msgr2=0x7f7f7c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 shutdown_connections 2024-03-21T09:11:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.716+0000 7f7f7be00640 1 -- 172.21.15.37:0/2234939561 wait complete. 2024-03-21T09:11:54.729 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-21T09:11:54.729 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-21T09:11:54.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 -- 172.21.15.37:0/1603031769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 msgr2=0x7f50a8100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 --2- 172.21.15.37:0/1603031769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a8100d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5094009900 tx=0x7f509402bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 -- 172.21.15.37:0/1603031769 shutdown_connections 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 --2- 172.21.15.37:0/1603031769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a8101240 0x7f50a81016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 --2- 172.21.15.37:0/1603031769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a8100d00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 --2- 172.21.15.37:0/1603031769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50a80fff40 0x7f50a8100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 -- 172.21.15.37:0/1603031769 >> 172.21.15.37:0/1603031769 conn(0x7f50a80fbb40 msgr2=0x7f50a80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 -- 172.21.15.37:0/1603031769 shutdown_connections 2024-03-21T09:11:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.948+0000 7f50aea00640 1 -- 172.21.15.37:0/1603031769 wait complete. 2024-03-21T09:11:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 Processor -- start 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 -- start start 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a80fff40 0x7f50a8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50a8101240 0x7f50a81a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a810f580 con 0x7f50a80fff40 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a810f400 con 0x7f50a81008a0 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.949+0000 7f50aea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a810f700 con 0x7f50a8101240 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a819a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50386/0 (socket says 172.21.15.37:50386) 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 -- 172.21.15.37:0/3850241548 learned_addr learned my addr 172.21.15.37:0/3850241548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a80fff40 0x7f50a8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50aca00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50a8101240 0x7f50a81a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 -- 172.21.15.37:0/3850241548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50a8101240 msgr2=0x7f50a81a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50a8101240 0x7f50a81a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 -- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a80fff40 msgr2=0x7f50a8199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a80fff40 0x7f50a8199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 -- 172.21.15.37:0/3850241548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a81a17e0 con 0x7f50a81008a0 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.950+0000 7f50a7400640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a819a100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5094000c00 tx=0x7f509402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.951+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094043020 con 0x7f50a81008a0 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.951+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50940054e0 con 0x7f50a81008a0 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.951+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a81a1a70 con 0x7f50a81008a0 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.951+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094003bd0 con 0x7f50a81008a0 2024-03-21T09:11:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.951+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a81a2010 con 0x7f50a81008a0 2024-03-21T09:11:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.952+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a810ae20 con 0x7f50a81008a0 2024-03-21T09:11:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.954+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f5094046020 con 0x7f50a81008a0 2024-03-21T09:11:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.955+0000 7f50a4c00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f508c0a3030 0x7f508c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.955+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50940f1ac0 con 0x7f50a81008a0 2024-03-21T09:11:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.956+0000 7f50a7e00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f508c0a3030 0x7f508c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.956+0000 7f50a7e00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f508c0a3030 0x7f508c0a54f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5098009bd0 tx=0x7f5098005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:54.958+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f509404d020 con 0x7f50a81008a0 2024-03-21T09:11:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.165+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50a8061ae0 con 0x7f508c0a3030 2024-03-21T09:11:55.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.185+0000 7f50a4c00640 1 -- 172.21.15.37:0/3850241548 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340812 (secure 0 0 0) 0x7f50a8061ae0 con 0x7f508c0a3030 2024-03-21T09:11:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:55.186 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:11:55.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f508c0a3030 msgr2=0x7f508c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f508c0a3030 0x7f508c0a54f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5098009bd0 tx=0x7f5098005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 msgr2=0x7f50a819a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a819a100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5094000c00 tx=0x7f509402c650 comp rx=0 tx=0).stop 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 shutdown_connections 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f508c0a3030 0x7f508c0a54f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50a8101240 0x7f50a81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a81008a0 0x7f50a819a100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 --2- 172.21.15.37:0/3850241548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a80fff40 0x7f50a8199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.190+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 >> 172.21.15.37:0/3850241548 conn(0x7f50a80fbb40 msgr2=0x7f50a80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.191+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 shutdown_connections 2024-03-21T09:11:55.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.191+0000 7f50aea00640 1 -- 172.21.15.37:0/3850241548 wait complete. 2024-03-21T09:11:55.203 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-03-21T09:11:53.849853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666352,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1080,"num_read_kb":672,"num_write":360,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":792,"ondisk_log_size":792,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382592,"kb_used_data":12592,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559232,"statfs":{"total":1151940427776,"available":1151548653568,"internally_reserved":0,"allocated":12894208,"data_stored":8276007,"data_compressed":433446,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9287,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":56,"num_write":9,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004892"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404943+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:10:47.404943+0000","last_peered":"2024-03-21T09:10:47.404943+0000","last_clean":"2024-03-21T09:10:47.404943+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:10:47.404943+0000","last_undegraded":"2024-03-21T09:10:47.404943+0000","last_fullsized":"2024-03-21T09:10:47.404943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438508+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:10:47.438508+0000","last_peered":"2024-03-21T09:10:47.438508+0000","last_clean":"2024-03-21T09:10:47.438508+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:10:47.438508+0000","last_undegraded":"2024-03-21T09:10:47.438508+0000","last_fullsized":"2024-03-21T09:10:47.438508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437937+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:10:47.437937+0000","last_peered":"2024-03-21T09:10:47.437937+0000","last_clean":"2024-03-21T09:10:47.437937+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:10:47.437937+0000","last_undegraded":"2024-03-21T09:10:47.437937+0000","last_fullsized":"2024-03-21T09:10:47.437937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409123+0000","last_change":"2024-03-21T09:09:47.203174+0000","last_active":"2024-03-21T09:10:47.409123+0000","last_peered":"2024-03-21T09:10:47.409123+0000","last_clean":"2024-03-21T09:10:47.409123+0000","last_became_active":"2024-03-21T09:09:47.202519+0000","last_became_peered":"2024-03-21T09:09:47.202519+0000","last_unstale":"2024-03-21T09:10:47.409123+0000","last_undegraded":"2024-03-21T09:10:47.409123+0000","last_fullsized":"2024-03-21T09:10:47.409123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:48.445311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408507+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:10:47.408507+0000","last_peered":"2024-03-21T09:10:47.408507+0000","last_clean":"2024-03-21T09:10:47.408507+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:10:47.408507+0000","last_undegraded":"2024-03-21T09:10:47.408507+0000","last_fullsized":"2024-03-21T09:10:47.408507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402461+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:10:47.402461+0000","last_peered":"2024-03-21T09:10:47.402461+0000","last_clean":"2024-03-21T09:10:47.402461+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:10:47.402461+0000","last_undegraded":"2024-03-21T09:10:47.402461+0000","last_fullsized":"2024-03-21T09:10:47.402461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436131+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:10:47.436131+0000","last_peered":"2024-03-21T09:10:47.436131+0000","last_clean":"2024-03-21T09:10:47.436131+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:10:47.436131+0000","last_undegraded":"2024-03-21T09:10:47.436131+0000","last_fullsized":"2024-03-21T09:10:47.436131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406994+0000","last_change":"2024-03-21T09:09:47.204487+0000","last_active":"2024-03-21T09:10:47.406994+0000","last_peered":"2024-03-21T09:10:47.406994+0000","last_clean":"2024-03-21T09:10:47.406994+0000","last_became_active":"2024-03-21T09:09:47.202961+0000","last_became_peered":"2024-03-21T09:09:47.202961+0000","last_unstale":"2024-03-21T09:10:47.406994+0000","last_undegraded":"2024-03-21T09:10:47.406994+0000","last_fullsized":"2024-03-21T09:10:47.406994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:50.524550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406889+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:10:47.406889+0000","last_peered":"2024-03-21T09:10:47.406889+0000","last_clean":"2024-03-21T09:10:47.406889+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:10:47.406889+0000","last_undegraded":"2024-03-21T09:10:47.406889+0000","last_fullsized":"2024-03-21T09:10:47.406889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402825+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:10:47.402825+0000","last_peered":"2024-03-21T09:10:47.402825+0000","last_clean":"2024-03-21T09:10:47.402825+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:10:47.402825+0000","last_undegraded":"2024-03-21T09:10:47.402825+0000","last_fullsized":"2024-03-21T09:10:47.402825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436663+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:10:47.436663+0000","last_peered":"2024-03-21T09:10:47.436663+0000","last_clean":"2024-03-21T09:10:47.436663+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:10:47.436663+0000","last_undegraded":"2024-03-21T09:10:47.436663+0000","last_fullsized":"2024-03-21T09:10:47.436663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436967+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:10:47.436967+0000","last_peered":"2024-03-21T09:10:47.436967+0000","last_clean":"2024-03-21T09:10:47.436967+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:10:47.436967+0000","last_undegraded":"2024-03-21T09:10:47.436967+0000","last_fullsized":"2024-03-21T09:10:47.436967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439147+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:10:47.439147+0000","last_peered":"2024-03-21T09:10:47.439147+0000","last_clean":"2024-03-21T09:10:47.439147+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:10:47.439147+0000","last_undegraded":"2024-03-21T09:10:47.439147+0000","last_fullsized":"2024-03-21T09:10:47.439147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437049+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:10:47.437049+0000","last_peered":"2024-03-21T09:10:47.437049+0000","last_clean":"2024-03-21T09:10:47.437049+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:10:47.437049+0000","last_undegraded":"2024-03-21T09:10:47.437049+0000","last_fullsized":"2024-03-21T09:10:47.437049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437308+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:10:47.437308+0000","last_peered":"2024-03-21T09:10:47.437308+0000","last_clean":"2024-03-21T09:10:47.437308+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:10:47.437308+0000","last_undegraded":"2024-03-21T09:10:47.437308+0000","last_fullsized":"2024-03-21T09:10:47.437308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406169+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:10:47.406169+0000","last_peered":"2024-03-21T09:10:47.406169+0000","last_clean":"2024-03-21T09:10:47.406169+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:10:47.406169+0000","last_undegraded":"2024-03-21T09:10:47.406169+0000","last_fullsized":"2024-03-21T09:10:47.406169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404683+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:10:47.404683+0000","last_peered":"2024-03-21T09:10:47.404683+0000","last_clean":"2024-03-21T09:10:47.404683+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:10:47.404683+0000","last_undegraded":"2024-03-21T09:10:47.404683+0000","last_fullsized":"2024-03-21T09:10:47.404683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408836+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:10:47.408836+0000","last_peered":"2024-03-21T09:10:47.408836+0000","last_clean":"2024-03-21T09:10:47.408836+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:10:47.408836+0000","last_undegraded":"2024-03-21T09:10:47.408836+0000","last_fullsized":"2024-03-21T09:10:47.408836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406741+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:10:47.406741+0000","last_peered":"2024-03-21T09:10:47.406741+0000","last_clean":"2024-03-21T09:10:47.406741+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:10:47.406741+0000","last_undegraded":"2024-03-21T09:10:47.406741+0000","last_fullsized":"2024-03-21T09:10:47.406741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406108+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:10:47.406108+0000","last_peered":"2024-03-21T09:10:47.406108+0000","last_clean":"2024-03-21T09:10:47.406108+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:10:47.406108+0000","last_undegraded":"2024-03-21T09:10:47.406108+0000","last_fullsized":"2024-03-21T09:10:47.406108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408966+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:10:47.408966+0000","last_peered":"2024-03-21T09:10:47.408966+0000","last_clean":"2024-03-21T09:10:47.408966+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:10:47.408966+0000","last_undegraded":"2024-03-21T09:10:47.408966+0000","last_fullsized":"2024-03-21T09:10:47.408966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411324+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:10:47.411324+0000","last_peered":"2024-03-21T09:10:47.411324+0000","last_clean":"2024-03-21T09:10:47.411324+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:10:47.411324+0000","last_undegraded":"2024-03-21T09:10:47.411324+0000","last_fullsized":"2024-03-21T09:10:47.411324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435847+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:10:47.435847+0000","last_peered":"2024-03-21T09:10:47.435847+0000","last_clean":"2024-03-21T09:10:47.435847+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:10:47.435847+0000","last_undegraded":"2024-03-21T09:10:47.435847+0000","last_fullsized":"2024-03-21T09:10:47.435847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411396+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:10:47.411396+0000","last_peered":"2024-03-21T09:10:47.411396+0000","last_clean":"2024-03-21T09:10:47.411396+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:10:47.411396+0000","last_undegraded":"2024-03-21T09:10:47.411396+0000","last_fullsized":"2024-03-21T09:10:47.411396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439238+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:10:47.439238+0000","last_peered":"2024-03-21T09:10:47.439238+0000","last_clean":"2024-03-21T09:10:47.439238+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:10:47.439238+0000","last_undegraded":"2024-03-21T09:10:47.439238+0000","last_fullsized":"2024-03-21T09:10:47.439238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436274+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:10:47.436274+0000","last_peered":"2024-03-21T09:10:47.436274+0000","last_clean":"2024-03-21T09:10:47.436274+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:10:47.436274+0000","last_undegraded":"2024-03-21T09:10:47.436274+0000","last_fullsized":"2024-03-21T09:10:47.436274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409035+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:10:47.409035+0000","last_peered":"2024-03-21T09:10:47.409035+0000","last_clean":"2024-03-21T09:10:47.409035+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:10:47.409035+0000","last_undegraded":"2024-03-21T09:10:47.409035+0000","last_fullsized":"2024-03-21T09:10:47.409035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406668+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:10:47.406668+0000","last_peered":"2024-03-21T09:10:47.406668+0000","last_clean":"2024-03-21T09:10:47.406668+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:10:47.406668+0000","last_undegraded":"2024-03-21T09:10:47.406668+0000","last_fullsized":"2024-03-21T09:10:47.406668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436234+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:10:47.436234+0000","last_peered":"2024-03-21T09:10:47.436234+0000","last_clean":"2024-03-21T09:10:47.436234+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:10:47.436234+0000","last_undegraded":"2024-03-21T09:10:47.436234+0000","last_fullsized":"2024-03-21T09:10:47.436234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411729+0000","last_change":"2024-03-21T09:10:47.411335+0000","last_active":"2024-03-21T09:10:47.411729+0000","last_peered":"2024-03-21T09:10:47.411729+0000","last_clean":"2024-03-21T09:10:47.411729+0000","last_became_active":"2024-03-21T09:10:47.408103+0000","last_became_peered":"2024-03-21T09:10:47.408103+0000","last_unstale":"2024-03-21T09:10:47.411729+0000","last_undegraded":"2024-03-21T09:10:47.411729+0000","last_fullsized":"2024-03-21T09:10:47.411729+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:51.600076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407587+0000","last_change":"2024-03-21T09:09:47.433837+0000","last_active":"2024-03-21T09:10:47.407587+0000","last_peered":"2024-03-21T09:10:47.407587+0000","last_clean":"2024-03-21T09:10:47.407587+0000","last_became_active":"2024-03-21T09:09:47.433538+0000","last_became_peered":"2024-03-21T09:09:47.433538+0000","last_unstale":"2024-03-21T09:10:47.407587+0000","last_undegraded":"2024-03-21T09:10:47.407587+0000","last_fullsized":"2024-03-21T09:10:47.407587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:41.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406895+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:10:47.406895+0000","last_peered":"2024-03-21T09:10:47.406895+0000","last_clean":"2024-03-21T09:10:47.406895+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:10:47.406895+0000","last_undegraded":"2024-03-21T09:10:47.406895+0000","last_fullsized":"2024-03-21T09:10:47.406895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:51.961475+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:11:51.961475+0000","last_peered":"2024-03-21T09:11:51.961475+0000","last_clean":"2024-03-21T09:11:51.961475+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:11:51.961475+0000","last_undegraded":"2024-03-21T09:11:51.961475+0000","last_fullsized":"2024-03-21T09:11:51.961475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408270+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:10:47.408270+0000","last_peered":"2024-03-21T09:10:47.408270+0000","last_clean":"2024-03-21T09:10:47.408270+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:10:47.408270+0000","last_undegraded":"2024-03-21T09:10:47.408270+0000","last_fullsized":"2024-03-21T09:10:47.408270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406440+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:10:47.406440+0000","last_peered":"2024-03-21T09:10:47.406440+0000","last_clean":"2024-03-21T09:10:47.406440+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:10:47.406440+0000","last_undegraded":"2024-03-21T09:10:47.406440+0000","last_fullsized":"2024-03-21T09:10:47.406440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402428+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:10:47.402428+0000","last_peered":"2024-03-21T09:10:47.402428+0000","last_clean":"2024-03-21T09:10:47.402428+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:10:47.402428+0000","last_undegraded":"2024-03-21T09:10:47.402428+0000","last_fullsized":"2024-03-21T09:10:47.402428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407441+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:10:47.407441+0000","last_peered":"2024-03-21T09:10:47.407441+0000","last_clean":"2024-03-21T09:10:47.407441+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:10:47.407441+0000","last_undegraded":"2024-03-21T09:10:47.407441+0000","last_fullsized":"2024-03-21T09:10:47.407441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404654+0000","last_change":"2024-03-21T09:09:47.432250+0000","last_active":"2024-03-21T09:10:47.404654+0000","last_peered":"2024-03-21T09:10:47.404654+0000","last_clean":"2024-03-21T09:10:47.404654+0000","last_became_active":"2024-03-21T09:09:47.431784+0000","last_became_peered":"2024-03-21T09:09:47.431784+0000","last_unstale":"2024-03-21T09:10:47.404654+0000","last_undegraded":"2024-03-21T09:10:47.404654+0000","last_fullsized":"2024-03-21T09:10:47.404654+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:58.438472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438375+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:10:47.438375+0000","last_peered":"2024-03-21T09:10:47.438375+0000","last_clean":"2024-03-21T09:10:47.438375+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:10:47.438375+0000","last_undegraded":"2024-03-21T09:10:47.438375+0000","last_fullsized":"2024-03-21T09:10:47.438375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410399+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:10:47.410399+0000","last_peered":"2024-03-21T09:10:47.410399+0000","last_clean":"2024-03-21T09:10:47.410399+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:10:47.410399+0000","last_undegraded":"2024-03-21T09:10:47.410399+0000","last_fullsized":"2024-03-21T09:10:47.410399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438232+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:10:47.438232+0000","last_peered":"2024-03-21T09:10:47.438232+0000","last_clean":"2024-03-21T09:10:47.438232+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:10:47.438232+0000","last_undegraded":"2024-03-21T09:10:47.438232+0000","last_fullsized":"2024-03-21T09:10:47.438232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406786+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:10:47.406786+0000","last_peered":"2024-03-21T09:10:47.406786+0000","last_clean":"2024-03-21T09:10:47.406786+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:10:47.406786+0000","last_undegraded":"2024-03-21T09:10:47.406786+0000","last_fullsized":"2024-03-21T09:10:47.406786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407682+0000","last_change":"2024-03-21T09:09:47.205219+0000","last_active":"2024-03-21T09:10:47.407682+0000","last_peered":"2024-03-21T09:10:47.407682+0000","last_clean":"2024-03-21T09:10:47.407682+0000","last_became_active":"2024-03-21T09:09:47.204655+0000","last_became_peered":"2024-03-21T09:09:47.204655+0000","last_unstale":"2024-03-21T09:10:47.407682+0000","last_undegraded":"2024-03-21T09:10:47.407682+0000","last_fullsized":"2024-03-21T09:10:47.407682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:44.407879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409872+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:10:47.409872+0000","last_peered":"2024-03-21T09:10:47.409872+0000","last_clean":"2024-03-21T09:10:47.409872+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:10:47.409872+0000","last_undegraded":"2024-03-21T09:10:47.409872+0000","last_fullsized":"2024-03-21T09:10:47.409872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":186,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:50.960036+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:11:50.960036+0000","last_peered":"2024-03-21T09:11:50.960036+0000","last_clean":"2024-03-21T09:11:50.960036+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:11:50.960036+0000","last_undegraded":"2024-03-21T09:11:50.960036+0000","last_fullsized":"2024-03-21T09:11:50.960036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406846+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:10:47.406846+0000","last_peered":"2024-03-21T09:10:47.406846+0000","last_clean":"2024-03-21T09:10:47.406846+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:10:47.406846+0000","last_undegraded":"2024-03-21T09:10:47.406846+0000","last_fullsized":"2024-03-21T09:10:47.406846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407552+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:10:47.407552+0000","last_peered":"2024-03-21T09:10:47.407552+0000","last_clean":"2024-03-21T09:10:47.407552+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:10:47.407552+0000","last_undegraded":"2024-03-21T09:10:47.407552+0000","last_fullsized":"2024-03-21T09:10:47.407552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436943+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:10:47.436943+0000","last_peered":"2024-03-21T09:10:47.436943+0000","last_clean":"2024-03-21T09:10:47.436943+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:10:47.436943+0000","last_undegraded":"2024-03-21T09:10:47.436943+0000","last_fullsized":"2024-03-21T09:10:47.436943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439194+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:10:47.439194+0000","last_peered":"2024-03-21T09:10:47.439194+0000","last_clean":"2024-03-21T09:10:47.439194+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:10:47.439194+0000","last_undegraded":"2024-03-21T09:10:47.439194+0000","last_fullsized":"2024-03-21T09:10:47.439194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436421+0000","last_change":"2024-03-21T09:09:47.283764+0000","last_active":"2024-03-21T09:10:47.436421+0000","last_peered":"2024-03-21T09:10:47.436421+0000","last_clean":"2024-03-21T09:10:47.436421+0000","last_became_active":"2024-03-21T09:09:47.283203+0000","last_became_peered":"2024-03-21T09:09:47.283203+0000","last_unstale":"2024-03-21T09:10:47.436421+0000","last_undegraded":"2024-03-21T09:10:47.436421+0000","last_fullsized":"2024-03-21T09:10:47.436421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:55.502176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407507+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:10:47.407507+0000","last_peered":"2024-03-21T09:10:47.407507+0000","last_clean":"2024-03-21T09:10:47.407507+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:10:47.407507+0000","last_undegraded":"2024-03-21T09:10:47.407507+0000","last_fullsized":"2024-03-21T09:10:47.407507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406746+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:10:47.406746+0000","last_peered":"2024-03-21T09:10:47.406746+0000","last_clean":"2024-03-21T09:10:47.406746+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:10:47.406746+0000","last_undegraded":"2024-03-21T09:10:47.406746+0000","last_fullsized":"2024-03-21T09:10:47.406746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407596+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:10:47.407596+0000","last_peered":"2024-03-21T09:10:47.407596+0000","last_clean":"2024-03-21T09:10:47.407596+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:10:47.407596+0000","last_undegraded":"2024-03-21T09:10:47.407596+0000","last_fullsized":"2024-03-21T09:10:47.407596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":385,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:51.962060+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:11:51.962060+0000","last_peered":"2024-03-21T09:11:51.962060+0000","last_clean":"2024-03-21T09:11:51.962060+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:11:51.962060+0000","last_undegraded":"2024-03-21T09:11:51.962060+0000","last_fullsized":"2024-03-21T09:11:51.962060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70911,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":227,"num_write":39,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408857+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:10:47.408857+0000","last_peered":"2024-03-21T09:10:47.408857+0000","last_clean":"2024-03-21T09:10:47.408857+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:10:47.408857+0000","last_undegraded":"2024-03-21T09:10:47.408857+0000","last_fullsized":"2024-03-21T09:10:47.408857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407740+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:10:47.407740+0000","last_peered":"2024-03-21T09:10:47.407740+0000","last_clean":"2024-03-21T09:10:47.407740+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:10:47.407740+0000","last_undegraded":"2024-03-21T09:10:47.407740+0000","last_fullsized":"2024-03-21T09:10:47.407740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.420395+0000","last_change":"2024-03-21T09:10:47.420051+0000","last_active":"2024-03-21T09:10:47.420395+0000","last_peered":"2024-03-21T09:10:47.420395+0000","last_clean":"2024-03-21T09:10:47.420395+0000","last_became_active":"2024-03-21T09:10:47.412598+0000","last_became_peered":"2024-03-21T09:10:47.412598+0000","last_unstale":"2024-03-21T09:10:47.420395+0000","last_undegraded":"2024-03-21T09:10:47.420395+0000","last_fullsized":"2024-03-21T09:10:47.420395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:43.675518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438823+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:10:47.438823+0000","last_peered":"2024-03-21T09:10:47.438823+0000","last_clean":"2024-03-21T09:10:47.438823+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:10:47.438823+0000","last_undegraded":"2024-03-21T09:10:47.438823+0000","last_fullsized":"2024-03-21T09:10:47.438823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404558+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:10:47.404558+0000","last_peered":"2024-03-21T09:10:47.404558+0000","last_clean":"2024-03-21T09:10:47.404558+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:10:47.404558+0000","last_undegraded":"2024-03-21T09:10:47.404558+0000","last_fullsized":"2024-03-21T09:10:47.404558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407208+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:10:47.407208+0000","last_peered":"2024-03-21T09:10:47.407208+0000","last_clean":"2024-03-21T09:10:47.407208+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:10:47.407208+0000","last_undegraded":"2024-03-21T09:10:47.407208+0000","last_fullsized":"2024-03-21T09:10:47.407208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436363+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:10:47.436363+0000","last_peered":"2024-03-21T09:10:47.436363+0000","last_clean":"2024-03-21T09:10:47.436363+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:10:47.436363+0000","last_undegraded":"2024-03-21T09:10:47.436363+0000","last_fullsized":"2024-03-21T09:10:47.436363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408459+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:10:47.408459+0000","last_peered":"2024-03-21T09:10:47.408459+0000","last_clean":"2024-03-21T09:10:47.408459+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:10:47.408459+0000","last_undegraded":"2024-03-21T09:10:47.408459+0000","last_fullsized":"2024-03-21T09:10:47.408459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:49.038543+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:11:49.038543+0000","last_peered":"2024-03-21T09:11:49.038543+0000","last_clean":"2024-03-21T09:11:49.038543+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:11:49.038543+0000","last_undegraded":"2024-03-21T09:11:49.038543+0000","last_fullsized":"2024-03-21T09:11:49.038543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436189+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:10:47.436189+0000","last_peered":"2024-03-21T09:10:47.436189+0000","last_clean":"2024-03-21T09:10:47.436189+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:10:47.436189+0000","last_undegraded":"2024-03-21T09:10:47.436189+0000","last_fullsized":"2024-03-21T09:10:47.436189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439612+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:10:47.439612+0000","last_peered":"2024-03-21T09:10:47.439612+0000","last_clean":"2024-03-21T09:10:47.439612+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:10:47.439612+0000","last_undegraded":"2024-03-21T09:10:47.439612+0000","last_fullsized":"2024-03-21T09:10:47.439612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408028+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:10:47.408028+0000","last_peered":"2024-03-21T09:10:47.408028+0000","last_clean":"2024-03-21T09:10:47.408028+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:10:47.408028+0000","last_undegraded":"2024-03-21T09:10:47.408028+0000","last_fullsized":"2024-03-21T09:10:47.408028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406881+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:10:47.406881+0000","last_peered":"2024-03-21T09:10:47.406881+0000","last_clean":"2024-03-21T09:10:47.406881+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:10:47.406881+0000","last_undegraded":"2024-03-21T09:10:47.406881+0000","last_fullsized":"2024-03-21T09:10:47.406881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406796+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:10:47.406796+0000","last_peered":"2024-03-21T09:10:47.406796+0000","last_clean":"2024-03-21T09:10:47.406796+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:10:47.406796+0000","last_undegraded":"2024-03-21T09:10:47.406796+0000","last_fullsized":"2024-03-21T09:10:47.406796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409143+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:10:47.409143+0000","last_peered":"2024-03-21T09:10:47.409143+0000","last_clean":"2024-03-21T09:10:47.409143+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:10:47.409143+0000","last_undegraded":"2024-03-21T09:10:47.409143+0000","last_fullsized":"2024-03-21T09:10:47.409143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404402+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:10:47.404402+0000","last_peered":"2024-03-21T09:10:47.404402+0000","last_clean":"2024-03-21T09:10:47.404402+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:10:47.404402+0000","last_undegraded":"2024-03-21T09:10:47.404402+0000","last_fullsized":"2024-03-21T09:10:47.404402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408215+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:10:47.408215+0000","last_peered":"2024-03-21T09:10:47.408215+0000","last_clean":"2024-03-21T09:10:47.408215+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:10:47.408215+0000","last_undegraded":"2024-03-21T09:10:47.408215+0000","last_fullsized":"2024-03-21T09:10:47.408215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408295+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:10:47.408295+0000","last_peered":"2024-03-21T09:10:47.408295+0000","last_clean":"2024-03-21T09:10:47.408295+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:10:47.408295+0000","last_undegraded":"2024-03-21T09:10:47.408295+0000","last_fullsized":"2024-03-21T09:10:47.408295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402804+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:10:47.402804+0000","last_peered":"2024-03-21T09:10:47.402804+0000","last_clean":"2024-03-21T09:10:47.402804+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:10:47.402804+0000","last_undegraded":"2024-03-21T09:10:47.402804+0000","last_fullsized":"2024-03-21T09:10:47.402804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439328+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:10:47.439328+0000","last_peered":"2024-03-21T09:10:47.439328+0000","last_clean":"2024-03-21T09:10:47.439328+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:10:47.439328+0000","last_undegraded":"2024-03-21T09:10:47.439328+0000","last_fullsized":"2024-03-21T09:10:47.439328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436162+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:10:47.436162+0000","last_peered":"2024-03-21T09:10:47.436162+0000","last_clean":"2024-03-21T09:10:47.436162+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:10:47.436162+0000","last_undegraded":"2024-03-21T09:10:47.436162+0000","last_fullsized":"2024-03-21T09:10:47.436162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402598+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:10:47.402598+0000","last_peered":"2024-03-21T09:10:47.402598+0000","last_clean":"2024-03-21T09:10:47.402598+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:10:47.402598+0000","last_undegraded":"2024-03-21T09:10:47.402598+0000","last_fullsized":"2024-03-21T09:10:47.402598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439700+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:10:47.439700+0000","last_peered":"2024-03-21T09:10:47.439700+0000","last_clean":"2024-03-21T09:10:47.439700+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:10:47.439700+0000","last_undegraded":"2024-03-21T09:10:47.439700+0000","last_fullsized":"2024-03-21T09:10:47.439700+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407244+0000","last_change":"2024-03-21T09:09:47.205699+0000","last_active":"2024-03-21T09:10:47.407244+0000","last_peered":"2024-03-21T09:10:47.407244+0000","last_clean":"2024-03-21T09:10:47.407244+0000","last_became_active":"2024-03-21T09:09:47.203657+0000","last_became_peered":"2024-03-21T09:09:47.203657+0000","last_unstale":"2024-03-21T09:10:47.407244+0000","last_undegraded":"2024-03-21T09:10:47.407244+0000","last_fullsized":"2024-03-21T09:10:47.407244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:42.930153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":191,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:52.042829+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:11:52.042829+0000","last_peered":"2024-03-21T09:11:52.042829+0000","last_clean":"2024-03-21T09:11:52.042829+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:11:52.042829+0000","last_undegraded":"2024-03-21T09:11:52.042829+0000","last_fullsized":"2024-03-21T09:11:52.042829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436708+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:10:47.436708+0000","last_peered":"2024-03-21T09:10:47.436708+0000","last_clean":"2024-03-21T09:10:47.436708+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:10:47.436708+0000","last_undegraded":"2024-03-21T09:10:47.436708+0000","last_fullsized":"2024-03-21T09:10:47.436708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409498+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:10:47.409498+0000","last_peered":"2024-03-21T09:10:47.409498+0000","last_clean":"2024-03-21T09:10:47.409498+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:10:47.409498+0000","last_undegraded":"2024-03-21T09:10:47.409498+0000","last_fullsized":"2024-03-21T09:10:47.409498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436333+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:10:47.436333+0000","last_peered":"2024-03-21T09:10:47.436333+0000","last_clean":"2024-03-21T09:10:47.436333+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:10:47.436333+0000","last_undegraded":"2024-03-21T09:10:47.436333+0000","last_fullsized":"2024-03-21T09:10:47.436333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435910+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:10:47.435910+0000","last_peered":"2024-03-21T09:10:47.435910+0000","last_clean":"2024-03-21T09:10:47.435910+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:10:47.435910+0000","last_undegraded":"2024-03-21T09:10:47.435910+0000","last_fullsized":"2024-03-21T09:10:47.435910+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407059+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:10:47.407059+0000","last_peered":"2024-03-21T09:10:47.407059+0000","last_clean":"2024-03-21T09:10:47.407059+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:10:47.407059+0000","last_undegraded":"2024-03-21T09:10:47.407059+0000","last_fullsized":"2024-03-21T09:10:47.407059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438390+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:10:47.438390+0000","last_peered":"2024-03-21T09:10:47.438390+0000","last_clean":"2024-03-21T09:10:47.438390+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:10:47.438390+0000","last_undegraded":"2024-03-21T09:10:47.438390+0000","last_fullsized":"2024-03-21T09:10:47.438390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436241+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:10:47.436241+0000","last_peered":"2024-03-21T09:10:47.436241+0000","last_clean":"2024-03-21T09:10:47.436241+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:10:47.436241+0000","last_undegraded":"2024-03-21T09:10:47.436241+0000","last_fullsized":"2024-03-21T09:10:47.436241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404852+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:10:47.404852+0000","last_peered":"2024-03-21T09:10:47.404852+0000","last_clean":"2024-03-21T09:10:47.404852+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:10:47.404852+0000","last_undegraded":"2024-03-21T09:10:47.404852+0000","last_fullsized":"2024-03-21T09:10:47.404852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409034+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:10:47.409034+0000","last_peered":"2024-03-21T09:10:47.409034+0000","last_clean":"2024-03-21T09:10:47.409034+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:10:47.409034+0000","last_undegraded":"2024-03-21T09:10:47.409034+0000","last_fullsized":"2024-03-21T09:10:47.409034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439473+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:10:47.439473+0000","last_peered":"2024-03-21T09:10:47.439473+0000","last_clean":"2024-03-21T09:10:47.439473+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:10:47.439473+0000","last_undegraded":"2024-03-21T09:10:47.439473+0000","last_fullsized":"2024-03-21T09:10:47.439473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410228+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:10:47.410228+0000","last_peered":"2024-03-21T09:10:47.410228+0000","last_clean":"2024-03-21T09:10:47.410228+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:10:47.410228+0000","last_undegraded":"2024-03-21T09:10:47.410228+0000","last_fullsized":"2024-03-21T09:10:47.410228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:54.894106+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:10:54.894106+0000","last_peered":"2024-03-21T09:10:54.894106+0000","last_clean":"2024-03-21T09:10:54.894106+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:10:54.894106+0000","last_undegraded":"2024-03-21T09:10:54.894106+0000","last_fullsized":"2024-03-21T09:10:54.894106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411304+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:10:47.411304+0000","last_peered":"2024-03-21T09:10:47.411304+0000","last_clean":"2024-03-21T09:10:47.411304+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:10:47.411304+0000","last_undegraded":"2024-03-21T09:10:47.411304+0000","last_fullsized":"2024-03-21T09:10:47.411304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405917+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:10:47.405917+0000","last_peered":"2024-03-21T09:10:47.405917+0000","last_clean":"2024-03-21T09:10:47.405917+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:10:47.405917+0000","last_undegraded":"2024-03-21T09:10:47.405917+0000","last_fullsized":"2024-03-21T09:10:47.405917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436899+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:10:47.436899+0000","last_peered":"2024-03-21T09:10:47.436899+0000","last_clean":"2024-03-21T09:10:47.436899+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:10:47.436899+0000","last_undegraded":"2024-03-21T09:10:47.436899+0000","last_fullsized":"2024-03-21T09:10:47.436899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438375+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:10:47.438375+0000","last_peered":"2024-03-21T09:10:47.438375+0000","last_clean":"2024-03-21T09:10:47.438375+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:10:47.438375+0000","last_undegraded":"2024-03-21T09:10:47.438375+0000","last_fullsized":"2024-03-21T09:10:47.438375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436534+0000","last_change":"2024-03-21T09:09:47.203560+0000","last_active":"2024-03-21T09:10:47.436534+0000","last_peered":"2024-03-21T09:10:47.436534+0000","last_clean":"2024-03-21T09:10:47.436534+0000","last_became_active":"2024-03-21T09:09:47.203072+0000","last_became_peered":"2024-03-21T09:09:47.203072+0000","last_unstale":"2024-03-21T09:10:47.436534+0000","last_undegraded":"2024-03-21T09:10:47.436534+0000","last_fullsized":"2024-03-21T09:10:47.436534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:03:39.037042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406193+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:10:47.406193+0000","last_peered":"2024-03-21T09:10:47.406193+0000","last_clean":"2024-03-21T09:10:47.406193+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:10:47.406193+0000","last_undegraded":"2024-03-21T09:10:47.406193+0000","last_fullsized":"2024-03-21T09:10:47.406193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408090+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:10:47.408090+0000","last_peered":"2024-03-21T09:10:47.408090+0000","last_clean":"2024-03-21T09:10:47.408090+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:10:47.408090+0000","last_undegraded":"2024-03-21T09:10:47.408090+0000","last_fullsized":"2024-03-21T09:10:47.408090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408581+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:10:47.408581+0000","last_peered":"2024-03-21T09:10:47.408581+0000","last_clean":"2024-03-21T09:10:47.408581+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:10:47.408581+0000","last_undegraded":"2024-03-21T09:10:47.408581+0000","last_fullsized":"2024-03-21T09:10:47.408581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409274+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:10:47.409274+0000","last_peered":"2024-03-21T09:10:47.409274+0000","last_clean":"2024-03-21T09:10:47.409274+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:10:47.409274+0000","last_undegraded":"2024-03-21T09:10:47.409274+0000","last_fullsized":"2024-03-21T09:10:47.409274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404093+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:10:47.404093+0000","last_peered":"2024-03-21T09:10:47.404093+0000","last_clean":"2024-03-21T09:10:47.404093+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:10:47.404093+0000","last_undegraded":"2024-03-21T09:10:47.404093+0000","last_fullsized":"2024-03-21T09:10:47.404093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436903+0000","last_change":"2024-03-21T09:09:47.206281+0000","last_active":"2024-03-21T09:10:47.436903+0000","last_peered":"2024-03-21T09:10:47.436903+0000","last_clean":"2024-03-21T09:10:47.436903+0000","last_became_active":"2024-03-21T09:09:47.198427+0000","last_became_peered":"2024-03-21T09:09:47.198427+0000","last_unstale":"2024-03-21T09:10:47.436903+0000","last_undegraded":"2024-03-21T09:10:47.436903+0000","last_fullsized":"2024-03-21T09:10:47.436903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:32:46.182225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406103+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:10:47.406103+0000","last_peered":"2024-03-21T09:10:47.406103+0000","last_clean":"2024-03-21T09:10:47.406103+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:10:47.406103+0000","last_undegraded":"2024-03-21T09:10:47.406103+0000","last_fullsized":"2024-03-21T09:10:47.406103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410703+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:10:47.410703+0000","last_peered":"2024-03-21T09:10:47.410703+0000","last_clean":"2024-03-21T09:10:47.410703+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:10:47.410703+0000","last_undegraded":"2024-03-21T09:10:47.410703+0000","last_fullsized":"2024-03-21T09:10:47.410703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436507+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:10:47.436507+0000","last_peered":"2024-03-21T09:10:47.436507+0000","last_clean":"2024-03-21T09:10:47.436507+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:10:47.436507+0000","last_undegraded":"2024-03-21T09:10:47.436507+0000","last_fullsized":"2024-03-21T09:10:47.436507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408535+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:10:47.408535+0000","last_peered":"2024-03-21T09:10:47.408535+0000","last_clean":"2024-03-21T09:10:47.408535+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:10:47.408535+0000","last_undegraded":"2024-03-21T09:10:47.408535+0000","last_fullsized":"2024-03-21T09:10:47.408535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436195+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:10:47.436195+0000","last_peered":"2024-03-21T09:10:47.436195+0000","last_clean":"2024-03-21T09:10:47.436195+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:10:47.436195+0000","last_undegraded":"2024-03-21T09:10:47.436195+0000","last_fullsized":"2024-03-21T09:10:47.436195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406267+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:10:47.406267+0000","last_peered":"2024-03-21T09:10:47.406267+0000","last_clean":"2024-03-21T09:10:47.406267+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:10:47.406267+0000","last_undegraded":"2024-03-21T09:10:47.406267+0000","last_fullsized":"2024-03-21T09:10:47.406267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406667+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:10:47.406667+0000","last_peered":"2024-03-21T09:10:47.406667+0000","last_clean":"2024-03-21T09:10:47.406667+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:10:47.406667+0000","last_undegraded":"2024-03-21T09:10:47.406667+0000","last_fullsized":"2024-03-21T09:10:47.406667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436775+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:10:47.436775+0000","last_peered":"2024-03-21T09:10:47.436775+0000","last_clean":"2024-03-21T09:10:47.436775+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:10:47.436775+0000","last_undegraded":"2024-03-21T09:10:47.436775+0000","last_fullsized":"2024-03-21T09:10:47.436775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437679+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:10:47.437679+0000","last_peered":"2024-03-21T09:10:47.437679+0000","last_clean":"2024-03-21T09:10:47.437679+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:10:47.437679+0000","last_undegraded":"2024-03-21T09:10:47.437679+0000","last_fullsized":"2024-03-21T09:10:47.437679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407624+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:10:47.407624+0000","last_peered":"2024-03-21T09:10:47.407624+0000","last_clean":"2024-03-21T09:10:47.407624+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:10:47.407624+0000","last_undegraded":"2024-03-21T09:10:47.407624+0000","last_fullsized":"2024-03-21T09:10:47.407624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436680+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:10:47.436680+0000","last_peered":"2024-03-21T09:10:47.436680+0000","last_clean":"2024-03-21T09:10:47.436680+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:10:47.436680+0000","last_undegraded":"2024-03-21T09:10:47.436680+0000","last_fullsized":"2024-03-21T09:10:47.436680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406417+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:10:47.406417+0000","last_peered":"2024-03-21T09:10:47.406417+0000","last_clean":"2024-03-21T09:10:47.406417+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:10:47.406417+0000","last_undegraded":"2024-03-21T09:10:47.406417+0000","last_fullsized":"2024-03-21T09:10:47.406417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439536+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:10:47.439536+0000","last_peered":"2024-03-21T09:10:47.439536+0000","last_clean":"2024-03-21T09:10:47.439536+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:10:47.439536+0000","last_undegraded":"2024-03-21T09:10:47.439536+0000","last_fullsized":"2024-03-21T09:10:47.439536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436796+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:10:47.436796+0000","last_peered":"2024-03-21T09:10:47.436796+0000","last_clean":"2024-03-21T09:10:47.436796+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:10:47.436796+0000","last_undegraded":"2024-03-21T09:10:47.436796+0000","last_fullsized":"2024-03-21T09:10:47.436796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404217+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:10:47.404217+0000","last_peered":"2024-03-21T09:10:47.404217+0000","last_clean":"2024-03-21T09:10:47.404217+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:10:47.404217+0000","last_undegraded":"2024-03-21T09:10:47.404217+0000","last_fullsized":"2024-03-21T09:10:47.404217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407361+0000","last_change":"2024-03-21T09:09:47.200898+0000","last_active":"2024-03-21T09:10:47.407361+0000","last_peered":"2024-03-21T09:10:47.407361+0000","last_clean":"2024-03-21T09:10:47.407361+0000","last_became_active":"2024-03-21T09:09:47.200548+0000","last_became_peered":"2024-03-21T09:09:47.200548+0000","last_unstale":"2024-03-21T09:10:47.407361+0000","last_undegraded":"2024-03-21T09:10:47.407361+0000","last_fullsized":"2024-03-21T09:10:47.407361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:29.793107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405957+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:10:47.405957+0000","last_peered":"2024-03-21T09:10:47.405957+0000","last_clean":"2024-03-21T09:10:47.405957+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:10:47.405957+0000","last_undegraded":"2024-03-21T09:10:47.405957+0000","last_fullsized":"2024-03-21T09:10:47.405957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437955+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:10:47.437955+0000","last_peered":"2024-03-21T09:10:47.437955+0000","last_clean":"2024-03-21T09:10:47.437955+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:10:47.437955+0000","last_undegraded":"2024-03-21T09:10:47.437955+0000","last_fullsized":"2024-03-21T09:10:47.437955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439688+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:10:47.439688+0000","last_peered":"2024-03-21T09:10:47.439688+0000","last_clean":"2024-03-21T09:10:47.439688+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:10:47.439688+0000","last_undegraded":"2024-03-21T09:10:47.439688+0000","last_fullsized":"2024-03-21T09:10:47.439688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404565+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:10:47.404565+0000","last_peered":"2024-03-21T09:10:47.404565+0000","last_clean":"2024-03-21T09:10:47.404565+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:10:47.404565+0000","last_undegraded":"2024-03-21T09:10:47.404565+0000","last_fullsized":"2024-03-21T09:10:47.404565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407263+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:10:47.407263+0000","last_peered":"2024-03-21T09:10:47.407263+0000","last_clean":"2024-03-21T09:10:47.407263+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:10:47.407263+0000","last_undegraded":"2024-03-21T09:10:47.407263+0000","last_fullsized":"2024-03-21T09:10:47.407263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411684+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:10:47.411684+0000","last_peered":"2024-03-21T09:10:47.411684+0000","last_clean":"2024-03-21T09:10:47.411684+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:10:47.411684+0000","last_undegraded":"2024-03-21T09:10:47.411684+0000","last_fullsized":"2024-03-21T09:10:47.411684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406548+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:10:47.406548+0000","last_peered":"2024-03-21T09:10:47.406548+0000","last_clean":"2024-03-21T09:10:47.406548+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:10:47.406548+0000","last_undegraded":"2024-03-21T09:10:47.406548+0000","last_fullsized":"2024-03-21T09:10:47.406548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409488+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:10:47.409488+0000","last_peered":"2024-03-21T09:10:47.409488+0000","last_clean":"2024-03-21T09:10:47.409488+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:10:47.409488+0000","last_undegraded":"2024-03-21T09:10:47.409488+0000","last_fullsized":"2024-03-21T09:10:47.409488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406502+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:10:47.406502+0000","last_peered":"2024-03-21T09:10:47.406502+0000","last_clean":"2024-03-21T09:10:47.406502+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:10:47.406502+0000","last_undegraded":"2024-03-21T09:10:47.406502+0000","last_fullsized":"2024-03-21T09:10:47.406502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439038+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:10:47.439038+0000","last_peered":"2024-03-21T09:10:47.439038+0000","last_clean":"2024-03-21T09:10:47.439038+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:10:47.439038+0000","last_undegraded":"2024-03-21T09:10:47.439038+0000","last_fullsized":"2024-03-21T09:10:47.439038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404751+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:10:47.404751+0000","last_peered":"2024-03-21T09:10:47.404751+0000","last_clean":"2024-03-21T09:10:47.404751+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:10:47.404751+0000","last_undegraded":"2024-03-21T09:10:47.404751+0000","last_fullsized":"2024-03-21T09:10:47.404751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75750,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":944,"num_read_kb":553,"num_write":177,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":222210,"data_compressed":5499,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351029,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":526262,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":1187212,"data_compressed":64715,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808895,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":526464,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":526384,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332170,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1123131,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":521640,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":589941,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":523028,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":585493,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":521834,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":527128,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1117490,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68391,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69707,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73718,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:11:55.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T09:11:55.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:55 smithi037 ceph-mon[31830]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T09:11:55.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:55 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2088413331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:55.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:55 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2234939561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:55.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:55 smithi071 ceph-mon[36597]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T09:11:55.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:55 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2088413331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:55.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:55 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2234939561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:55.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:55 smithi053 ceph-mon[36169]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T09:11:55.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:55 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2088413331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:55.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:55 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2234939561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 -- 172.21.15.37:0/2230120732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 msgr2=0x7f062c10bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 --2- 172.21.15.37:0/2230120732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c10bc20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0624009ca0 tx=0x7f062402c100 comp rx=0 tx=0).stop 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 -- 172.21.15.37:0/2230120732 shutdown_connections 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 --2- 172.21.15.37:0/2230120732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c10bc20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 --2- 172.21.15.37:0/2230120732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1049d0 0x7f062c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 --2- 172.21.15.37:0/2230120732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1037e0 0x7f062c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 -- 172.21.15.37:0/2230120732 >> 172.21.15.37:0/2230120732 conn(0x7f062c0fed70 msgr2=0x7f062c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 -- 172.21.15.37:0/2230120732 shutdown_connections 2024-03-21T09:11:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.474+0000 7f0632600640 1 -- 172.21.15.37:0/2230120732 wait complete. 2024-03-21T09:11:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.476+0000 7f0632600640 1 Processor -- start 2024-03-21T09:11:55.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.476+0000 7f0632600640 1 -- start start 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0632600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 0x7f062c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0632600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1049d0 0x7f062c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0632600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0632600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c10f460 con 0x7f062c1049d0 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0632600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c10f2e0 con 0x7f062c105390 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0632600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c10f5e0 con 0x7f062c1037e0 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0631200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 0x7f062c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0631200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 0x7f062c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57142/0 (socket says 172.21.15.37:57142) 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0631200640 1 -- 172.21.15.37:0/334088842 learned_addr learned my addr 172.21.15.37:0/334088842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:55.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f0631c00640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.477+0000 7f062be00640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1049d0 0x7f062c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631200640 1 -- 172.21.15.37:0/334088842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 msgr2=0x7f062c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631200640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631200640 1 -- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1049d0 msgr2=0x7f062c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631200640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1049d0 0x7f062c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631200640 1 -- 172.21.15.37:0/334088842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c1a16f0 con 0x7f062c1037e0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631c00640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f062be00640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1049d0 0x7f062c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0631200640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 0x7f062c199b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f062000e9c0 tx=0x7f062000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000cd80 con 0x7f062c1037e0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06200035f0 con 0x7f062c1037e0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620010690 con 0x7f062c1037e0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c1a19e0 con 0x7f062c1037e0 2024-03-21T09:11:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.478+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c1a1ee0 con 0x7f062c1037e0 2024-03-21T09:11:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.480+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c10aaa0 con 0x7f062c1037e0 2024-03-21T09:11:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.481+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f06200037a0 con 0x7f062c1037e0 2024-03-21T09:11:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.483+0000 7f0629600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06140a3020 0x7f06140a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.483+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f06200cfe50 con 0x7f062c1037e0 2024-03-21T09:11:55.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.483+0000 7f062be00640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06140a3020 0x7f06140a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:55.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.484+0000 7f062be00640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06140a3020 0x7f06140a54e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f062c19afe0 tx=0x7f0618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.486+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f062009c730 con 0x7f062c1037e0 2024-03-21T09:11:55.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.686+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f062c061ae0 con 0x7f06140a3020 2024-03-21T09:11:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.715+0000 7f0629600640 1 -- 172.21.15.37:0/334088842 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340812 (secure 0 0 0) 0x7f062c061ae0 con 0x7f06140a3020 2024-03-21T09:11:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.725+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06140a3020 msgr2=0x7f06140a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.725+0000 7f0632600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06140a3020 0x7f06140a54e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f062c19afe0 tx=0x7f0618005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.725+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 msgr2=0x7f062c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.725+0000 7f0632600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 0x7f062c199b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f062000e9c0 tx=0x7f062000ee80 comp rx=0 tx=0).stop 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 shutdown_connections 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f06140a3020 0x7f06140a54e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f062c105390 0x7f062c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062c1049d0 0x7f062c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 --2- 172.21.15.37:0/334088842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c1037e0 0x7f062c199b70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 >> 172.21.15.37:0/334088842 conn(0x7f062c0fed70 msgr2=0x7f062c107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 shutdown_connections 2024-03-21T09:11:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:55.726+0000 7f0632600640 1 -- 172.21.15.37:0/334088842 wait complete. 2024-03-21T09:11:55.752 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-03-21T09:11:53.849853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666352,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1080,"num_read_kb":672,"num_write":360,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":792,"ondisk_log_size":792,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382592,"kb_used_data":12592,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559232,"statfs":{"total":1151940427776,"available":1151548653568,"internally_reserved":0,"allocated":12894208,"data_stored":8276007,"data_compressed":433446,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9287,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":56,"num_write":9,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004892"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404943+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:10:47.404943+0000","last_peered":"2024-03-21T09:10:47.404943+0000","last_clean":"2024-03-21T09:10:47.404943+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:10:47.404943+0000","last_undegraded":"2024-03-21T09:10:47.404943+0000","last_fullsized":"2024-03-21T09:10:47.404943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438508+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:10:47.438508+0000","last_peered":"2024-03-21T09:10:47.438508+0000","last_clean":"2024-03-21T09:10:47.438508+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:10:47.438508+0000","last_undegraded":"2024-03-21T09:10:47.438508+0000","last_fullsized":"2024-03-21T09:10:47.438508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437937+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:10:47.437937+0000","last_peered":"2024-03-21T09:10:47.437937+0000","last_clean":"2024-03-21T09:10:47.437937+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:10:47.437937+0000","last_undegraded":"2024-03-21T09:10:47.437937+0000","last_fullsized":"2024-03-21T09:10:47.437937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409123+0000","last_change":"2024-03-21T09:09:47.203174+0000","last_active":"2024-03-21T09:10:47.409123+0000","last_peered":"2024-03-21T09:10:47.409123+0000","last_clean":"2024-03-21T09:10:47.409123+0000","last_became_active":"2024-03-21T09:09:47.202519+0000","last_became_peered":"2024-03-21T09:09:47.202519+0000","last_unstale":"2024-03-21T09:10:47.409123+0000","last_undegraded":"2024-03-21T09:10:47.409123+0000","last_fullsized":"2024-03-21T09:10:47.409123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:48.445311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408507+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:10:47.408507+0000","last_peered":"2024-03-21T09:10:47.408507+0000","last_clean":"2024-03-21T09:10:47.408507+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:10:47.408507+0000","last_undegraded":"2024-03-21T09:10:47.408507+0000","last_fullsized":"2024-03-21T09:10:47.408507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402461+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:10:47.402461+0000","last_peered":"2024-03-21T09:10:47.402461+0000","last_clean":"2024-03-21T09:10:47.402461+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:10:47.402461+0000","last_undegraded":"2024-03-21T09:10:47.402461+0000","last_fullsized":"2024-03-21T09:10:47.402461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436131+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:10:47.436131+0000","last_peered":"2024-03-21T09:10:47.436131+0000","last_clean":"2024-03-21T09:10:47.436131+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:10:47.436131+0000","last_undegraded":"2024-03-21T09:10:47.436131+0000","last_fullsized":"2024-03-21T09:10:47.436131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406994+0000","last_change":"2024-03-21T09:09:47.204487+0000","last_active":"2024-03-21T09:10:47.406994+0000","last_peered":"2024-03-21T09:10:47.406994+0000","last_clean":"2024-03-21T09:10:47.406994+0000","last_became_active":"2024-03-21T09:09:47.202961+0000","last_became_peered":"2024-03-21T09:09:47.202961+0000","last_unstale":"2024-03-21T09:10:47.406994+0000","last_undegraded":"2024-03-21T09:10:47.406994+0000","last_fullsized":"2024-03-21T09:10:47.406994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:50.524550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406889+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:10:47.406889+0000","last_peered":"2024-03-21T09:10:47.406889+0000","last_clean":"2024-03-21T09:10:47.406889+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:10:47.406889+0000","last_undegraded":"2024-03-21T09:10:47.406889+0000","last_fullsized":"2024-03-21T09:10:47.406889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402825+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:10:47.402825+0000","last_peered":"2024-03-21T09:10:47.402825+0000","last_clean":"2024-03-21T09:10:47.402825+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:10:47.402825+0000","last_undegraded":"2024-03-21T09:10:47.402825+0000","last_fullsized":"2024-03-21T09:10:47.402825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436663+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:10:47.436663+0000","last_peered":"2024-03-21T09:10:47.436663+0000","last_clean":"2024-03-21T09:10:47.436663+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:10:47.436663+0000","last_undegraded":"2024-03-21T09:10:47.436663+0000","last_fullsized":"2024-03-21T09:10:47.436663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436967+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:10:47.436967+0000","last_peered":"2024-03-21T09:10:47.436967+0000","last_clean":"2024-03-21T09:10:47.436967+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:10:47.436967+0000","last_undegraded":"2024-03-21T09:10:47.436967+0000","last_fullsized":"2024-03-21T09:10:47.436967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439147+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:10:47.439147+0000","last_peered":"2024-03-21T09:10:47.439147+0000","last_clean":"2024-03-21T09:10:47.439147+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:10:47.439147+0000","last_undegraded":"2024-03-21T09:10:47.439147+0000","last_fullsized":"2024-03-21T09:10:47.439147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437049+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:10:47.437049+0000","last_peered":"2024-03-21T09:10:47.437049+0000","last_clean":"2024-03-21T09:10:47.437049+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:10:47.437049+0000","last_undegraded":"2024-03-21T09:10:47.437049+0000","last_fullsized":"2024-03-21T09:10:47.437049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437308+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:10:47.437308+0000","last_peered":"2024-03-21T09:10:47.437308+0000","last_clean":"2024-03-21T09:10:47.437308+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:10:47.437308+0000","last_undegraded":"2024-03-21T09:10:47.437308+0000","last_fullsized":"2024-03-21T09:10:47.437308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406169+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:10:47.406169+0000","last_peered":"2024-03-21T09:10:47.406169+0000","last_clean":"2024-03-21T09:10:47.406169+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:10:47.406169+0000","last_undegraded":"2024-03-21T09:10:47.406169+0000","last_fullsized":"2024-03-21T09:10:47.406169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404683+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:10:47.404683+0000","last_peered":"2024-03-21T09:10:47.404683+0000","last_clean":"2024-03-21T09:10:47.404683+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:10:47.404683+0000","last_undegraded":"2024-03-21T09:10:47.404683+0000","last_fullsized":"2024-03-21T09:10:47.404683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408836+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:10:47.408836+0000","last_peered":"2024-03-21T09:10:47.408836+0000","last_clean":"2024-03-21T09:10:47.408836+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:10:47.408836+0000","last_undegraded":"2024-03-21T09:10:47.408836+0000","last_fullsized":"2024-03-21T09:10:47.408836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406741+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:10:47.406741+0000","last_peered":"2024-03-21T09:10:47.406741+0000","last_clean":"2024-03-21T09:10:47.406741+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:10:47.406741+0000","last_undegraded":"2024-03-21T09:10:47.406741+0000","last_fullsized":"2024-03-21T09:10:47.406741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406108+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:10:47.406108+0000","last_peered":"2024-03-21T09:10:47.406108+0000","last_clean":"2024-03-21T09:10:47.406108+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:10:47.406108+0000","last_undegraded":"2024-03-21T09:10:47.406108+0000","last_fullsized":"2024-03-21T09:10:47.406108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408966+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:10:47.408966+0000","last_peered":"2024-03-21T09:10:47.408966+0000","last_clean":"2024-03-21T09:10:47.408966+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:10:47.408966+0000","last_undegraded":"2024-03-21T09:10:47.408966+0000","last_fullsized":"2024-03-21T09:10:47.408966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411324+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:10:47.411324+0000","last_peered":"2024-03-21T09:10:47.411324+0000","last_clean":"2024-03-21T09:10:47.411324+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:10:47.411324+0000","last_undegraded":"2024-03-21T09:10:47.411324+0000","last_fullsized":"2024-03-21T09:10:47.411324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435847+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:10:47.435847+0000","last_peered":"2024-03-21T09:10:47.435847+0000","last_clean":"2024-03-21T09:10:47.435847+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:10:47.435847+0000","last_undegraded":"2024-03-21T09:10:47.435847+0000","last_fullsized":"2024-03-21T09:10:47.435847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411396+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:10:47.411396+0000","last_peered":"2024-03-21T09:10:47.411396+0000","last_clean":"2024-03-21T09:10:47.411396+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:10:47.411396+0000","last_undegraded":"2024-03-21T09:10:47.411396+0000","last_fullsized":"2024-03-21T09:10:47.411396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439238+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:10:47.439238+0000","last_peered":"2024-03-21T09:10:47.439238+0000","last_clean":"2024-03-21T09:10:47.439238+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:10:47.439238+0000","last_undegraded":"2024-03-21T09:10:47.439238+0000","last_fullsized":"2024-03-21T09:10:47.439238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436274+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:10:47.436274+0000","last_peered":"2024-03-21T09:10:47.436274+0000","last_clean":"2024-03-21T09:10:47.436274+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:10:47.436274+0000","last_undegraded":"2024-03-21T09:10:47.436274+0000","last_fullsized":"2024-03-21T09:10:47.436274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409035+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:10:47.409035+0000","last_peered":"2024-03-21T09:10:47.409035+0000","last_clean":"2024-03-21T09:10:47.409035+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:10:47.409035+0000","last_undegraded":"2024-03-21T09:10:47.409035+0000","last_fullsized":"2024-03-21T09:10:47.409035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406668+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:10:47.406668+0000","last_peered":"2024-03-21T09:10:47.406668+0000","last_clean":"2024-03-21T09:10:47.406668+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:10:47.406668+0000","last_undegraded":"2024-03-21T09:10:47.406668+0000","last_fullsized":"2024-03-21T09:10:47.406668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436234+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:10:47.436234+0000","last_peered":"2024-03-21T09:10:47.436234+0000","last_clean":"2024-03-21T09:10:47.436234+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:10:47.436234+0000","last_undegraded":"2024-03-21T09:10:47.436234+0000","last_fullsized":"2024-03-21T09:10:47.436234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411729+0000","last_change":"2024-03-21T09:10:47.411335+0000","last_active":"2024-03-21T09:10:47.411729+0000","last_peered":"2024-03-21T09:10:47.411729+0000","last_clean":"2024-03-21T09:10:47.411729+0000","last_became_active":"2024-03-21T09:10:47.408103+0000","last_became_peered":"2024-03-21T09:10:47.408103+0000","last_unstale":"2024-03-21T09:10:47.411729+0000","last_undegraded":"2024-03-21T09:10:47.411729+0000","last_fullsized":"2024-03-21T09:10:47.411729+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:51.600076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407587+0000","last_change":"2024-03-21T09:09:47.433837+0000","last_active":"2024-03-21T09:10:47.407587+0000","last_peered":"2024-03-21T09:10:47.407587+0000","last_clean":"2024-03-21T09:10:47.407587+0000","last_became_active":"2024-03-21T09:09:47.433538+0000","last_became_peered":"2024-03-21T09:09:47.433538+0000","last_unstale":"2024-03-21T09:10:47.407587+0000","last_undegraded":"2024-03-21T09:10:47.407587+0000","last_fullsized":"2024-03-21T09:10:47.407587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:41.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406895+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:10:47.406895+0000","last_peered":"2024-03-21T09:10:47.406895+0000","last_clean":"2024-03-21T09:10:47.406895+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:10:47.406895+0000","last_undegraded":"2024-03-21T09:10:47.406895+0000","last_fullsized":"2024-03-21T09:10:47.406895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:51.961475+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:11:51.961475+0000","last_peered":"2024-03-21T09:11:51.961475+0000","last_clean":"2024-03-21T09:11:51.961475+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:11:51.961475+0000","last_undegraded":"2024-03-21T09:11:51.961475+0000","last_fullsized":"2024-03-21T09:11:51.961475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408270+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:10:47.408270+0000","last_peered":"2024-03-21T09:10:47.408270+0000","last_clean":"2024-03-21T09:10:47.408270+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:10:47.408270+0000","last_undegraded":"2024-03-21T09:10:47.408270+0000","last_fullsized":"2024-03-21T09:10:47.408270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406440+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:10:47.406440+0000","last_peered":"2024-03-21T09:10:47.406440+0000","last_clean":"2024-03-21T09:10:47.406440+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:10:47.406440+0000","last_undegraded":"2024-03-21T09:10:47.406440+0000","last_fullsized":"2024-03-21T09:10:47.406440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402428+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:10:47.402428+0000","last_peered":"2024-03-21T09:10:47.402428+0000","last_clean":"2024-03-21T09:10:47.402428+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:10:47.402428+0000","last_undegraded":"2024-03-21T09:10:47.402428+0000","last_fullsized":"2024-03-21T09:10:47.402428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407441+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:10:47.407441+0000","last_peered":"2024-03-21T09:10:47.407441+0000","last_clean":"2024-03-21T09:10:47.407441+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:10:47.407441+0000","last_undegraded":"2024-03-21T09:10:47.407441+0000","last_fullsized":"2024-03-21T09:10:47.407441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404654+0000","last_change":"2024-03-21T09:09:47.432250+0000","last_active":"2024-03-21T09:10:47.404654+0000","last_peered":"2024-03-21T09:10:47.404654+0000","last_clean":"2024-03-21T09:10:47.404654+0000","last_became_active":"2024-03-21T09:09:47.431784+0000","last_became_peered":"2024-03-21T09:09:47.431784+0000","last_unstale":"2024-03-21T09:10:47.404654+0000","last_undegraded":"2024-03-21T09:10:47.404654+0000","last_fullsized":"2024-03-21T09:10:47.404654+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:58.438472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438375+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:10:47.438375+0000","last_peered":"2024-03-21T09:10:47.438375+0000","last_clean":"2024-03-21T09:10:47.438375+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:10:47.438375+0000","last_undegraded":"2024-03-21T09:10:47.438375+0000","last_fullsized":"2024-03-21T09:10:47.438375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410399+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:10:47.410399+0000","last_peered":"2024-03-21T09:10:47.410399+0000","last_clean":"2024-03-21T09:10:47.410399+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:10:47.410399+0000","last_undegraded":"2024-03-21T09:10:47.410399+0000","last_fullsized":"2024-03-21T09:10:47.410399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438232+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:10:47.438232+0000","last_peered":"2024-03-21T09:10:47.438232+0000","last_clean":"2024-03-21T09:10:47.438232+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:10:47.438232+0000","last_undegraded":"2024-03-21T09:10:47.438232+0000","last_fullsized":"2024-03-21T09:10:47.438232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406786+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:10:47.406786+0000","last_peered":"2024-03-21T09:10:47.406786+0000","last_clean":"2024-03-21T09:10:47.406786+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:10:47.406786+0000","last_undegraded":"2024-03-21T09:10:47.406786+0000","last_fullsized":"2024-03-21T09:10:47.406786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407682+0000","last_change":"2024-03-21T09:09:47.205219+0000","last_active":"2024-03-21T09:10:47.407682+0000","last_peered":"2024-03-21T09:10:47.407682+0000","last_clean":"2024-03-21T09:10:47.407682+0000","last_became_active":"2024-03-21T09:09:47.204655+0000","last_became_peered":"2024-03-21T09:09:47.204655+0000","last_unstale":"2024-03-21T09:10:47.407682+0000","last_undegraded":"2024-03-21T09:10:47.407682+0000","last_fullsized":"2024-03-21T09:10:47.407682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:44.407879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409872+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:10:47.409872+0000","last_peered":"2024-03-21T09:10:47.409872+0000","last_clean":"2024-03-21T09:10:47.409872+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:10:47.409872+0000","last_undegraded":"2024-03-21T09:10:47.409872+0000","last_fullsized":"2024-03-21T09:10:47.409872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":186,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:50.960036+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:11:50.960036+0000","last_peered":"2024-03-21T09:11:50.960036+0000","last_clean":"2024-03-21T09:11:50.960036+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:11:50.960036+0000","last_undegraded":"2024-03-21T09:11:50.960036+0000","last_fullsized":"2024-03-21T09:11:50.960036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406846+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:10:47.406846+0000","last_peered":"2024-03-21T09:10:47.406846+0000","last_clean":"2024-03-21T09:10:47.406846+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:10:47.406846+0000","last_undegraded":"2024-03-21T09:10:47.406846+0000","last_fullsized":"2024-03-21T09:10:47.406846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407552+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:10:47.407552+0000","last_peered":"2024-03-21T09:10:47.407552+0000","last_clean":"2024-03-21T09:10:47.407552+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:10:47.407552+0000","last_undegraded":"2024-03-21T09:10:47.407552+0000","last_fullsized":"2024-03-21T09:10:47.407552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436943+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:10:47.436943+0000","last_peered":"2024-03-21T09:10:47.436943+0000","last_clean":"2024-03-21T09:10:47.436943+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:10:47.436943+0000","last_undegraded":"2024-03-21T09:10:47.436943+0000","last_fullsized":"2024-03-21T09:10:47.436943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439194+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:10:47.439194+0000","last_peered":"2024-03-21T09:10:47.439194+0000","last_clean":"2024-03-21T09:10:47.439194+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:10:47.439194+0000","last_undegraded":"2024-03-21T09:10:47.439194+0000","last_fullsized":"2024-03-21T09:10:47.439194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436421+0000","last_change":"2024-03-21T09:09:47.283764+0000","last_active":"2024-03-21T09:10:47.436421+0000","last_peered":"2024-03-21T09:10:47.436421+0000","last_clean":"2024-03-21T09:10:47.436421+0000","last_became_active":"2024-03-21T09:09:47.283203+0000","last_became_peered":"2024-03-21T09:09:47.283203+0000","last_unstale":"2024-03-21T09:10:47.436421+0000","last_undegraded":"2024-03-21T09:10:47.436421+0000","last_fullsized":"2024-03-21T09:10:47.436421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:55.502176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407507+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:10:47.407507+0000","last_peered":"2024-03-21T09:10:47.407507+0000","last_clean":"2024-03-21T09:10:47.407507+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:10:47.407507+0000","last_undegraded":"2024-03-21T09:10:47.407507+0000","last_fullsized":"2024-03-21T09:10:47.407507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406746+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:10:47.406746+0000","last_peered":"2024-03-21T09:10:47.406746+0000","last_clean":"2024-03-21T09:10:47.406746+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:10:47.406746+0000","last_undegraded":"2024-03-21T09:10:47.406746+0000","last_fullsized":"2024-03-21T09:10:47.406746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407596+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:10:47.407596+0000","last_peered":"2024-03-21T09:10:47.407596+0000","last_clean":"2024-03-21T09:10:47.407596+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:10:47.407596+0000","last_undegraded":"2024-03-21T09:10:47.407596+0000","last_fullsized":"2024-03-21T09:10:47.407596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":385,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:51.962060+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:11:51.962060+0000","last_peered":"2024-03-21T09:11:51.962060+0000","last_clean":"2024-03-21T09:11:51.962060+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:11:51.962060+0000","last_undegraded":"2024-03-21T09:11:51.962060+0000","last_fullsized":"2024-03-21T09:11:51.962060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70911,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":227,"num_write":39,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408857+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:10:47.408857+0000","last_peered":"2024-03-21T09:10:47.408857+0000","last_clean":"2024-03-21T09:10:47.408857+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:10:47.408857+0000","last_undegraded":"2024-03-21T09:10:47.408857+0000","last_fullsized":"2024-03-21T09:10:47.408857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407740+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:10:47.407740+0000","last_peered":"2024-03-21T09:10:47.407740+0000","last_clean":"2024-03-21T09:10:47.407740+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:10:47.407740+0000","last_undegraded":"2024-03-21T09:10:47.407740+0000","last_fullsized":"2024-03-21T09:10:47.407740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.420395+0000","last_change":"2024-03-21T09:10:47.420051+0000","last_active":"2024-03-21T09:10:47.420395+0000","last_peered":"2024-03-21T09:10:47.420395+0000","last_clean":"2024-03-21T09:10:47.420395+0000","last_became_active":"2024-03-21T09:10:47.412598+0000","last_became_peered":"2024-03-21T09:10:47.412598+0000","last_unstale":"2024-03-21T09:10:47.420395+0000","last_undegraded":"2024-03-21T09:10:47.420395+0000","last_fullsized":"2024-03-21T09:10:47.420395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:43.675518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438823+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:10:47.438823+0000","last_peered":"2024-03-21T09:10:47.438823+0000","last_clean":"2024-03-21T09:10:47.438823+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:10:47.438823+0000","last_undegraded":"2024-03-21T09:10:47.438823+0000","last_fullsized":"2024-03-21T09:10:47.438823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404558+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:10:47.404558+0000","last_peered":"2024-03-21T09:10:47.404558+0000","last_clean":"2024-03-21T09:10:47.404558+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:10:47.404558+0000","last_undegraded":"2024-03-21T09:10:47.404558+0000","last_fullsized":"2024-03-21T09:10:47.404558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407208+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:10:47.407208+0000","last_peered":"2024-03-21T09:10:47.407208+0000","last_clean":"2024-03-21T09:10:47.407208+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:10:47.407208+0000","last_undegraded":"2024-03-21T09:10:47.407208+0000","last_fullsized":"2024-03-21T09:10:47.407208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436363+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:10:47.436363+0000","last_peered":"2024-03-21T09:10:47.436363+0000","last_clean":"2024-03-21T09:10:47.436363+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:10:47.436363+0000","last_undegraded":"2024-03-21T09:10:47.436363+0000","last_fullsized":"2024-03-21T09:10:47.436363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408459+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:10:47.408459+0000","last_peered":"2024-03-21T09:10:47.408459+0000","last_clean":"2024-03-21T09:10:47.408459+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:10:47.408459+0000","last_undegraded":"2024-03-21T09:10:47.408459+0000","last_fullsized":"2024-03-21T09:10:47.408459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:49.038543+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:11:49.038543+0000","last_peered":"2024-03-21T09:11:49.038543+0000","last_clean":"2024-03-21T09:11:49.038543+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:11:49.038543+0000","last_undegraded":"2024-03-21T09:11:49.038543+0000","last_fullsized":"2024-03-21T09:11:49.038543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436189+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:10:47.436189+0000","last_peered":"2024-03-21T09:10:47.436189+0000","last_clean":"2024-03-21T09:10:47.436189+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:10:47.436189+0000","last_undegraded":"2024-03-21T09:10:47.436189+0000","last_fullsized":"2024-03-21T09:10:47.436189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439612+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:10:47.439612+0000","last_peered":"2024-03-21T09:10:47.439612+0000","last_clean":"2024-03-21T09:10:47.439612+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:10:47.439612+0000","last_undegraded":"2024-03-21T09:10:47.439612+0000","last_fullsized":"2024-03-21T09:10:47.439612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408028+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:10:47.408028+0000","last_peered":"2024-03-21T09:10:47.408028+0000","last_clean":"2024-03-21T09:10:47.408028+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:10:47.408028+0000","last_undegraded":"2024-03-21T09:10:47.408028+0000","last_fullsized":"2024-03-21T09:10:47.408028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406881+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:10:47.406881+0000","last_peered":"2024-03-21T09:10:47.406881+0000","last_clean":"2024-03-21T09:10:47.406881+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:10:47.406881+0000","last_undegraded":"2024-03-21T09:10:47.406881+0000","last_fullsized":"2024-03-21T09:10:47.406881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406796+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:10:47.406796+0000","last_peered":"2024-03-21T09:10:47.406796+0000","last_clean":"2024-03-21T09:10:47.406796+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:10:47.406796+0000","last_undegraded":"2024-03-21T09:10:47.406796+0000","last_fullsized":"2024-03-21T09:10:47.406796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409143+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:10:47.409143+0000","last_peered":"2024-03-21T09:10:47.409143+0000","last_clean":"2024-03-21T09:10:47.409143+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:10:47.409143+0000","last_undegraded":"2024-03-21T09:10:47.409143+0000","last_fullsized":"2024-03-21T09:10:47.409143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404402+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:10:47.404402+0000","last_peered":"2024-03-21T09:10:47.404402+0000","last_clean":"2024-03-21T09:10:47.404402+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:10:47.404402+0000","last_undegraded":"2024-03-21T09:10:47.404402+0000","last_fullsized":"2024-03-21T09:10:47.404402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408215+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:10:47.408215+0000","last_peered":"2024-03-21T09:10:47.408215+0000","last_clean":"2024-03-21T09:10:47.408215+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:10:47.408215+0000","last_undegraded":"2024-03-21T09:10:47.408215+0000","last_fullsized":"2024-03-21T09:10:47.408215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408295+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:10:47.408295+0000","last_peered":"2024-03-21T09:10:47.408295+0000","last_clean":"2024-03-21T09:10:47.408295+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:10:47.408295+0000","last_undegraded":"2024-03-21T09:10:47.408295+0000","last_fullsized":"2024-03-21T09:10:47.408295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402804+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:10:47.402804+0000","last_peered":"2024-03-21T09:10:47.402804+0000","last_clean":"2024-03-21T09:10:47.402804+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:10:47.402804+0000","last_undegraded":"2024-03-21T09:10:47.402804+0000","last_fullsized":"2024-03-21T09:10:47.402804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439328+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:10:47.439328+0000","last_peered":"2024-03-21T09:10:47.439328+0000","last_clean":"2024-03-21T09:10:47.439328+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:10:47.439328+0000","last_undegraded":"2024-03-21T09:10:47.439328+0000","last_fullsized":"2024-03-21T09:10:47.439328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436162+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:10:47.436162+0000","last_peered":"2024-03-21T09:10:47.436162+0000","last_clean":"2024-03-21T09:10:47.436162+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:10:47.436162+0000","last_undegraded":"2024-03-21T09:10:47.436162+0000","last_fullsized":"2024-03-21T09:10:47.436162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402598+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:10:47.402598+0000","last_peered":"2024-03-21T09:10:47.402598+0000","last_clean":"2024-03-21T09:10:47.402598+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:10:47.402598+0000","last_undegraded":"2024-03-21T09:10:47.402598+0000","last_fullsized":"2024-03-21T09:10:47.402598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439700+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:10:47.439700+0000","last_peered":"2024-03-21T09:10:47.439700+0000","last_clean":"2024-03-21T09:10:47.439700+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:10:47.439700+0000","last_undegraded":"2024-03-21T09:10:47.439700+0000","last_fullsized":"2024-03-21T09:10:47.439700+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407244+0000","last_change":"2024-03-21T09:09:47.205699+0000","last_active":"2024-03-21T09:10:47.407244+0000","last_peered":"2024-03-21T09:10:47.407244+0000","last_clean":"2024-03-21T09:10:47.407244+0000","last_became_active":"2024-03-21T09:09:47.203657+0000","last_became_peered":"2024-03-21T09:09:47.203657+0000","last_unstale":"2024-03-21T09:10:47.407244+0000","last_undegraded":"2024-03-21T09:10:47.407244+0000","last_fullsized":"2024-03-21T09:10:47.407244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:42.930153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":191,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:52.042829+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:11:52.042829+0000","last_peered":"2024-03-21T09:11:52.042829+0000","last_clean":"2024-03-21T09:11:52.042829+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:11:52.042829+0000","last_undegraded":"2024-03-21T09:11:52.042829+0000","last_fullsized":"2024-03-21T09:11:52.042829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436708+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:10:47.436708+0000","last_peered":"2024-03-21T09:10:47.436708+0000","last_clean":"2024-03-21T09:10:47.436708+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:10:47.436708+0000","last_undegraded":"2024-03-21T09:10:47.436708+0000","last_fullsized":"2024-03-21T09:10:47.436708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409498+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:10:47.409498+0000","last_peered":"2024-03-21T09:10:47.409498+0000","last_clean":"2024-03-21T09:10:47.409498+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:10:47.409498+0000","last_undegraded":"2024-03-21T09:10:47.409498+0000","last_fullsized":"2024-03-21T09:10:47.409498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436333+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:10:47.436333+0000","last_peered":"2024-03-21T09:10:47.436333+0000","last_clean":"2024-03-21T09:10:47.436333+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:10:47.436333+0000","last_undegraded":"2024-03-21T09:10:47.436333+0000","last_fullsized":"2024-03-21T09:10:47.436333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435910+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:10:47.435910+0000","last_peered":"2024-03-21T09:10:47.435910+0000","last_clean":"2024-03-21T09:10:47.435910+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:10:47.435910+0000","last_undegraded":"2024-03-21T09:10:47.435910+0000","last_fullsized":"2024-03-21T09:10:47.435910+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407059+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:10:47.407059+0000","last_peered":"2024-03-21T09:10:47.407059+0000","last_clean":"2024-03-21T09:10:47.407059+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:10:47.407059+0000","last_undegraded":"2024-03-21T09:10:47.407059+0000","last_fullsized":"2024-03-21T09:10:47.407059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438390+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:10:47.438390+0000","last_peered":"2024-03-21T09:10:47.438390+0000","last_clean":"2024-03-21T09:10:47.438390+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:10:47.438390+0000","last_undegraded":"2024-03-21T09:10:47.438390+0000","last_fullsized":"2024-03-21T09:10:47.438390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436241+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:10:47.436241+0000","last_peered":"2024-03-21T09:10:47.436241+0000","last_clean":"2024-03-21T09:10:47.436241+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:10:47.436241+0000","last_undegraded":"2024-03-21T09:10:47.436241+0000","last_fullsized":"2024-03-21T09:10:47.436241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404852+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:10:47.404852+0000","last_peered":"2024-03-21T09:10:47.404852+0000","last_clean":"2024-03-21T09:10:47.404852+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:10:47.404852+0000","last_undegraded":"2024-03-21T09:10:47.404852+0000","last_fullsized":"2024-03-21T09:10:47.404852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409034+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:10:47.409034+0000","last_peered":"2024-03-21T09:10:47.409034+0000","last_clean":"2024-03-21T09:10:47.409034+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:10:47.409034+0000","last_undegraded":"2024-03-21T09:10:47.409034+0000","last_fullsized":"2024-03-21T09:10:47.409034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439473+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:10:47.439473+0000","last_peered":"2024-03-21T09:10:47.439473+0000","last_clean":"2024-03-21T09:10:47.439473+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:10:47.439473+0000","last_undegraded":"2024-03-21T09:10:47.439473+0000","last_fullsized":"2024-03-21T09:10:47.439473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410228+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:10:47.410228+0000","last_peered":"2024-03-21T09:10:47.410228+0000","last_clean":"2024-03-21T09:10:47.410228+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:10:47.410228+0000","last_undegraded":"2024-03-21T09:10:47.410228+0000","last_fullsized":"2024-03-21T09:10:47.410228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:54.894106+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:10:54.894106+0000","last_peered":"2024-03-21T09:10:54.894106+0000","last_clean":"2024-03-21T09:10:54.894106+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:10:54.894106+0000","last_undegraded":"2024-03-21T09:10:54.894106+0000","last_fullsized":"2024-03-21T09:10:54.894106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411304+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:10:47.411304+0000","last_peered":"2024-03-21T09:10:47.411304+0000","last_clean":"2024-03-21T09:10:47.411304+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:10:47.411304+0000","last_undegraded":"2024-03-21T09:10:47.411304+0000","last_fullsized":"2024-03-21T09:10:47.411304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405917+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:10:47.405917+0000","last_peered":"2024-03-21T09:10:47.405917+0000","last_clean":"2024-03-21T09:10:47.405917+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:10:47.405917+0000","last_undegraded":"2024-03-21T09:10:47.405917+0000","last_fullsized":"2024-03-21T09:10:47.405917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436899+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:10:47.436899+0000","last_peered":"2024-03-21T09:10:47.436899+0000","last_clean":"2024-03-21T09:10:47.436899+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:10:47.436899+0000","last_undegraded":"2024-03-21T09:10:47.436899+0000","last_fullsized":"2024-03-21T09:10:47.436899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438375+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:10:47.438375+0000","last_peered":"2024-03-21T09:10:47.438375+0000","last_clean":"2024-03-21T09:10:47.438375+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:10:47.438375+0000","last_undegraded":"2024-03-21T09:10:47.438375+0000","last_fullsized":"2024-03-21T09:10:47.438375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436534+0000","last_change":"2024-03-21T09:09:47.203560+0000","last_active":"2024-03-21T09:10:47.436534+0000","last_peered":"2024-03-21T09:10:47.436534+0000","last_clean":"2024-03-21T09:10:47.436534+0000","last_became_active":"2024-03-21T09:09:47.203072+0000","last_became_peered":"2024-03-21T09:09:47.203072+0000","last_unstale":"2024-03-21T09:10:47.436534+0000","last_undegraded":"2024-03-21T09:10:47.436534+0000","last_fullsized":"2024-03-21T09:10:47.436534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:03:39.037042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406193+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:10:47.406193+0000","last_peered":"2024-03-21T09:10:47.406193+0000","last_clean":"2024-03-21T09:10:47.406193+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:10:47.406193+0000","last_undegraded":"2024-03-21T09:10:47.406193+0000","last_fullsized":"2024-03-21T09:10:47.406193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408090+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:10:47.408090+0000","last_peered":"2024-03-21T09:10:47.408090+0000","last_clean":"2024-03-21T09:10:47.408090+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:10:47.408090+0000","last_undegraded":"2024-03-21T09:10:47.408090+0000","last_fullsized":"2024-03-21T09:10:47.408090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408581+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:10:47.408581+0000","last_peered":"2024-03-21T09:10:47.408581+0000","last_clean":"2024-03-21T09:10:47.408581+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:10:47.408581+0000","last_undegraded":"2024-03-21T09:10:47.408581+0000","last_fullsized":"2024-03-21T09:10:47.408581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409274+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:10:47.409274+0000","last_peered":"2024-03-21T09:10:47.409274+0000","last_clean":"2024-03-21T09:10:47.409274+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:10:47.409274+0000","last_undegraded":"2024-03-21T09:10:47.409274+0000","last_fullsized":"2024-03-21T09:10:47.409274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404093+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:10:47.404093+0000","last_peered":"2024-03-21T09:10:47.404093+0000","last_clean":"2024-03-21T09:10:47.404093+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:10:47.404093+0000","last_undegraded":"2024-03-21T09:10:47.404093+0000","last_fullsized":"2024-03-21T09:10:47.404093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436903+0000","last_change":"2024-03-21T09:09:47.206281+0000","last_active":"2024-03-21T09:10:47.436903+0000","last_peered":"2024-03-21T09:10:47.436903+0000","last_clean":"2024-03-21T09:10:47.436903+0000","last_became_active":"2024-03-21T09:09:47.198427+0000","last_became_peered":"2024-03-21T09:09:47.198427+0000","last_unstale":"2024-03-21T09:10:47.436903+0000","last_undegraded":"2024-03-21T09:10:47.436903+0000","last_fullsized":"2024-03-21T09:10:47.436903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:32:46.182225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406103+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:10:47.406103+0000","last_peered":"2024-03-21T09:10:47.406103+0000","last_clean":"2024-03-21T09:10:47.406103+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:10:47.406103+0000","last_undegraded":"2024-03-21T09:10:47.406103+0000","last_fullsized":"2024-03-21T09:10:47.406103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410703+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:10:47.410703+0000","last_peered":"2024-03-21T09:10:47.410703+0000","last_clean":"2024-03-21T09:10:47.410703+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:10:47.410703+0000","last_undegraded":"2024-03-21T09:10:47.410703+0000","last_fullsized":"2024-03-21T09:10:47.410703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436507+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:10:47.436507+0000","last_peered":"2024-03-21T09:10:47.436507+0000","last_clean":"2024-03-21T09:10:47.436507+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:10:47.436507+0000","last_undegraded":"2024-03-21T09:10:47.436507+0000","last_fullsized":"2024-03-21T09:10:47.436507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408535+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:10:47.408535+0000","last_peered":"2024-03-21T09:10:47.408535+0000","last_clean":"2024-03-21T09:10:47.408535+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:10:47.408535+0000","last_undegraded":"2024-03-21T09:10:47.408535+0000","last_fullsized":"2024-03-21T09:10:47.408535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436195+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:10:47.436195+0000","last_peered":"2024-03-21T09:10:47.436195+0000","last_clean":"2024-03-21T09:10:47.436195+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:10:47.436195+0000","last_undegraded":"2024-03-21T09:10:47.436195+0000","last_fullsized":"2024-03-21T09:10:47.436195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406267+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:10:47.406267+0000","last_peered":"2024-03-21T09:10:47.406267+0000","last_clean":"2024-03-21T09:10:47.406267+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:10:47.406267+0000","last_undegraded":"2024-03-21T09:10:47.406267+0000","last_fullsized":"2024-03-21T09:10:47.406267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406667+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:10:47.406667+0000","last_peered":"2024-03-21T09:10:47.406667+0000","last_clean":"2024-03-21T09:10:47.406667+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:10:47.406667+0000","last_undegraded":"2024-03-21T09:10:47.406667+0000","last_fullsized":"2024-03-21T09:10:47.406667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436775+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:10:47.436775+0000","last_peered":"2024-03-21T09:10:47.436775+0000","last_clean":"2024-03-21T09:10:47.436775+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:10:47.436775+0000","last_undegraded":"2024-03-21T09:10:47.436775+0000","last_fullsized":"2024-03-21T09:10:47.436775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437679+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:10:47.437679+0000","last_peered":"2024-03-21T09:10:47.437679+0000","last_clean":"2024-03-21T09:10:47.437679+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:10:47.437679+0000","last_undegraded":"2024-03-21T09:10:47.437679+0000","last_fullsized":"2024-03-21T09:10:47.437679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407624+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:10:47.407624+0000","last_peered":"2024-03-21T09:10:47.407624+0000","last_clean":"2024-03-21T09:10:47.407624+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:10:47.407624+0000","last_undegraded":"2024-03-21T09:10:47.407624+0000","last_fullsized":"2024-03-21T09:10:47.407624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436680+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:10:47.436680+0000","last_peered":"2024-03-21T09:10:47.436680+0000","last_clean":"2024-03-21T09:10:47.436680+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:10:47.436680+0000","last_undegraded":"2024-03-21T09:10:47.436680+0000","last_fullsized":"2024-03-21T09:10:47.436680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406417+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:10:47.406417+0000","last_peered":"2024-03-21T09:10:47.406417+0000","last_clean":"2024-03-21T09:10:47.406417+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:10:47.406417+0000","last_undegraded":"2024-03-21T09:10:47.406417+0000","last_fullsized":"2024-03-21T09:10:47.406417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439536+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:10:47.439536+0000","last_peered":"2024-03-21T09:10:47.439536+0000","last_clean":"2024-03-21T09:10:47.439536+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:10:47.439536+0000","last_undegraded":"2024-03-21T09:10:47.439536+0000","last_fullsized":"2024-03-21T09:10:47.439536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436796+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:10:47.436796+0000","last_peered":"2024-03-21T09:10:47.436796+0000","last_clean":"2024-03-21T09:10:47.436796+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:10:47.436796+0000","last_undegraded":"2024-03-21T09:10:47.436796+0000","last_fullsized":"2024-03-21T09:10:47.436796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404217+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:10:47.404217+0000","last_peered":"2024-03-21T09:10:47.404217+0000","last_clean":"2024-03-21T09:10:47.404217+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:10:47.404217+0000","last_undegraded":"2024-03-21T09:10:47.404217+0000","last_fullsized":"2024-03-21T09:10:47.404217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407361+0000","last_change":"2024-03-21T09:09:47.200898+0000","last_active":"2024-03-21T09:10:47.407361+0000","last_peered":"2024-03-21T09:10:47.407361+0000","last_clean":"2024-03-21T09:10:47.407361+0000","last_became_active":"2024-03-21T09:09:47.200548+0000","last_became_peered":"2024-03-21T09:09:47.200548+0000","last_unstale":"2024-03-21T09:10:47.407361+0000","last_undegraded":"2024-03-21T09:10:47.407361+0000","last_fullsized":"2024-03-21T09:10:47.407361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:29.793107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405957+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:10:47.405957+0000","last_peered":"2024-03-21T09:10:47.405957+0000","last_clean":"2024-03-21T09:10:47.405957+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:10:47.405957+0000","last_undegraded":"2024-03-21T09:10:47.405957+0000","last_fullsized":"2024-03-21T09:10:47.405957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437955+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:10:47.437955+0000","last_peered":"2024-03-21T09:10:47.437955+0000","last_clean":"2024-03-21T09:10:47.437955+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:10:47.437955+0000","last_undegraded":"2024-03-21T09:10:47.437955+0000","last_fullsized":"2024-03-21T09:10:47.437955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439688+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:10:47.439688+0000","last_peered":"2024-03-21T09:10:47.439688+0000","last_clean":"2024-03-21T09:10:47.439688+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:10:47.439688+0000","last_undegraded":"2024-03-21T09:10:47.439688+0000","last_fullsized":"2024-03-21T09:10:47.439688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404565+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:10:47.404565+0000","last_peered":"2024-03-21T09:10:47.404565+0000","last_clean":"2024-03-21T09:10:47.404565+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:10:47.404565+0000","last_undegraded":"2024-03-21T09:10:47.404565+0000","last_fullsized":"2024-03-21T09:10:47.404565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407263+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:10:47.407263+0000","last_peered":"2024-03-21T09:10:47.407263+0000","last_clean":"2024-03-21T09:10:47.407263+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:10:47.407263+0000","last_undegraded":"2024-03-21T09:10:47.407263+0000","last_fullsized":"2024-03-21T09:10:47.407263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411684+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:10:47.411684+0000","last_peered":"2024-03-21T09:10:47.411684+0000","last_clean":"2024-03-21T09:10:47.411684+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:10:47.411684+0000","last_undegraded":"2024-03-21T09:10:47.411684+0000","last_fullsized":"2024-03-21T09:10:47.411684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406548+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:10:47.406548+0000","last_peered":"2024-03-21T09:10:47.406548+0000","last_clean":"2024-03-21T09:10:47.406548+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:10:47.406548+0000","last_undegraded":"2024-03-21T09:10:47.406548+0000","last_fullsized":"2024-03-21T09:10:47.406548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409488+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:10:47.409488+0000","last_peered":"2024-03-21T09:10:47.409488+0000","last_clean":"2024-03-21T09:10:47.409488+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:10:47.409488+0000","last_undegraded":"2024-03-21T09:10:47.409488+0000","last_fullsized":"2024-03-21T09:10:47.409488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406502+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:10:47.406502+0000","last_peered":"2024-03-21T09:10:47.406502+0000","last_clean":"2024-03-21T09:10:47.406502+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:10:47.406502+0000","last_undegraded":"2024-03-21T09:10:47.406502+0000","last_fullsized":"2024-03-21T09:10:47.406502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439038+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:10:47.439038+0000","last_peered":"2024-03-21T09:10:47.439038+0000","last_clean":"2024-03-21T09:10:47.439038+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:10:47.439038+0000","last_undegraded":"2024-03-21T09:10:47.439038+0000","last_fullsized":"2024-03-21T09:10:47.439038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404751+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:10:47.404751+0000","last_peered":"2024-03-21T09:10:47.404751+0000","last_clean":"2024-03-21T09:10:47.404751+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:10:47.404751+0000","last_undegraded":"2024-03-21T09:10:47.404751+0000","last_fullsized":"2024-03-21T09:10:47.404751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75750,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":944,"num_read_kb":553,"num_write":177,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":222210,"data_compressed":5499,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351029,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":526262,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":1187212,"data_compressed":64715,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808895,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":526464,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":526384,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332170,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1123131,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":521640,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":589941,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":523028,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":585493,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":521834,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":527128,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1117490,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68391,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69707,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73718,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:11:55.758 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-21T09:11:55.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T09:11:56.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.017+0000 7f2633e00640 1 -- 172.21.15.37:0/2995900152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2634103170 msgr2=0x7f2634109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.017+0000 7f2633e00640 1 --2- 172.21.15.37:0/2995900152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2634103170 0x7f2634109a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f262c009ca0 tx=0x7f262c02c100 comp rx=0 tx=0).stop 2024-03-21T09:11:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.018+0000 7f2633e00640 1 -- 172.21.15.37:0/2995900152 shutdown_connections 2024-03-21T09:11:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.018+0000 7f2633e00640 1 --2- 172.21.15.37:0/2995900152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2634103170 0x7f2634109a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.018+0000 7f2633e00640 1 --2- 172.21.15.37:0/2995900152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.018+0000 7f2633e00640 1 --2- 172.21.15.37:0/2995900152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26341015c0 0x7f26341019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.018+0000 7f2633e00640 1 -- 172.21.15.37:0/2995900152 >> 172.21.15.37:0/2995900152 conn(0x7f26340fcb30 msgr2=0x7f26340fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.018+0000 7f2633e00640 1 -- 172.21.15.37:0/2995900152 shutdown_connections 2024-03-21T09:11:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.019+0000 7f2633e00640 1 -- 172.21.15.37:0/2995900152 wait complete. 2024-03-21T09:11:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 Processor -- start 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 -- start start 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 0x7f2634075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2634103170 0x7f2634073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263410d220 con 0x7f2634103170 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263410d0a0 con 0x7f26341015c0 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263410d3a0 con 0x7f26341027b0 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2632a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 0x7f2634075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2632000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2632a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 0x7f2634075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50428/0 (socket says 172.21.15.37:50428) 2024-03-21T09:11:56.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2633400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2634103170 0x7f2634073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2632000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57160/0 (socket says 172.21.15.37:57160) 2024-03-21T09:11:56.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.020+0000 7f2632a00640 1 -- 172.21.15.37:0/750101124 learned_addr learned my addr 172.21.15.37:0/750101124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632a00640 1 -- 172.21.15.37:0/750101124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 msgr2=0x7f2634073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632a00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632a00640 1 -- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2634103170 msgr2=0x7f2634073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632a00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2634103170 0x7f2634073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632a00640 1 -- 172.21.15.37:0/750101124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634074860 con 0x7f26341015c0 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632000640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2632a00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 0x7f2634075450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f262400d770 tx=0x7f262400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624017070 con 0x7f26341015c0 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26341a1920 con 0x7f26341015c0 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.021+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2624009c30 con 0x7f26341015c0 2024-03-21T09:11:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.022+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624003dd0 con 0x7f26341015c0 2024-03-21T09:11:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.022+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26341a1e70 con 0x7f26341015c0 2024-03-21T09:11:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.023+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263404f470 con 0x7f26341015c0 2024-03-21T09:11:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.024+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2624004020 con 0x7f26341015c0 2024-03-21T09:11:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.026+0000 7f262b400640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f26180a2fa0 0x7f26180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.026+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26240cf270 con 0x7f26341015c0 2024-03-21T09:11:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.026+0000 7f2632000640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f26180a2fa0 0x7f26180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.027+0000 7f2632000640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f26180a2fa0 0x7f26180a5460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f26340751c0 tx=0x7f2620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:56.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.030+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2624009de0 con 0x7f26341015c0 2024-03-21T09:11:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.230+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2634061ae0 con 0x7f26180a2fa0 2024-03-21T09:11:56.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.260+0000 7f262b400640 1 -- 172.21.15.37:0/750101124 <== mgr.14152 v2:172.21.15.37:6800/1300592626 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340837 (secure 0 0 0) 0x7f2634061ae0 con 0x7f26180a2fa0 2024-03-21T09:11:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:56.260 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f26180a2fa0 msgr2=0x7f26180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f26180a2fa0 0x7f26180a5460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f26340751c0 tx=0x7f2620005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 msgr2=0x7f2634075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 0x7f2634075450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f262400d770 tx=0x7f262400dc30 comp rx=0 tx=0).stop 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 shutdown_connections 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f26180a2fa0 0x7f26180a5460 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2634103170 0x7f2634073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26341027b0 0x7f2634073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 --2- 172.21.15.37:0/750101124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26341015c0 0x7f2634075450 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 >> 172.21.15.37:0/750101124 conn(0x7f26340fcb30 msgr2=0x7f2634105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 shutdown_connections 2024-03-21T09:11:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.267+0000 7f2633e00640 1 -- 172.21.15.37:0/750101124 wait complete. 2024-03-21T09:11:56.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:56 smithi037 ceph-mon[31830]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:56.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:56 smithi037 ceph-mon[31830]: from='client.25480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:56.281 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-03-21T09:11:55.850416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666352,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1086,"num_read_kb":673,"num_write":364,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382580,"kb_used_data":12600,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559244,"statfs":{"total":1151940427776,"available":1151548665856,"internally_reserved":0,"allocated":12902400,"data_stored":8284682,"data_compressed":433446,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9287,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":42,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004914"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404943+0000","last_change":"2024-03-21T09:08:46.851482+0000","last_active":"2024-03-21T09:10:47.404943+0000","last_peered":"2024-03-21T09:10:47.404943+0000","last_clean":"2024-03-21T09:10:47.404943+0000","last_became_active":"2024-03-21T09:08:46.851095+0000","last_became_peered":"2024-03-21T09:08:46.851095+0000","last_unstale":"2024-03-21T09:10:47.404943+0000","last_undegraded":"2024-03-21T09:10:47.404943+0000","last_fullsized":"2024-03-21T09:10:47.404943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:06:17.044618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438508+0000","last_change":"2024-03-21T09:08:46.851664+0000","last_active":"2024-03-21T09:10:47.438508+0000","last_peered":"2024-03-21T09:10:47.438508+0000","last_clean":"2024-03-21T09:10:47.438508+0000","last_became_active":"2024-03-21T09:08:46.849688+0000","last_became_peered":"2024-03-21T09:08:46.849688+0000","last_unstale":"2024-03-21T09:10:47.438508+0000","last_undegraded":"2024-03-21T09:10:47.438508+0000","last_fullsized":"2024-03-21T09:10:47.438508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:22.890657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437937+0000","last_change":"2024-03-21T09:08:46.851178+0000","last_active":"2024-03-21T09:10:47.437937+0000","last_peered":"2024-03-21T09:10:47.437937+0000","last_clean":"2024-03-21T09:10:47.437937+0000","last_became_active":"2024-03-21T09:08:46.850072+0000","last_became_peered":"2024-03-21T09:08:46.850072+0000","last_unstale":"2024-03-21T09:10:47.437937+0000","last_undegraded":"2024-03-21T09:10:47.437937+0000","last_fullsized":"2024-03-21T09:10:47.437937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:23:45.290805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409123+0000","last_change":"2024-03-21T09:09:47.203174+0000","last_active":"2024-03-21T09:10:47.409123+0000","last_peered":"2024-03-21T09:10:47.409123+0000","last_clean":"2024-03-21T09:10:47.409123+0000","last_became_active":"2024-03-21T09:09:47.202519+0000","last_became_peered":"2024-03-21T09:09:47.202519+0000","last_unstale":"2024-03-21T09:10:47.409123+0000","last_undegraded":"2024-03-21T09:10:47.409123+0000","last_fullsized":"2024-03-21T09:10:47.409123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:06:48.445311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408507+0000","last_change":"2024-03-21T09:08:46.842512+0000","last_active":"2024-03-21T09:10:47.408507+0000","last_peered":"2024-03-21T09:10:47.408507+0000","last_clean":"2024-03-21T09:10:47.408507+0000","last_became_active":"2024-03-21T09:08:46.842050+0000","last_became_peered":"2024-03-21T09:08:46.842050+0000","last_unstale":"2024-03-21T09:10:47.408507+0000","last_undegraded":"2024-03-21T09:10:47.408507+0000","last_fullsized":"2024-03-21T09:10:47.408507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.800465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402461+0000","last_change":"2024-03-21T09:08:46.827504+0000","last_active":"2024-03-21T09:10:47.402461+0000","last_peered":"2024-03-21T09:10:47.402461+0000","last_clean":"2024-03-21T09:10:47.402461+0000","last_became_active":"2024-03-21T09:08:46.826757+0000","last_became_peered":"2024-03-21T09:08:46.826757+0000","last_unstale":"2024-03-21T09:10:47.402461+0000","last_undegraded":"2024-03-21T09:10:47.402461+0000","last_fullsized":"2024-03-21T09:10:47.402461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:48:35.230572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436131+0000","last_change":"2024-03-21T09:08:46.826631+0000","last_active":"2024-03-21T09:10:47.436131+0000","last_peered":"2024-03-21T09:10:47.436131+0000","last_clean":"2024-03-21T09:10:47.436131+0000","last_became_active":"2024-03-21T09:08:46.826129+0000","last_became_peered":"2024-03-21T09:08:46.826129+0000","last_unstale":"2024-03-21T09:10:47.436131+0000","last_undegraded":"2024-03-21T09:10:47.436131+0000","last_fullsized":"2024-03-21T09:10:47.436131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:29.351974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406994+0000","last_change":"2024-03-21T09:09:47.204487+0000","last_active":"2024-03-21T09:10:47.406994+0000","last_peered":"2024-03-21T09:10:47.406994+0000","last_clean":"2024-03-21T09:10:47.406994+0000","last_became_active":"2024-03-21T09:09:47.202961+0000","last_became_peered":"2024-03-21T09:09:47.202961+0000","last_unstale":"2024-03-21T09:10:47.406994+0000","last_undegraded":"2024-03-21T09:10:47.406994+0000","last_fullsized":"2024-03-21T09:10:47.406994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:50.524550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406889+0000","last_change":"2024-03-21T09:08:46.818427+0000","last_active":"2024-03-21T09:10:47.406889+0000","last_peered":"2024-03-21T09:10:47.406889+0000","last_clean":"2024-03-21T09:10:47.406889+0000","last_became_active":"2024-03-21T09:08:46.817945+0000","last_became_peered":"2024-03-21T09:08:46.817945+0000","last_unstale":"2024-03-21T09:10:47.406889+0000","last_undegraded":"2024-03-21T09:10:47.406889+0000","last_fullsized":"2024-03-21T09:10:47.406889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:42.267060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402825+0000","last_change":"2024-03-21T09:08:46.821244+0000","last_active":"2024-03-21T09:10:47.402825+0000","last_peered":"2024-03-21T09:10:47.402825+0000","last_clean":"2024-03-21T09:10:47.402825+0000","last_became_active":"2024-03-21T09:08:46.820896+0000","last_became_peered":"2024-03-21T09:08:46.820896+0000","last_unstale":"2024-03-21T09:10:47.402825+0000","last_undegraded":"2024-03-21T09:10:47.402825+0000","last_fullsized":"2024-03-21T09:10:47.402825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:16.276475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436663+0000","last_change":"2024-03-21T09:08:46.824900+0000","last_active":"2024-03-21T09:10:47.436663+0000","last_peered":"2024-03-21T09:10:47.436663+0000","last_clean":"2024-03-21T09:10:47.436663+0000","last_became_active":"2024-03-21T09:08:46.824530+0000","last_became_peered":"2024-03-21T09:08:46.824530+0000","last_unstale":"2024-03-21T09:10:47.436663+0000","last_undegraded":"2024-03-21T09:10:47.436663+0000","last_fullsized":"2024-03-21T09:10:47.436663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:49:39.304798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436967+0000","last_change":"2024-03-21T09:08:46.820457+0000","last_active":"2024-03-21T09:10:47.436967+0000","last_peered":"2024-03-21T09:10:47.436967+0000","last_clean":"2024-03-21T09:10:47.436967+0000","last_became_active":"2024-03-21T09:08:46.820076+0000","last_became_peered":"2024-03-21T09:08:46.820076+0000","last_unstale":"2024-03-21T09:10:47.436967+0000","last_undegraded":"2024-03-21T09:10:47.436967+0000","last_fullsized":"2024-03-21T09:10:47.436967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:37:59.857837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439147+0000","last_change":"2024-03-21T09:08:46.831914+0000","last_active":"2024-03-21T09:10:47.439147+0000","last_peered":"2024-03-21T09:10:47.439147+0000","last_clean":"2024-03-21T09:10:47.439147+0000","last_became_active":"2024-03-21T09:08:46.831522+0000","last_became_peered":"2024-03-21T09:08:46.831522+0000","last_unstale":"2024-03-21T09:10:47.439147+0000","last_undegraded":"2024-03-21T09:10:47.439147+0000","last_fullsized":"2024-03-21T09:10:47.439147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:44:50.812096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437049+0000","last_change":"2024-03-21T09:08:46.842669+0000","last_active":"2024-03-21T09:10:47.437049+0000","last_peered":"2024-03-21T09:10:47.437049+0000","last_clean":"2024-03-21T09:10:47.437049+0000","last_became_active":"2024-03-21T09:08:46.840180+0000","last_became_peered":"2024-03-21T09:08:46.840180+0000","last_unstale":"2024-03-21T09:10:47.437049+0000","last_undegraded":"2024-03-21T09:10:47.437049+0000","last_fullsized":"2024-03-21T09:10:47.437049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:52:23.403786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437308+0000","last_change":"2024-03-21T09:08:46.839719+0000","last_active":"2024-03-21T09:10:47.437308+0000","last_peered":"2024-03-21T09:10:47.437308+0000","last_clean":"2024-03-21T09:10:47.437308+0000","last_became_active":"2024-03-21T09:08:46.839343+0000","last_became_peered":"2024-03-21T09:08:46.839343+0000","last_unstale":"2024-03-21T09:10:47.437308+0000","last_undegraded":"2024-03-21T09:10:47.437308+0000","last_fullsized":"2024-03-21T09:10:47.437308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:35.027274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406169+0000","last_change":"2024-03-21T09:08:46.841361+0000","last_active":"2024-03-21T09:10:47.406169+0000","last_peered":"2024-03-21T09:10:47.406169+0000","last_clean":"2024-03-21T09:10:47.406169+0000","last_became_active":"2024-03-21T09:08:46.840985+0000","last_became_peered":"2024-03-21T09:08:46.840985+0000","last_unstale":"2024-03-21T09:10:47.406169+0000","last_undegraded":"2024-03-21T09:10:47.406169+0000","last_fullsized":"2024-03-21T09:10:47.406169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:21.017721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404683+0000","last_change":"2024-03-21T09:08:46.851783+0000","last_active":"2024-03-21T09:10:47.404683+0000","last_peered":"2024-03-21T09:10:47.404683+0000","last_clean":"2024-03-21T09:10:47.404683+0000","last_became_active":"2024-03-21T09:08:46.851054+0000","last_became_peered":"2024-03-21T09:08:46.851054+0000","last_unstale":"2024-03-21T09:10:47.404683+0000","last_undegraded":"2024-03-21T09:10:47.404683+0000","last_fullsized":"2024-03-21T09:10:47.404683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:07.404089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408836+0000","last_change":"2024-03-21T09:08:46.821032+0000","last_active":"2024-03-21T09:10:47.408836+0000","last_peered":"2024-03-21T09:10:47.408836+0000","last_clean":"2024-03-21T09:10:47.408836+0000","last_became_active":"2024-03-21T09:08:46.818886+0000","last_became_peered":"2024-03-21T09:08:46.818886+0000","last_unstale":"2024-03-21T09:10:47.408836+0000","last_undegraded":"2024-03-21T09:10:47.408836+0000","last_fullsized":"2024-03-21T09:10:47.408836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:15:07.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406741+0000","last_change":"2024-03-21T09:08:46.837370+0000","last_active":"2024-03-21T09:10:47.406741+0000","last_peered":"2024-03-21T09:10:47.406741+0000","last_clean":"2024-03-21T09:10:47.406741+0000","last_became_active":"2024-03-21T09:08:46.837075+0000","last_became_peered":"2024-03-21T09:08:46.837075+0000","last_unstale":"2024-03-21T09:10:47.406741+0000","last_undegraded":"2024-03-21T09:10:47.406741+0000","last_fullsized":"2024-03-21T09:10:47.406741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:25.914035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406108+0000","last_change":"2024-03-21T09:08:50.970842+0000","last_active":"2024-03-21T09:10:47.406108+0000","last_peered":"2024-03-21T09:10:47.406108+0000","last_clean":"2024-03-21T09:10:47.406108+0000","last_became_active":"2024-03-21T09:08:50.970450+0000","last_became_peered":"2024-03-21T09:08:50.970450+0000","last_unstale":"2024-03-21T09:10:47.406108+0000","last_undegraded":"2024-03-21T09:10:47.406108+0000","last_fullsized":"2024-03-21T09:10:47.406108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:23:50.338152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408966+0000","last_change":"2024-03-21T09:08:45.819219+0000","last_active":"2024-03-21T09:10:47.408966+0000","last_peered":"2024-03-21T09:10:47.408966+0000","last_clean":"2024-03-21T09:10:47.408966+0000","last_became_active":"2024-03-21T09:08:45.817658+0000","last_became_peered":"2024-03-21T09:08:45.817658+0000","last_unstale":"2024-03-21T09:10:47.408966+0000","last_undegraded":"2024-03-21T09:10:47.408966+0000","last_fullsized":"2024-03-21T09:10:47.408966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:46.345413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411324+0000","last_change":"2024-03-21T09:08:46.846154+0000","last_active":"2024-03-21T09:10:47.411324+0000","last_peered":"2024-03-21T09:10:47.411324+0000","last_clean":"2024-03-21T09:10:47.411324+0000","last_became_active":"2024-03-21T09:08:46.845112+0000","last_became_peered":"2024-03-21T09:08:46.845112+0000","last_unstale":"2024-03-21T09:10:47.411324+0000","last_undegraded":"2024-03-21T09:10:47.411324+0000","last_fullsized":"2024-03-21T09:10:47.411324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:26.150968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435847+0000","last_change":"2024-03-21T09:08:50.985986+0000","last_active":"2024-03-21T09:10:47.435847+0000","last_peered":"2024-03-21T09:10:47.435847+0000","last_clean":"2024-03-21T09:10:47.435847+0000","last_became_active":"2024-03-21T09:08:50.985571+0000","last_became_peered":"2024-03-21T09:08:50.985571+0000","last_unstale":"2024-03-21T09:10:47.435847+0000","last_undegraded":"2024-03-21T09:10:47.435847+0000","last_fullsized":"2024-03-21T09:10:47.435847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:12:29.843619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411396+0000","last_change":"2024-03-21T09:08:45.834386+0000","last_active":"2024-03-21T09:10:47.411396+0000","last_peered":"2024-03-21T09:10:47.411396+0000","last_clean":"2024-03-21T09:10:47.411396+0000","last_became_active":"2024-03-21T09:08:45.834029+0000","last_became_peered":"2024-03-21T09:08:45.834029+0000","last_unstale":"2024-03-21T09:10:47.411396+0000","last_undegraded":"2024-03-21T09:10:47.411396+0000","last_fullsized":"2024-03-21T09:10:47.411396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:00:05.649675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439238+0000","last_change":"2024-03-21T09:08:46.852117+0000","last_active":"2024-03-21T09:10:47.439238+0000","last_peered":"2024-03-21T09:10:47.439238+0000","last_clean":"2024-03-21T09:10:47.439238+0000","last_became_active":"2024-03-21T09:08:46.851394+0000","last_became_peered":"2024-03-21T09:08:46.851394+0000","last_unstale":"2024-03-21T09:10:47.439238+0000","last_undegraded":"2024-03-21T09:10:47.439238+0000","last_fullsized":"2024-03-21T09:10:47.439238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:56:27.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436274+0000","last_change":"2024-03-21T09:08:50.974515+0000","last_active":"2024-03-21T09:10:47.436274+0000","last_peered":"2024-03-21T09:10:47.436274+0000","last_clean":"2024-03-21T09:10:47.436274+0000","last_became_active":"2024-03-21T09:08:50.974006+0000","last_became_peered":"2024-03-21T09:08:50.974006+0000","last_unstale":"2024-03-21T09:10:47.436274+0000","last_undegraded":"2024-03-21T09:10:47.436274+0000","last_fullsized":"2024-03-21T09:10:47.436274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:32:35.737594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409035+0000","last_change":"2024-03-21T09:08:45.816851+0000","last_active":"2024-03-21T09:10:47.409035+0000","last_peered":"2024-03-21T09:10:47.409035+0000","last_clean":"2024-03-21T09:10:47.409035+0000","last_became_active":"2024-03-21T09:08:45.816528+0000","last_became_peered":"2024-03-21T09:08:45.816528+0000","last_unstale":"2024-03-21T09:10:47.409035+0000","last_undegraded":"2024-03-21T09:10:47.409035+0000","last_fullsized":"2024-03-21T09:10:47.409035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:29.524162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406668+0000","last_change":"2024-03-21T09:08:46.837753+0000","last_active":"2024-03-21T09:10:47.406668+0000","last_peered":"2024-03-21T09:10:47.406668+0000","last_clean":"2024-03-21T09:10:47.406668+0000","last_became_active":"2024-03-21T09:08:46.837322+0000","last_became_peered":"2024-03-21T09:08:46.837322+0000","last_unstale":"2024-03-21T09:10:47.406668+0000","last_undegraded":"2024-03-21T09:10:47.406668+0000","last_fullsized":"2024-03-21T09:10:47.406668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:01:09.218005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436234+0000","last_change":"2024-03-21T09:08:50.984521+0000","last_active":"2024-03-21T09:10:47.436234+0000","last_peered":"2024-03-21T09:10:47.436234+0000","last_clean":"2024-03-21T09:10:47.436234+0000","last_became_active":"2024-03-21T09:08:50.983945+0000","last_became_peered":"2024-03-21T09:08:50.983945+0000","last_unstale":"2024-03-21T09:10:47.436234+0000","last_undegraded":"2024-03-21T09:10:47.436234+0000","last_fullsized":"2024-03-21T09:10:47.436234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:54.497497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411729+0000","last_change":"2024-03-21T09:10:47.411335+0000","last_active":"2024-03-21T09:10:47.411729+0000","last_peered":"2024-03-21T09:10:47.411729+0000","last_clean":"2024-03-21T09:10:47.411729+0000","last_became_active":"2024-03-21T09:10:47.408103+0000","last_became_peered":"2024-03-21T09:10:47.408103+0000","last_unstale":"2024-03-21T09:10:47.411729+0000","last_undegraded":"2024-03-21T09:10:47.411729+0000","last_fullsized":"2024-03-21T09:10:47.411729+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:51.600076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407587+0000","last_change":"2024-03-21T09:09:47.433837+0000","last_active":"2024-03-21T09:10:47.407587+0000","last_peered":"2024-03-21T09:10:47.407587+0000","last_clean":"2024-03-21T09:10:47.407587+0000","last_became_active":"2024-03-21T09:09:47.433538+0000","last_became_peered":"2024-03-21T09:09:47.433538+0000","last_unstale":"2024-03-21T09:10:47.407587+0000","last_undegraded":"2024-03-21T09:10:47.407587+0000","last_fullsized":"2024-03-21T09:10:47.407587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:22:41.858858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406895+0000","last_change":"2024-03-21T09:08:50.955867+0000","last_active":"2024-03-21T09:10:47.406895+0000","last_peered":"2024-03-21T09:10:47.406895+0000","last_clean":"2024-03-21T09:10:47.406895+0000","last_became_active":"2024-03-21T09:08:50.955448+0000","last_became_peered":"2024-03-21T09:08:50.955448+0000","last_unstale":"2024-03-21T09:10:47.406895+0000","last_undegraded":"2024-03-21T09:10:47.406895+0000","last_fullsized":"2024-03-21T09:10:47.406895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:45:23.827958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:51.961475+0000","last_change":"2024-03-21T09:08:45.812142+0000","last_active":"2024-03-21T09:11:51.961475+0000","last_peered":"2024-03-21T09:11:51.961475+0000","last_clean":"2024-03-21T09:11:51.961475+0000","last_became_active":"2024-03-21T09:08:45.811786+0000","last_became_peered":"2024-03-21T09:08:45.811786+0000","last_unstale":"2024-03-21T09:11:51.961475+0000","last_undegraded":"2024-03-21T09:11:51.961475+0000","last_fullsized":"2024-03-21T09:11:51.961475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:46.277658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408270+0000","last_change":"2024-03-21T09:08:46.827849+0000","last_active":"2024-03-21T09:10:47.408270+0000","last_peered":"2024-03-21T09:10:47.408270+0000","last_clean":"2024-03-21T09:10:47.408270+0000","last_became_active":"2024-03-21T09:08:46.827519+0000","last_became_peered":"2024-03-21T09:08:46.827519+0000","last_unstale":"2024-03-21T09:10:47.408270+0000","last_undegraded":"2024-03-21T09:10:47.408270+0000","last_fullsized":"2024-03-21T09:10:47.408270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:00:44.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406440+0000","last_change":"2024-03-21T09:08:50.984030+0000","last_active":"2024-03-21T09:10:47.406440+0000","last_peered":"2024-03-21T09:10:47.406440+0000","last_clean":"2024-03-21T09:10:47.406440+0000","last_became_active":"2024-03-21T09:08:50.983519+0000","last_became_peered":"2024-03-21T09:08:50.983519+0000","last_unstale":"2024-03-21T09:10:47.406440+0000","last_undegraded":"2024-03-21T09:10:47.406440+0000","last_fullsized":"2024-03-21T09:10:47.406440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:46.660033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402428+0000","last_change":"2024-03-21T09:08:45.817875+0000","last_active":"2024-03-21T09:10:47.402428+0000","last_peered":"2024-03-21T09:10:47.402428+0000","last_clean":"2024-03-21T09:10:47.402428+0000","last_became_active":"2024-03-21T09:08:45.817431+0000","last_became_peered":"2024-03-21T09:08:45.817431+0000","last_unstale":"2024-03-21T09:10:47.402428+0000","last_undegraded":"2024-03-21T09:10:47.402428+0000","last_fullsized":"2024-03-21T09:10:47.402428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:57:18.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407441+0000","last_change":"2024-03-21T09:08:46.844182+0000","last_active":"2024-03-21T09:10:47.407441+0000","last_peered":"2024-03-21T09:10:47.407441+0000","last_clean":"2024-03-21T09:10:47.407441+0000","last_became_active":"2024-03-21T09:08:46.843755+0000","last_became_peered":"2024-03-21T09:08:46.843755+0000","last_unstale":"2024-03-21T09:10:47.407441+0000","last_undegraded":"2024-03-21T09:10:47.407441+0000","last_fullsized":"2024-03-21T09:10:47.407441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:16:10.024239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404654+0000","last_change":"2024-03-21T09:09:47.432250+0000","last_active":"2024-03-21T09:10:47.404654+0000","last_peered":"2024-03-21T09:10:47.404654+0000","last_clean":"2024-03-21T09:10:47.404654+0000","last_became_active":"2024-03-21T09:09:47.431784+0000","last_became_peered":"2024-03-21T09:09:47.431784+0000","last_unstale":"2024-03-21T09:10:47.404654+0000","last_undegraded":"2024-03-21T09:10:47.404654+0000","last_fullsized":"2024-03-21T09:10:47.404654+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:58.438472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438375+0000","last_change":"2024-03-21T09:08:45.815097+0000","last_active":"2024-03-21T09:10:47.438375+0000","last_peered":"2024-03-21T09:10:47.438375+0000","last_clean":"2024-03-21T09:10:47.438375+0000","last_became_active":"2024-03-21T09:08:45.814592+0000","last_became_peered":"2024-03-21T09:08:45.814592+0000","last_unstale":"2024-03-21T09:10:47.438375+0000","last_undegraded":"2024-03-21T09:10:47.438375+0000","last_fullsized":"2024-03-21T09:10:47.438375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:21.223711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410399+0000","last_change":"2024-03-21T09:08:46.842000+0000","last_active":"2024-03-21T09:10:47.410399+0000","last_peered":"2024-03-21T09:10:47.410399+0000","last_clean":"2024-03-21T09:10:47.410399+0000","last_became_active":"2024-03-21T09:08:46.841508+0000","last_became_peered":"2024-03-21T09:08:46.841508+0000","last_unstale":"2024-03-21T09:10:47.410399+0000","last_undegraded":"2024-03-21T09:10:47.410399+0000","last_fullsized":"2024-03-21T09:10:47.410399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:36.839914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438232+0000","last_change":"2024-03-21T09:08:50.983993+0000","last_active":"2024-03-21T09:10:47.438232+0000","last_peered":"2024-03-21T09:10:47.438232+0000","last_clean":"2024-03-21T09:10:47.438232+0000","last_became_active":"2024-03-21T09:08:50.982325+0000","last_became_peered":"2024-03-21T09:08:50.982325+0000","last_unstale":"2024-03-21T09:10:47.438232+0000","last_undegraded":"2024-03-21T09:10:47.438232+0000","last_fullsized":"2024-03-21T09:10:47.438232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:04.919504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406786+0000","last_change":"2024-03-21T09:08:45.809764+0000","last_active":"2024-03-21T09:10:47.406786+0000","last_peered":"2024-03-21T09:10:47.406786+0000","last_clean":"2024-03-21T09:10:47.406786+0000","last_became_active":"2024-03-21T09:08:45.809280+0000","last_became_peered":"2024-03-21T09:08:45.809280+0000","last_unstale":"2024-03-21T09:10:47.406786+0000","last_undegraded":"2024-03-21T09:10:47.406786+0000","last_fullsized":"2024-03-21T09:10:47.406786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:43:11.626148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407682+0000","last_change":"2024-03-21T09:09:47.205219+0000","last_active":"2024-03-21T09:10:47.407682+0000","last_peered":"2024-03-21T09:10:47.407682+0000","last_clean":"2024-03-21T09:10:47.407682+0000","last_became_active":"2024-03-21T09:09:47.204655+0000","last_became_peered":"2024-03-21T09:09:47.204655+0000","last_unstale":"2024-03-21T09:10:47.407682+0000","last_undegraded":"2024-03-21T09:10:47.407682+0000","last_fullsized":"2024-03-21T09:10:47.407682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:03:44.407879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409872+0000","last_change":"2024-03-21T09:08:50.966686+0000","last_active":"2024-03-21T09:10:47.409872+0000","last_peered":"2024-03-21T09:10:47.409872+0000","last_clean":"2024-03-21T09:10:47.409872+0000","last_became_active":"2024-03-21T09:08:50.965562+0000","last_became_peered":"2024-03-21T09:08:50.965562+0000","last_unstale":"2024-03-21T09:10:47.409872+0000","last_undegraded":"2024-03-21T09:10:47.409872+0000","last_fullsized":"2024-03-21T09:10:47.409872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:05:04.525968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":186,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:50.960036+0000","last_change":"2024-03-21T09:08:45.813157+0000","last_active":"2024-03-21T09:11:50.960036+0000","last_peered":"2024-03-21T09:11:50.960036+0000","last_clean":"2024-03-21T09:11:50.960036+0000","last_became_active":"2024-03-21T09:08:45.812693+0000","last_became_peered":"2024-03-21T09:08:45.812693+0000","last_unstale":"2024-03-21T09:11:50.960036+0000","last_undegraded":"2024-03-21T09:11:50.960036+0000","last_fullsized":"2024-03-21T09:11:50.960036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:27:40.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406846+0000","last_change":"2024-03-21T09:08:46.821715+0000","last_active":"2024-03-21T09:10:47.406846+0000","last_peered":"2024-03-21T09:10:47.406846+0000","last_clean":"2024-03-21T09:10:47.406846+0000","last_became_active":"2024-03-21T09:08:46.821331+0000","last_became_peered":"2024-03-21T09:08:46.821331+0000","last_unstale":"2024-03-21T09:10:47.406846+0000","last_undegraded":"2024-03-21T09:10:47.406846+0000","last_fullsized":"2024-03-21T09:10:47.406846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:40.148395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407552+0000","last_change":"2024-03-21T09:08:50.970314+0000","last_active":"2024-03-21T09:10:47.407552+0000","last_peered":"2024-03-21T09:10:47.407552+0000","last_clean":"2024-03-21T09:10:47.407552+0000","last_became_active":"2024-03-21T09:08:50.969861+0000","last_became_peered":"2024-03-21T09:08:50.969861+0000","last_unstale":"2024-03-21T09:10:47.407552+0000","last_undegraded":"2024-03-21T09:10:47.407552+0000","last_fullsized":"2024-03-21T09:10:47.407552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.972352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436943+0000","last_change":"2024-03-21T09:08:45.813870+0000","last_active":"2024-03-21T09:10:47.436943+0000","last_peered":"2024-03-21T09:10:47.436943+0000","last_clean":"2024-03-21T09:10:47.436943+0000","last_became_active":"2024-03-21T09:08:45.813498+0000","last_became_peered":"2024-03-21T09:08:45.813498+0000","last_unstale":"2024-03-21T09:10:47.436943+0000","last_undegraded":"2024-03-21T09:10:47.436943+0000","last_fullsized":"2024-03-21T09:10:47.436943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:50.962276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439194+0000","last_change":"2024-03-21T09:08:46.850119+0000","last_active":"2024-03-21T09:10:47.439194+0000","last_peered":"2024-03-21T09:10:47.439194+0000","last_clean":"2024-03-21T09:10:47.439194+0000","last_became_active":"2024-03-21T09:08:46.849735+0000","last_became_peered":"2024-03-21T09:08:46.849735+0000","last_unstale":"2024-03-21T09:10:47.439194+0000","last_undegraded":"2024-03-21T09:10:47.439194+0000","last_fullsized":"2024-03-21T09:10:47.439194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:54:30.753593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436421+0000","last_change":"2024-03-21T09:09:47.283764+0000","last_active":"2024-03-21T09:10:47.436421+0000","last_peered":"2024-03-21T09:10:47.436421+0000","last_clean":"2024-03-21T09:10:47.436421+0000","last_became_active":"2024-03-21T09:09:47.283203+0000","last_became_peered":"2024-03-21T09:09:47.283203+0000","last_unstale":"2024-03-21T09:10:47.436421+0000","last_undegraded":"2024-03-21T09:10:47.436421+0000","last_fullsized":"2024-03-21T09:10:47.436421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:34:55.502176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407507+0000","last_change":"2024-03-21T09:08:45.810228+0000","last_active":"2024-03-21T09:10:47.407507+0000","last_peered":"2024-03-21T09:10:47.407507+0000","last_clean":"2024-03-21T09:10:47.407507+0000","last_became_active":"2024-03-21T09:08:45.809722+0000","last_became_peered":"2024-03-21T09:08:45.809722+0000","last_unstale":"2024-03-21T09:10:47.407507+0000","last_undegraded":"2024-03-21T09:10:47.407507+0000","last_fullsized":"2024-03-21T09:10:47.407507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:12:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406746+0000","last_change":"2024-03-21T09:08:46.821016+0000","last_active":"2024-03-21T09:10:47.406746+0000","last_peered":"2024-03-21T09:10:47.406746+0000","last_clean":"2024-03-21T09:10:47.406746+0000","last_became_active":"2024-03-21T09:08:46.820630+0000","last_became_peered":"2024-03-21T09:08:46.820630+0000","last_unstale":"2024-03-21T09:10:47.406746+0000","last_undegraded":"2024-03-21T09:10:47.406746+0000","last_fullsized":"2024-03-21T09:10:47.406746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:13:10.177283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407596+0000","last_change":"2024-03-21T09:08:50.958984+0000","last_active":"2024-03-21T09:10:47.407596+0000","last_peered":"2024-03-21T09:10:47.407596+0000","last_clean":"2024-03-21T09:10:47.407596+0000","last_became_active":"2024-03-21T09:08:50.958567+0000","last_became_peered":"2024-03-21T09:08:50.958567+0000","last_unstale":"2024-03-21T09:10:47.407596+0000","last_undegraded":"2024-03-21T09:10:47.407596+0000","last_fullsized":"2024-03-21T09:10:47.407596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:51.990893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":385,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:51.962060+0000","last_change":"2024-03-21T09:08:45.806546+0000","last_active":"2024-03-21T09:11:51.962060+0000","last_peered":"2024-03-21T09:11:51.962060+0000","last_clean":"2024-03-21T09:11:51.962060+0000","last_became_active":"2024-03-21T09:08:45.806138+0000","last_became_peered":"2024-03-21T09:08:45.806138+0000","last_unstale":"2024-03-21T09:11:51.962060+0000","last_undegraded":"2024-03-21T09:11:51.962060+0000","last_fullsized":"2024-03-21T09:11:51.962060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:16.987573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70911,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":227,"num_write":39,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408857+0000","last_change":"2024-03-21T09:08:46.820010+0000","last_active":"2024-03-21T09:10:47.408857+0000","last_peered":"2024-03-21T09:10:47.408857+0000","last_clean":"2024-03-21T09:10:47.408857+0000","last_became_active":"2024-03-21T09:08:46.819604+0000","last_became_peered":"2024-03-21T09:08:46.819604+0000","last_unstale":"2024-03-21T09:10:47.408857+0000","last_undegraded":"2024-03-21T09:10:47.408857+0000","last_fullsized":"2024-03-21T09:10:47.408857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:53:45.499824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407740+0000","last_change":"2024-03-21T09:08:50.984976+0000","last_active":"2024-03-21T09:10:47.407740+0000","last_peered":"2024-03-21T09:10:47.407740+0000","last_clean":"2024-03-21T09:10:47.407740+0000","last_became_active":"2024-03-21T09:08:50.984509+0000","last_became_peered":"2024-03-21T09:08:50.984509+0000","last_unstale":"2024-03-21T09:10:47.407740+0000","last_undegraded":"2024-03-21T09:10:47.407740+0000","last_fullsized":"2024-03-21T09:10:47.407740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:20:06.757550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.420395+0000","last_change":"2024-03-21T09:10:47.420051+0000","last_active":"2024-03-21T09:10:47.420395+0000","last_peered":"2024-03-21T09:10:47.420395+0000","last_clean":"2024-03-21T09:10:47.420395+0000","last_became_active":"2024-03-21T09:10:47.412598+0000","last_became_peered":"2024-03-21T09:10:47.412598+0000","last_unstale":"2024-03-21T09:10:47.420395+0000","last_undegraded":"2024-03-21T09:10:47.420395+0000","last_fullsized":"2024-03-21T09:10:47.420395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:43.675518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438823+0000","last_change":"2024-03-21T09:08:46.844545+0000","last_active":"2024-03-21T09:10:47.438823+0000","last_peered":"2024-03-21T09:10:47.438823+0000","last_clean":"2024-03-21T09:10:47.438823+0000","last_became_active":"2024-03-21T09:08:46.842310+0000","last_became_peered":"2024-03-21T09:08:46.842310+0000","last_unstale":"2024-03-21T09:10:47.438823+0000","last_undegraded":"2024-03-21T09:10:47.438823+0000","last_fullsized":"2024-03-21T09:10:47.438823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:11:40.911964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404558+0000","last_change":"2024-03-21T09:08:50.983782+0000","last_active":"2024-03-21T09:10:47.404558+0000","last_peered":"2024-03-21T09:10:47.404558+0000","last_clean":"2024-03-21T09:10:47.404558+0000","last_became_active":"2024-03-21T09:08:50.983357+0000","last_became_peered":"2024-03-21T09:08:50.983357+0000","last_unstale":"2024-03-21T09:10:47.404558+0000","last_undegraded":"2024-03-21T09:10:47.404558+0000","last_fullsized":"2024-03-21T09:10:47.404558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:16.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407208+0000","last_change":"2024-03-21T09:08:45.825454+0000","last_active":"2024-03-21T09:10:47.407208+0000","last_peered":"2024-03-21T09:10:47.407208+0000","last_clean":"2024-03-21T09:10:47.407208+0000","last_became_active":"2024-03-21T09:08:45.825099+0000","last_became_peered":"2024-03-21T09:08:45.825099+0000","last_unstale":"2024-03-21T09:10:47.407208+0000","last_undegraded":"2024-03-21T09:10:47.407208+0000","last_fullsized":"2024-03-21T09:10:47.407208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:56:16.357034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436363+0000","last_change":"2024-03-21T09:08:46.837025+0000","last_active":"2024-03-21T09:10:47.436363+0000","last_peered":"2024-03-21T09:10:47.436363+0000","last_clean":"2024-03-21T09:10:47.436363+0000","last_became_active":"2024-03-21T09:08:46.836619+0000","last_became_peered":"2024-03-21T09:08:46.836619+0000","last_unstale":"2024-03-21T09:10:47.436363+0000","last_undegraded":"2024-03-21T09:10:47.436363+0000","last_fullsized":"2024-03-21T09:10:47.436363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:22:34.220723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408459+0000","last_change":"2024-03-21T09:08:50.980143+0000","last_active":"2024-03-21T09:10:47.408459+0000","last_peered":"2024-03-21T09:10:47.408459+0000","last_clean":"2024-03-21T09:10:47.408459+0000","last_became_active":"2024-03-21T09:08:50.979579+0000","last_became_peered":"2024-03-21T09:08:50.979579+0000","last_unstale":"2024-03-21T09:10:47.408459+0000","last_undegraded":"2024-03-21T09:10:47.408459+0000","last_fullsized":"2024-03-21T09:10:47.408459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:42:50.039194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":198,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:53.045042+0000","last_change":"2024-03-21T09:08:45.813029+0000","last_active":"2024-03-21T09:11:53.045042+0000","last_peered":"2024-03-21T09:11:53.045042+0000","last_clean":"2024-03-21T09:11:53.045042+0000","last_became_active":"2024-03-21T09:08:45.812619+0000","last_became_peered":"2024-03-21T09:08:45.812619+0000","last_unstale":"2024-03-21T09:11:53.045042+0000","last_undegraded":"2024-03-21T09:11:53.045042+0000","last_fullsized":"2024-03-21T09:11:53.045042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:02:52.226563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":161,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436189+0000","last_change":"2024-03-21T09:08:46.820097+0000","last_active":"2024-03-21T09:10:47.436189+0000","last_peered":"2024-03-21T09:10:47.436189+0000","last_clean":"2024-03-21T09:10:47.436189+0000","last_became_active":"2024-03-21T09:08:46.819525+0000","last_became_peered":"2024-03-21T09:08:46.819525+0000","last_unstale":"2024-03-21T09:10:47.436189+0000","last_undegraded":"2024-03-21T09:10:47.436189+0000","last_fullsized":"2024-03-21T09:10:47.436189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:42.562671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439612+0000","last_change":"2024-03-21T09:08:50.979689+0000","last_active":"2024-03-21T09:10:47.439612+0000","last_peered":"2024-03-21T09:10:47.439612+0000","last_clean":"2024-03-21T09:10:47.439612+0000","last_became_active":"2024-03-21T09:08:50.979301+0000","last_became_peered":"2024-03-21T09:08:50.979301+0000","last_unstale":"2024-03-21T09:10:47.439612+0000","last_undegraded":"2024-03-21T09:10:47.439612+0000","last_fullsized":"2024-03-21T09:10:47.439612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:36:57.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408028+0000","last_change":"2024-03-21T09:08:45.815915+0000","last_active":"2024-03-21T09:10:47.408028+0000","last_peered":"2024-03-21T09:10:47.408028+0000","last_clean":"2024-03-21T09:10:47.408028+0000","last_became_active":"2024-03-21T09:08:45.815544+0000","last_became_peered":"2024-03-21T09:08:45.815544+0000","last_unstale":"2024-03-21T09:10:47.408028+0000","last_undegraded":"2024-03-21T09:10:47.408028+0000","last_fullsized":"2024-03-21T09:10:47.408028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:36.698678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406881+0000","last_change":"2024-03-21T09:08:46.818081+0000","last_active":"2024-03-21T09:10:47.406881+0000","last_peered":"2024-03-21T09:10:47.406881+0000","last_clean":"2024-03-21T09:10:47.406881+0000","last_became_active":"2024-03-21T09:08:46.817709+0000","last_became_peered":"2024-03-21T09:08:46.817709+0000","last_unstale":"2024-03-21T09:10:47.406881+0000","last_undegraded":"2024-03-21T09:10:47.406881+0000","last_fullsized":"2024-03-21T09:10:47.406881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:57:32.935098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406796+0000","last_change":"2024-03-21T09:08:50.965913+0000","last_active":"2024-03-21T09:10:47.406796+0000","last_peered":"2024-03-21T09:10:47.406796+0000","last_clean":"2024-03-21T09:10:47.406796+0000","last_became_active":"2024-03-21T09:08:50.965505+0000","last_became_peered":"2024-03-21T09:08:50.965505+0000","last_unstale":"2024-03-21T09:10:47.406796+0000","last_undegraded":"2024-03-21T09:10:47.406796+0000","last_fullsized":"2024-03-21T09:10:47.406796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:14.967598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:59.860354+0000","last_change":"2024-03-21T09:08:45.817893+0000","last_active":"2024-03-21T09:10:59.860354+0000","last_peered":"2024-03-21T09:10:59.860354+0000","last_clean":"2024-03-21T09:10:59.860354+0000","last_became_active":"2024-03-21T09:08:45.817293+0000","last_became_peered":"2024-03-21T09:08:45.817293+0000","last_unstale":"2024-03-21T09:10:59.860354+0000","last_undegraded":"2024-03-21T09:10:59.860354+0000","last_fullsized":"2024-03-21T09:10:59.860354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:18:28.663394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404402+0000","last_change":"2024-03-21T09:08:46.819254+0000","last_active":"2024-03-21T09:10:47.404402+0000","last_peered":"2024-03-21T09:10:47.404402+0000","last_clean":"2024-03-21T09:10:47.404402+0000","last_became_active":"2024-03-21T09:08:46.818907+0000","last_became_peered":"2024-03-21T09:08:46.818907+0000","last_unstale":"2024-03-21T09:10:47.404402+0000","last_undegraded":"2024-03-21T09:10:47.404402+0000","last_fullsized":"2024-03-21T09:10:47.404402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:03.424150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408215+0000","last_change":"2024-03-21T09:08:51.220091+0000","last_active":"2024-03-21T09:10:47.408215+0000","last_peered":"2024-03-21T09:10:47.408215+0000","last_clean":"2024-03-21T09:10:47.408215+0000","last_became_active":"2024-03-21T09:08:51.219562+0000","last_became_peered":"2024-03-21T09:08:51.219562+0000","last_unstale":"2024-03-21T09:10:47.408215+0000","last_undegraded":"2024-03-21T09:10:47.408215+0000","last_fullsized":"2024-03-21T09:10:47.408215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:20:33.412602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408295+0000","last_change":"2024-03-21T09:08:45.810632+0000","last_active":"2024-03-21T09:10:47.408295+0000","last_peered":"2024-03-21T09:10:47.408295+0000","last_clean":"2024-03-21T09:10:47.408295+0000","last_became_active":"2024-03-21T09:08:45.810306+0000","last_became_peered":"2024-03-21T09:08:45.810306+0000","last_unstale":"2024-03-21T09:10:47.408295+0000","last_undegraded":"2024-03-21T09:10:47.408295+0000","last_fullsized":"2024-03-21T09:10:47.408295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:18:14.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402804+0000","last_change":"2024-03-21T09:08:46.844513+0000","last_active":"2024-03-21T09:10:47.402804+0000","last_peered":"2024-03-21T09:10:47.402804+0000","last_clean":"2024-03-21T09:10:47.402804+0000","last_became_active":"2024-03-21T09:08:46.844125+0000","last_became_peered":"2024-03-21T09:08:46.844125+0000","last_unstale":"2024-03-21T09:10:47.402804+0000","last_undegraded":"2024-03-21T09:10:47.402804+0000","last_fullsized":"2024-03-21T09:10:47.402804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:25:50.155242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439328+0000","last_change":"2024-03-21T09:08:50.980183+0000","last_active":"2024-03-21T09:10:47.439328+0000","last_peered":"2024-03-21T09:10:47.439328+0000","last_clean":"2024-03-21T09:10:47.439328+0000","last_became_active":"2024-03-21T09:08:50.979646+0000","last_became_peered":"2024-03-21T09:08:50.979646+0000","last_unstale":"2024-03-21T09:10:47.439328+0000","last_undegraded":"2024-03-21T09:10:47.439328+0000","last_fullsized":"2024-03-21T09:10:47.439328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:35:46.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436162+0000","last_change":"2024-03-21T09:08:45.822824+0000","last_active":"2024-03-21T09:10:47.436162+0000","last_peered":"2024-03-21T09:10:47.436162+0000","last_clean":"2024-03-21T09:10:47.436162+0000","last_became_active":"2024-03-21T09:08:45.822425+0000","last_became_peered":"2024-03-21T09:08:45.822425+0000","last_unstale":"2024-03-21T09:10:47.436162+0000","last_undegraded":"2024-03-21T09:10:47.436162+0000","last_fullsized":"2024-03-21T09:10:47.436162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:57:46.449595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.402598+0000","last_change":"2024-03-21T09:08:46.841797+0000","last_active":"2024-03-21T09:10:47.402598+0000","last_peered":"2024-03-21T09:10:47.402598+0000","last_clean":"2024-03-21T09:10:47.402598+0000","last_became_active":"2024-03-21T09:08:46.841424+0000","last_became_peered":"2024-03-21T09:08:46.841424+0000","last_unstale":"2024-03-21T09:10:47.402598+0000","last_undegraded":"2024-03-21T09:10:47.402598+0000","last_fullsized":"2024-03-21T09:10:47.402598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:32:50.808618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439700+0000","last_change":"2024-03-21T09:08:51.221333+0000","last_active":"2024-03-21T09:10:47.439700+0000","last_peered":"2024-03-21T09:10:47.439700+0000","last_clean":"2024-03-21T09:10:47.439700+0000","last_became_active":"2024-03-21T09:08:51.220830+0000","last_became_peered":"2024-03-21T09:08:51.220830+0000","last_unstale":"2024-03-21T09:10:47.439700+0000","last_undegraded":"2024-03-21T09:10:47.439700+0000","last_fullsized":"2024-03-21T09:10:47.439700+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:42:57.501619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407244+0000","last_change":"2024-03-21T09:09:47.205699+0000","last_active":"2024-03-21T09:10:47.407244+0000","last_peered":"2024-03-21T09:10:47.407244+0000","last_clean":"2024-03-21T09:10:47.407244+0000","last_became_active":"2024-03-21T09:09:47.203657+0000","last_became_peered":"2024-03-21T09:09:47.203657+0000","last_unstale":"2024-03-21T09:10:47.407244+0000","last_undegraded":"2024-03-21T09:10:47.407244+0000","last_fullsized":"2024-03-21T09:10:47.407244+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:42.930153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":191,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:52.042829+0000","last_change":"2024-03-21T09:08:45.838550+0000","last_active":"2024-03-21T09:11:52.042829+0000","last_peered":"2024-03-21T09:11:52.042829+0000","last_clean":"2024-03-21T09:11:52.042829+0000","last_became_active":"2024-03-21T09:08:45.837971+0000","last_became_peered":"2024-03-21T09:08:45.837971+0000","last_unstale":"2024-03-21T09:11:52.042829+0000","last_undegraded":"2024-03-21T09:11:52.042829+0000","last_fullsized":"2024-03-21T09:11:52.042829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:08.805876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436708+0000","last_change":"2024-03-21T09:08:50.956249+0000","last_active":"2024-03-21T09:10:47.436708+0000","last_peered":"2024-03-21T09:10:47.436708+0000","last_clean":"2024-03-21T09:10:47.436708+0000","last_became_active":"2024-03-21T09:08:50.955737+0000","last_became_peered":"2024-03-21T09:08:50.955737+0000","last_unstale":"2024-03-21T09:10:47.436708+0000","last_undegraded":"2024-03-21T09:10:47.436708+0000","last_fullsized":"2024-03-21T09:10:47.436708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:56:46.754880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409498+0000","last_change":"2024-03-21T09:08:46.840527+0000","last_active":"2024-03-21T09:10:47.409498+0000","last_peered":"2024-03-21T09:10:47.409498+0000","last_clean":"2024-03-21T09:10:47.409498+0000","last_became_active":"2024-03-21T09:08:46.836059+0000","last_became_peered":"2024-03-21T09:08:46.836059+0000","last_unstale":"2024-03-21T09:10:47.409498+0000","last_undegraded":"2024-03-21T09:10:47.409498+0000","last_fullsized":"2024-03-21T09:10:47.409498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:34:55.280586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436333+0000","last_change":"2024-03-21T09:08:45.809969+0000","last_active":"2024-03-21T09:10:47.436333+0000","last_peered":"2024-03-21T09:10:47.436333+0000","last_clean":"2024-03-21T09:10:47.436333+0000","last_became_active":"2024-03-21T09:08:45.809217+0000","last_became_peered":"2024-03-21T09:08:45.809217+0000","last_unstale":"2024-03-21T09:10:47.436333+0000","last_undegraded":"2024-03-21T09:10:47.436333+0000","last_fullsized":"2024-03-21T09:10:47.436333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:40:14.422576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.435910+0000","last_change":"2024-03-21T09:07:25.524151+0000","last_active":"2024-03-21T09:10:47.435910+0000","last_peered":"2024-03-21T09:10:47.435910+0000","last_clean":"2024-03-21T09:10:47.435910+0000","last_became_active":"2024-03-21T09:07:25.523699+0000","last_became_peered":"2024-03-21T09:07:25.523699+0000","last_unstale":"2024-03-21T09:10:47.435910+0000","last_undegraded":"2024-03-21T09:10:47.435910+0000","last_fullsized":"2024-03-21T09:10:47.435910+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:03:30.335528+0000","last_clean_scrub_stamp":"2024-03-21T09:03:30.335528+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:02.986083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407059+0000","last_change":"2024-03-21T09:08:51.220624+0000","last_active":"2024-03-21T09:10:47.407059+0000","last_peered":"2024-03-21T09:10:47.407059+0000","last_clean":"2024-03-21T09:10:47.407059+0000","last_became_active":"2024-03-21T09:08:51.220158+0000","last_became_peered":"2024-03-21T09:08:51.220158+0000","last_unstale":"2024-03-21T09:10:47.407059+0000","last_undegraded":"2024-03-21T09:10:47.407059+0000","last_fullsized":"2024-03-21T09:10:47.407059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:23:38.475668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438390+0000","last_change":"2024-03-21T09:08:46.837337+0000","last_active":"2024-03-21T09:10:47.438390+0000","last_peered":"2024-03-21T09:10:47.438390+0000","last_clean":"2024-03-21T09:10:47.438390+0000","last_became_active":"2024-03-21T09:08:46.834437+0000","last_became_peered":"2024-03-21T09:08:46.834437+0000","last_unstale":"2024-03-21T09:10:47.438390+0000","last_undegraded":"2024-03-21T09:10:47.438390+0000","last_fullsized":"2024-03-21T09:10:47.438390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:34.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436241+0000","last_change":"2024-03-21T09:08:45.817931+0000","last_active":"2024-03-21T09:10:47.436241+0000","last_peered":"2024-03-21T09:10:47.436241+0000","last_clean":"2024-03-21T09:10:47.436241+0000","last_became_active":"2024-03-21T09:08:45.817443+0000","last_became_peered":"2024-03-21T09:08:45.817443+0000","last_unstale":"2024-03-21T09:10:47.436241+0000","last_undegraded":"2024-03-21T09:10:47.436241+0000","last_fullsized":"2024-03-21T09:10:47.436241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:48:41.866505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404852+0000","last_change":"2024-03-21T09:08:50.986055+0000","last_active":"2024-03-21T09:10:47.404852+0000","last_peered":"2024-03-21T09:10:47.404852+0000","last_clean":"2024-03-21T09:10:47.404852+0000","last_became_active":"2024-03-21T09:08:50.985539+0000","last_became_peered":"2024-03-21T09:08:50.985539+0000","last_unstale":"2024-03-21T09:10:47.404852+0000","last_undegraded":"2024-03-21T09:10:47.404852+0000","last_fullsized":"2024-03-21T09:10:47.404852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:57.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409034+0000","last_change":"2024-03-21T09:08:45.817884+0000","last_active":"2024-03-21T09:10:47.409034+0000","last_peered":"2024-03-21T09:10:47.409034+0000","last_clean":"2024-03-21T09:10:47.409034+0000","last_became_active":"2024-03-21T09:08:45.817447+0000","last_became_peered":"2024-03-21T09:08:45.817447+0000","last_unstale":"2024-03-21T09:10:47.409034+0000","last_undegraded":"2024-03-21T09:10:47.409034+0000","last_fullsized":"2024-03-21T09:10:47.409034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:11:53.081993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439473+0000","last_change":"2024-03-21T09:08:46.834457+0000","last_active":"2024-03-21T09:10:47.439473+0000","last_peered":"2024-03-21T09:10:47.439473+0000","last_clean":"2024-03-21T09:10:47.439473+0000","last_became_active":"2024-03-21T09:08:46.833809+0000","last_became_peered":"2024-03-21T09:08:46.833809+0000","last_unstale":"2024-03-21T09:10:47.439473+0000","last_undegraded":"2024-03-21T09:10:47.439473+0000","last_fullsized":"2024-03-21T09:10:47.439473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:13:34.658254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410228+0000","last_change":"2024-03-21T09:08:51.220408+0000","last_active":"2024-03-21T09:10:47.410228+0000","last_peered":"2024-03-21T09:10:47.410228+0000","last_clean":"2024-03-21T09:10:47.410228+0000","last_became_active":"2024-03-21T09:08:51.219958+0000","last_became_peered":"2024-03-21T09:08:51.219958+0000","last_unstale":"2024-03-21T09:10:47.410228+0000","last_undegraded":"2024-03-21T09:10:47.410228+0000","last_fullsized":"2024-03-21T09:10:47.410228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:01.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:11:47.922308+0000","last_change":"2024-03-21T09:08:45.828057+0000","last_active":"2024-03-21T09:11:47.922308+0000","last_peered":"2024-03-21T09:11:47.922308+0000","last_clean":"2024-03-21T09:11:47.922308+0000","last_became_active":"2024-03-21T09:08:45.827671+0000","last_became_peered":"2024-03-21T09:08:45.827671+0000","last_unstale":"2024-03-21T09:11:47.922308+0000","last_undegraded":"2024-03-21T09:11:47.922308+0000","last_fullsized":"2024-03-21T09:11:47.922308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:51:59.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411304+0000","last_change":"2024-03-21T09:08:46.839584+0000","last_active":"2024-03-21T09:10:47.411304+0000","last_peered":"2024-03-21T09:10:47.411304+0000","last_clean":"2024-03-21T09:10:47.411304+0000","last_became_active":"2024-03-21T09:08:46.839140+0000","last_became_peered":"2024-03-21T09:08:46.839140+0000","last_unstale":"2024-03-21T09:10:47.411304+0000","last_undegraded":"2024-03-21T09:10:47.411304+0000","last_fullsized":"2024-03-21T09:10:47.411304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:06:41.938129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405917+0000","last_change":"2024-03-21T09:08:50.954443+0000","last_active":"2024-03-21T09:10:47.405917+0000","last_peered":"2024-03-21T09:10:47.405917+0000","last_clean":"2024-03-21T09:10:47.405917+0000","last_became_active":"2024-03-21T09:08:50.953982+0000","last_became_peered":"2024-03-21T09:08:50.953982+0000","last_unstale":"2024-03-21T09:10:47.405917+0000","last_undegraded":"2024-03-21T09:10:47.405917+0000","last_fullsized":"2024-03-21T09:10:47.405917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:26:58.544514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436899+0000","last_change":"2024-03-21T09:08:45.808580+0000","last_active":"2024-03-21T09:10:47.436899+0000","last_peered":"2024-03-21T09:10:47.436899+0000","last_clean":"2024-03-21T09:10:47.436899+0000","last_became_active":"2024-03-21T09:08:45.808220+0000","last_became_peered":"2024-03-21T09:08:45.808220+0000","last_unstale":"2024-03-21T09:10:47.436899+0000","last_undegraded":"2024-03-21T09:10:47.436899+0000","last_fullsized":"2024-03-21T09:10:47.436899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:26:58.939701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.438375+0000","last_change":"2024-03-21T09:08:46.845484+0000","last_active":"2024-03-21T09:10:47.438375+0000","last_peered":"2024-03-21T09:10:47.438375+0000","last_clean":"2024-03-21T09:10:47.438375+0000","last_became_active":"2024-03-21T09:08:46.845049+0000","last_became_peered":"2024-03-21T09:08:46.845049+0000","last_unstale":"2024-03-21T09:10:47.438375+0000","last_undegraded":"2024-03-21T09:10:47.438375+0000","last_fullsized":"2024-03-21T09:10:47.438375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:00.197778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436534+0000","last_change":"2024-03-21T09:09:47.203560+0000","last_active":"2024-03-21T09:10:47.436534+0000","last_peered":"2024-03-21T09:10:47.436534+0000","last_clean":"2024-03-21T09:10:47.436534+0000","last_became_active":"2024-03-21T09:09:47.203072+0000","last_became_peered":"2024-03-21T09:09:47.203072+0000","last_unstale":"2024-03-21T09:10:47.436534+0000","last_undegraded":"2024-03-21T09:10:47.436534+0000","last_fullsized":"2024-03-21T09:10:47.436534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:03:39.037042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406193+0000","last_change":"2024-03-21T09:08:45.811759+0000","last_active":"2024-03-21T09:10:47.406193+0000","last_peered":"2024-03-21T09:10:47.406193+0000","last_clean":"2024-03-21T09:10:47.406193+0000","last_became_active":"2024-03-21T09:08:45.811227+0000","last_became_peered":"2024-03-21T09:08:45.811227+0000","last_unstale":"2024-03-21T09:10:47.406193+0000","last_undegraded":"2024-03-21T09:10:47.406193+0000","last_fullsized":"2024-03-21T09:10:47.406193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:52:07.457281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408090+0000","last_change":"2024-03-21T09:08:46.821348+0000","last_active":"2024-03-21T09:10:47.408090+0000","last_peered":"2024-03-21T09:10:47.408090+0000","last_clean":"2024-03-21T09:10:47.408090+0000","last_became_active":"2024-03-21T09:08:46.820941+0000","last_became_peered":"2024-03-21T09:08:46.820941+0000","last_unstale":"2024-03-21T09:10:47.408090+0000","last_undegraded":"2024-03-21T09:10:47.408090+0000","last_fullsized":"2024-03-21T09:10:47.408090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:38:24.187569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408581+0000","last_change":"2024-03-21T09:08:50.982093+0000","last_active":"2024-03-21T09:10:47.408581+0000","last_peered":"2024-03-21T09:10:47.408581+0000","last_clean":"2024-03-21T09:10:47.408581+0000","last_became_active":"2024-03-21T09:08:50.981611+0000","last_became_peered":"2024-03-21T09:08:50.981611+0000","last_unstale":"2024-03-21T09:10:47.408581+0000","last_undegraded":"2024-03-21T09:10:47.408581+0000","last_fullsized":"2024-03-21T09:10:47.408581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:29:01.649636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409274+0000","last_change":"2024-03-21T09:08:45.827610+0000","last_active":"2024-03-21T09:10:47.409274+0000","last_peered":"2024-03-21T09:10:47.409274+0000","last_clean":"2024-03-21T09:10:47.409274+0000","last_became_active":"2024-03-21T09:08:45.827186+0000","last_became_peered":"2024-03-21T09:08:45.827186+0000","last_unstale":"2024-03-21T09:10:47.409274+0000","last_undegraded":"2024-03-21T09:10:47.409274+0000","last_fullsized":"2024-03-21T09:10:47.409274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:10.765173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404093+0000","last_change":"2024-03-21T09:08:46.844587+0000","last_active":"2024-03-21T09:10:47.404093+0000","last_peered":"2024-03-21T09:10:47.404093+0000","last_clean":"2024-03-21T09:10:47.404093+0000","last_became_active":"2024-03-21T09:08:46.844209+0000","last_became_peered":"2024-03-21T09:08:46.844209+0000","last_unstale":"2024-03-21T09:10:47.404093+0000","last_undegraded":"2024-03-21T09:10:47.404093+0000","last_fullsized":"2024-03-21T09:10:47.404093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.144615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436903+0000","last_change":"2024-03-21T09:09:47.206281+0000","last_active":"2024-03-21T09:10:47.436903+0000","last_peered":"2024-03-21T09:10:47.436903+0000","last_clean":"2024-03-21T09:10:47.436903+0000","last_became_active":"2024-03-21T09:09:47.198427+0000","last_became_peered":"2024-03-21T09:09:47.198427+0000","last_unstale":"2024-03-21T09:10:47.436903+0000","last_undegraded":"2024-03-21T09:10:47.436903+0000","last_fullsized":"2024-03-21T09:10:47.436903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:32:46.182225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406103+0000","last_change":"2024-03-21T09:08:45.827093+0000","last_active":"2024-03-21T09:10:47.406103+0000","last_peered":"2024-03-21T09:10:47.406103+0000","last_clean":"2024-03-21T09:10:47.406103+0000","last_became_active":"2024-03-21T09:08:45.826654+0000","last_became_peered":"2024-03-21T09:08:45.826654+0000","last_unstale":"2024-03-21T09:10:47.406103+0000","last_undegraded":"2024-03-21T09:10:47.406103+0000","last_fullsized":"2024-03-21T09:10:47.406103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:14.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.410703+0000","last_change":"2024-03-21T09:08:46.851535+0000","last_active":"2024-03-21T09:10:47.410703+0000","last_peered":"2024-03-21T09:10:47.410703+0000","last_clean":"2024-03-21T09:10:47.410703+0000","last_became_active":"2024-03-21T09:08:46.851063+0000","last_became_peered":"2024-03-21T09:08:46.851063+0000","last_unstale":"2024-03-21T09:10:47.410703+0000","last_undegraded":"2024-03-21T09:10:47.410703+0000","last_fullsized":"2024-03-21T09:10:47.410703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:02.824662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436507+0000","last_change":"2024-03-21T09:08:51.220374+0000","last_active":"2024-03-21T09:10:47.436507+0000","last_peered":"2024-03-21T09:10:47.436507+0000","last_clean":"2024-03-21T09:10:47.436507+0000","last_became_active":"2024-03-21T09:08:51.219949+0000","last_became_peered":"2024-03-21T09:08:51.219949+0000","last_unstale":"2024-03-21T09:10:47.436507+0000","last_undegraded":"2024-03-21T09:10:47.436507+0000","last_fullsized":"2024-03-21T09:10:47.436507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:21:49.278767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.408535+0000","last_change":"2024-03-21T09:08:45.816178+0000","last_active":"2024-03-21T09:10:47.408535+0000","last_peered":"2024-03-21T09:10:47.408535+0000","last_clean":"2024-03-21T09:10:47.408535+0000","last_became_active":"2024-03-21T09:08:45.815801+0000","last_became_peered":"2024-03-21T09:08:45.815801+0000","last_unstale":"2024-03-21T09:10:47.408535+0000","last_undegraded":"2024-03-21T09:10:47.408535+0000","last_fullsized":"2024-03-21T09:10:47.408535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:12.839514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436195+0000","last_change":"2024-03-21T09:08:46.836100+0000","last_active":"2024-03-21T09:10:47.436195+0000","last_peered":"2024-03-21T09:10:47.436195+0000","last_clean":"2024-03-21T09:10:47.436195+0000","last_became_active":"2024-03-21T09:08:46.835708+0000","last_became_peered":"2024-03-21T09:08:46.835708+0000","last_unstale":"2024-03-21T09:10:47.436195+0000","last_undegraded":"2024-03-21T09:10:47.436195+0000","last_fullsized":"2024-03-21T09:10:47.436195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:36:03.592068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406267+0000","last_change":"2024-03-21T09:08:50.984844+0000","last_active":"2024-03-21T09:10:47.406267+0000","last_peered":"2024-03-21T09:10:47.406267+0000","last_clean":"2024-03-21T09:10:47.406267+0000","last_became_active":"2024-03-21T09:08:50.984313+0000","last_became_peered":"2024-03-21T09:08:50.984313+0000","last_unstale":"2024-03-21T09:10:47.406267+0000","last_undegraded":"2024-03-21T09:10:47.406267+0000","last_fullsized":"2024-03-21T09:10:47.406267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:05.018681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406667+0000","last_change":"2024-03-21T09:08:45.810441+0000","last_active":"2024-03-21T09:10:47.406667+0000","last_peered":"2024-03-21T09:10:47.406667+0000","last_clean":"2024-03-21T09:10:47.406667+0000","last_became_active":"2024-03-21T09:08:45.810103+0000","last_became_peered":"2024-03-21T09:08:45.810103+0000","last_unstale":"2024-03-21T09:10:47.406667+0000","last_undegraded":"2024-03-21T09:10:47.406667+0000","last_fullsized":"2024-03-21T09:10:47.406667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:44.545808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436775+0000","last_change":"2024-03-21T09:08:46.823832+0000","last_active":"2024-03-21T09:10:47.436775+0000","last_peered":"2024-03-21T09:10:47.436775+0000","last_clean":"2024-03-21T09:10:47.436775+0000","last_became_active":"2024-03-21T09:08:46.823488+0000","last_became_peered":"2024-03-21T09:08:46.823488+0000","last_unstale":"2024-03-21T09:10:47.436775+0000","last_undegraded":"2024-03-21T09:10:47.436775+0000","last_fullsized":"2024-03-21T09:10:47.436775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:01.603283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437679+0000","last_change":"2024-03-21T09:08:50.972676+0000","last_active":"2024-03-21T09:10:47.437679+0000","last_peered":"2024-03-21T09:10:47.437679+0000","last_clean":"2024-03-21T09:10:47.437679+0000","last_became_active":"2024-03-21T09:08:50.970854+0000","last_became_peered":"2024-03-21T09:08:50.970854+0000","last_unstale":"2024-03-21T09:10:47.437679+0000","last_undegraded":"2024-03-21T09:10:47.437679+0000","last_fullsized":"2024-03-21T09:10:47.437679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.045367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407624+0000","last_change":"2024-03-21T09:08:45.828984+0000","last_active":"2024-03-21T09:10:47.407624+0000","last_peered":"2024-03-21T09:10:47.407624+0000","last_clean":"2024-03-21T09:10:47.407624+0000","last_became_active":"2024-03-21T09:08:45.828701+0000","last_became_peered":"2024-03-21T09:08:45.828701+0000","last_unstale":"2024-03-21T09:10:47.407624+0000","last_undegraded":"2024-03-21T09:10:47.407624+0000","last_fullsized":"2024-03-21T09:10:47.407624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:13:26.280763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436680+0000","last_change":"2024-03-21T09:08:46.844382+0000","last_active":"2024-03-21T09:10:47.436680+0000","last_peered":"2024-03-21T09:10:47.436680+0000","last_clean":"2024-03-21T09:10:47.436680+0000","last_became_active":"2024-03-21T09:08:46.843989+0000","last_became_peered":"2024-03-21T09:08:46.843989+0000","last_unstale":"2024-03-21T09:10:47.436680+0000","last_undegraded":"2024-03-21T09:10:47.436680+0000","last_fullsized":"2024-03-21T09:10:47.436680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:18:55.016259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406417+0000","last_change":"2024-03-21T09:08:50.955937+0000","last_active":"2024-03-21T09:10:47.406417+0000","last_peered":"2024-03-21T09:10:47.406417+0000","last_clean":"2024-03-21T09:10:47.406417+0000","last_became_active":"2024-03-21T09:08:50.955451+0000","last_became_peered":"2024-03-21T09:08:50.955451+0000","last_unstale":"2024-03-21T09:10:47.406417+0000","last_undegraded":"2024-03-21T09:10:47.406417+0000","last_fullsized":"2024-03-21T09:10:47.406417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:49.911653+0000","last_clean_scrub_stamp":"2024-03-21T09:08:49.911653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:59.867153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439536+0000","last_change":"2024-03-21T09:08:45.834748+0000","last_active":"2024-03-21T09:10:47.439536+0000","last_peered":"2024-03-21T09:10:47.439536+0000","last_clean":"2024-03-21T09:10:47.439536+0000","last_became_active":"2024-03-21T09:08:45.834217+0000","last_became_peered":"2024-03-21T09:08:45.834217+0000","last_unstale":"2024-03-21T09:10:47.439536+0000","last_undegraded":"2024-03-21T09:10:47.439536+0000","last_fullsized":"2024-03-21T09:10:47.439536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:44.702521+0000","last_clean_scrub_stamp":"2024-03-21T09:08:44.702521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:19:01.935283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.436796+0000","last_change":"2024-03-21T09:08:46.824265+0000","last_active":"2024-03-21T09:10:47.436796+0000","last_peered":"2024-03-21T09:10:47.436796+0000","last_clean":"2024-03-21T09:10:47.436796+0000","last_became_active":"2024-03-21T09:08:46.823397+0000","last_became_peered":"2024-03-21T09:08:46.823397+0000","last_unstale":"2024-03-21T09:10:47.436796+0000","last_undegraded":"2024-03-21T09:10:47.436796+0000","last_fullsized":"2024-03-21T09:10:47.436796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:45:56.619236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404217+0000","last_change":"2024-03-21T09:08:46.817366+0000","last_active":"2024-03-21T09:10:47.404217+0000","last_peered":"2024-03-21T09:10:47.404217+0000","last_clean":"2024-03-21T09:10:47.404217+0000","last_became_active":"2024-03-21T09:08:46.817035+0000","last_became_peered":"2024-03-21T09:08:46.817035+0000","last_unstale":"2024-03-21T09:10:47.404217+0000","last_undegraded":"2024-03-21T09:10:47.404217+0000","last_fullsized":"2024-03-21T09:10:47.404217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:59:44.360940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407361+0000","last_change":"2024-03-21T09:09:47.200898+0000","last_active":"2024-03-21T09:10:47.407361+0000","last_peered":"2024-03-21T09:10:47.407361+0000","last_clean":"2024-03-21T09:10:47.407361+0000","last_became_active":"2024-03-21T09:09:47.200548+0000","last_became_peered":"2024-03-21T09:09:47.200548+0000","last_unstale":"2024-03-21T09:10:47.407361+0000","last_undegraded":"2024-03-21T09:10:47.407361+0000","last_fullsized":"2024-03-21T09:10:47.407361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:29.793107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.405957+0000","last_change":"2024-03-21T09:08:46.842220+0000","last_active":"2024-03-21T09:10:47.405957+0000","last_peered":"2024-03-21T09:10:47.405957+0000","last_clean":"2024-03-21T09:10:47.405957+0000","last_became_active":"2024-03-21T09:08:46.841821+0000","last_became_peered":"2024-03-21T09:08:46.841821+0000","last_unstale":"2024-03-21T09:10:47.405957+0000","last_undegraded":"2024-03-21T09:10:47.405957+0000","last_fullsized":"2024-03-21T09:10:47.405957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:28.361390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.437955+0000","last_change":"2024-03-21T09:08:46.845715+0000","last_active":"2024-03-21T09:10:47.437955+0000","last_peered":"2024-03-21T09:10:47.437955+0000","last_clean":"2024-03-21T09:10:47.437955+0000","last_became_active":"2024-03-21T09:08:46.843121+0000","last_became_peered":"2024-03-21T09:08:46.843121+0000","last_unstale":"2024-03-21T09:10:47.437955+0000","last_undegraded":"2024-03-21T09:10:47.437955+0000","last_fullsized":"2024-03-21T09:10:47.437955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:24:23.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439688+0000","last_change":"2024-03-21T09:08:46.846702+0000","last_active":"2024-03-21T09:10:47.439688+0000","last_peered":"2024-03-21T09:10:47.439688+0000","last_clean":"2024-03-21T09:10:47.439688+0000","last_became_active":"2024-03-21T09:08:46.844398+0000","last_became_peered":"2024-03-21T09:08:46.844398+0000","last_unstale":"2024-03-21T09:10:47.439688+0000","last_undegraded":"2024-03-21T09:10:47.439688+0000","last_fullsized":"2024-03-21T09:10:47.439688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:13.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404565+0000","last_change":"2024-03-21T09:08:46.819048+0000","last_active":"2024-03-21T09:10:47.404565+0000","last_peered":"2024-03-21T09:10:47.404565+0000","last_clean":"2024-03-21T09:10:47.404565+0000","last_became_active":"2024-03-21T09:08:46.818689+0000","last_became_peered":"2024-03-21T09:08:46.818689+0000","last_unstale":"2024-03-21T09:10:47.404565+0000","last_undegraded":"2024-03-21T09:10:47.404565+0000","last_fullsized":"2024-03-21T09:10:47.404565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:13:44.348225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.407263+0000","last_change":"2024-03-21T09:08:46.815356+0000","last_active":"2024-03-21T09:10:47.407263+0000","last_peered":"2024-03-21T09:10:47.407263+0000","last_clean":"2024-03-21T09:10:47.407263+0000","last_became_active":"2024-03-21T09:08:46.814964+0000","last_became_peered":"2024-03-21T09:08:46.814964+0000","last_unstale":"2024-03-21T09:10:47.407263+0000","last_undegraded":"2024-03-21T09:10:47.407263+0000","last_fullsized":"2024-03-21T09:10:47.407263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:36.833667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.411684+0000","last_change":"2024-03-21T09:08:46.847759+0000","last_active":"2024-03-21T09:10:47.411684+0000","last_peered":"2024-03-21T09:10:47.411684+0000","last_clean":"2024-03-21T09:10:47.411684+0000","last_became_active":"2024-03-21T09:08:46.847385+0000","last_became_peered":"2024-03-21T09:08:46.847385+0000","last_unstale":"2024-03-21T09:10:47.411684+0000","last_undegraded":"2024-03-21T09:10:47.411684+0000","last_fullsized":"2024-03-21T09:10:47.411684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:55:31.774577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406548+0000","last_change":"2024-03-21T09:08:46.844322+0000","last_active":"2024-03-21T09:10:47.406548+0000","last_peered":"2024-03-21T09:10:47.406548+0000","last_clean":"2024-03-21T09:10:47.406548+0000","last_became_active":"2024-03-21T09:08:46.843927+0000","last_became_peered":"2024-03-21T09:08:46.843927+0000","last_unstale":"2024-03-21T09:10:47.406548+0000","last_undegraded":"2024-03-21T09:10:47.406548+0000","last_fullsized":"2024-03-21T09:10:47.406548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:10.009153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.409488+0000","last_change":"2024-03-21T09:08:46.838649+0000","last_active":"2024-03-21T09:10:47.409488+0000","last_peered":"2024-03-21T09:10:47.409488+0000","last_clean":"2024-03-21T09:10:47.409488+0000","last_became_active":"2024-03-21T09:08:46.836528+0000","last_became_peered":"2024-03-21T09:08:46.836528+0000","last_unstale":"2024-03-21T09:10:47.409488+0000","last_undegraded":"2024-03-21T09:10:47.409488+0000","last_fullsized":"2024-03-21T09:10:47.409488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:04:53.154321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.406502+0000","last_change":"2024-03-21T09:08:46.842336+0000","last_active":"2024-03-21T09:10:47.406502+0000","last_peered":"2024-03-21T09:10:47.406502+0000","last_clean":"2024-03-21T09:10:47.406502+0000","last_became_active":"2024-03-21T09:08:46.841984+0000","last_became_peered":"2024-03-21T09:08:46.841984+0000","last_unstale":"2024-03-21T09:10:47.406502+0000","last_undegraded":"2024-03-21T09:10:47.406502+0000","last_fullsized":"2024-03-21T09:10:47.406502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:57:40.012923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.439038+0000","last_change":"2024-03-21T09:08:46.845127+0000","last_active":"2024-03-21T09:10:47.439038+0000","last_peered":"2024-03-21T09:10:47.439038+0000","last_clean":"2024-03-21T09:10:47.439038+0000","last_became_active":"2024-03-21T09:08:46.844610+0000","last_became_peered":"2024-03-21T09:08:46.844610+0000","last_unstale":"2024-03-21T09:10:47.439038+0000","last_undegraded":"2024-03-21T09:10:47.439038+0000","last_fullsized":"2024-03-21T09:10:47.439038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:50:17.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-21T09:10:47.404751+0000","last_change":"2024-03-21T09:08:46.815609+0000","last_active":"2024-03-21T09:10:47.404751+0000","last_peered":"2024-03-21T09:10:47.404751+0000","last_clean":"2024-03-21T09:10:47.404751+0000","last_became_active":"2024-03-21T09:08:46.815262+0000","last_became_peered":"2024-03-21T09:08:46.815262+0000","last_unstale":"2024-03-21T09:10:47.404751+0000","last_undegraded":"2024-03-21T09:10:47.404751+0000","last_fullsized":"2024-03-21T09:10:47.404751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T09:08:45.782135+0000","last_clean_scrub_stamp":"2024-03-21T09:08:45.782135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:32.038100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285421,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75750,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":950,"num_read_kb":554,"num_write":181,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":224730,"data_compressed":5499,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351029,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":526262,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":1187212,"data_compressed":64715,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808896,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":526464,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070533,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":526384,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332171,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1123131,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":521640,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":589941,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":523028,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":593777,"data_compressed":28422,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":521834,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":527519,"data_compressed":26589,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1117490,"data_compressed":62882,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36293,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68391,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72227,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73718,"data_compressed":1833,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T09:11:56.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:56.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:56 smithi071 ceph-mon[36597]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:56.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:56 smithi071 ceph-mon[36597]: from='client.25480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:56.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:56 smithi053 ceph-mon[36169]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:56.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:56 smithi053 ceph-mon[36169]: from='client.25480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 -- 172.21.15.37:0/2339583928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c102790 msgr2=0x7f0d2c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 --2- 172.21.15.37:0/2339583928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c102790 0x7f0d2c104c00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009860 tx=0x7f0d1402bb70 comp rx=0 tx=0).stop 2024-03-21T09:11:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 -- 172.21.15.37:0/2339583928 shutdown_connections 2024-03-21T09:11:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 --2- 172.21.15.37:0/2339583928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d2c105140 0x7f0d2c1055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 --2- 172.21.15.37:0/2339583928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c102790 0x7f0d2c104c00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 --2- 172.21.15.37:0/2339583928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 -- 172.21.15.37:0/2339583928 >> 172.21.15.37:0/2339583928 conn(0x7f0d2c0fba20 msgr2=0x7f0d2c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 -- 172.21.15.37:0/2339583928 shutdown_connections 2024-03-21T09:11:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.554+0000 7f0d31600640 1 -- 172.21.15.37:0/2339583928 wait complete. 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.555+0000 7f0d31600640 1 Processor -- start 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.555+0000 7f0d31600640 1 -- start start 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d31600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d31600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d2c102790 0x7f0d2c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d31600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c105140 0x7f0d2c1a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d31600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c113800 con 0x7f0d2c105140 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d31600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c113680 con 0x7f0d2c102790 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d31600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c113980 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57192/0 (socket says 172.21.15.37:57192) 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 -- 172.21.15.37:0/397562550 learned_addr learned my addr 172.21.15.37:0/397562550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d23400640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d2c102790 0x7f0d2c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d30c00640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c105140 0x7f0d2c1a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 -- 172.21.15.37:0/397562550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d2c102790 msgr2=0x7f0d2c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d2c102790 0x7f0d2c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 -- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c105140 msgr2=0x7f0d2c1a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c105140 0x7f0d2c1a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.556+0000 7f0d2be00640 1 -- 172.21.15.37:0/397562550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c1a5ac0 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.557+0000 7f0d2be00640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c19dea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00e9f0 tx=0x7f0d1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.557+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00cd80 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.557+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c010910 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.557+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c1a5db0 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.557+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c018990 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.557+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c1a6210 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.558+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c0fd510 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.560+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f0d1c010ac0 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.561+0000 7f0d29600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0d100a2fa0 0x7f0d100a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:56.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.562+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0d1c0cfc70 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.562+0000 7f0d23400640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0d100a2fa0 0x7f0d100a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.562+0000 7f0d23400640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0d100a2fa0 0x7f0d100a5460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009830 tx=0x7f0d14034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:56.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.565+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d1c09bf00 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.784+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d2c0622a0 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.786+0000 7f0d29600640 1 -- 172.21.15.37:0/397562550 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7f0d1c09ffa0 con 0x7f0d2c0ffe20 2024-03-21T09:11:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:56.780 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0d100a2fa0 msgr2=0x7f0d100a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0d100a2fa0 0x7f0d100a5460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009830 tx=0x7f0d14034ab0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 msgr2=0x7f0d2c19dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c19dea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00e9f0 tx=0x7f0d1c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 shutdown_connections 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f0d100a2fa0 0x7f0d100a5460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c105140 0x7f0d2c1a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d2c102790 0x7f0d2c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 --2- 172.21.15.37:0/397562550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffe20 0x7f0d2c19dea0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 >> 172.21.15.37:0/397562550 conn(0x7f0d2c0fba20 msgr2=0x7f0d2c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 shutdown_connections 2024-03-21T09:11:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:56.789+0000 7f0d31600640 1 -- 172.21.15.37:0/397562550 wait complete. 2024-03-21T09:11:56.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:11:56.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 -- 172.21.15.37:0/1048041295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd394101f70 msgr2=0x7fd394104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 --2- 172.21.15.37:0/1048041295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd394101f70 0x7fd394104360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd384009ca0 tx=0x7fd38402c100 comp rx=0 tx=0).stop 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 -- 172.21.15.37:0/1048041295 shutdown_connections 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 --2- 172.21.15.37:0/1048041295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3941071d0 0x7fd394109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 --2- 172.21.15.37:0/1048041295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 0x7fd394106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 --2- 172.21.15.37:0/1048041295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd394101f70 0x7fd394104360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 -- 172.21.15.37:0/1048041295 >> 172.21.15.37:0/1048041295 conn(0x7fd3940fb980 msgr2=0x7fd3940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 -- 172.21.15.37:0/1048041295 shutdown_connections 2024-03-21T09:11:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.035+0000 7fd39a200640 1 -- 172.21.15.37:0/1048041295 wait complete. 2024-03-21T09:11:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.036+0000 7fd39a200640 1 Processor -- start 2024-03-21T09:11:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 -- start start 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd3941977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 0x7fd394197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3941071d0 0x7fd39419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39410d180 con 0x7fd394101f70 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39410d000 con 0x7fd3941071d0 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd39a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39410d300 con 0x7fd3941048a0 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.037+0000 7fd398e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd3941977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 0x7fd394197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd398e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd3941977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37548/0 (socket says 172.21.15.37:37548) 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd398e00640 1 -- 172.21.15.37:0/2347065660 learned_addr learned my addr 172.21.15.37:0/2347065660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd399800640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3941071d0 0x7fd39419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 -- 172.21.15.37:0/2347065660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3941071d0 msgr2=0x7fd39419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3941071d0 0x7fd39419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 -- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 msgr2=0x7fd3941977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd3941977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 -- 172.21.15.37:0/2347065660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39419f3f0 con 0x7fd3941048a0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd398e00640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd3941977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd38be00640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 0x7fd394197d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9c0 tx=0x7fd37c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00ccc0 con 0x7fd3941048a0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.038+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd37c010910 con 0x7fd3941048a0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.039+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0188c0 con 0x7fd3941048a0 2024-03-21T09:11:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.039+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39419f6e0 con 0x7fd3941048a0 2024-03-21T09:11:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.039+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39419fbe0 con 0x7fd3941048a0 2024-03-21T09:11:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.040+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3940fd470 con 0x7fd3941048a0 2024-03-21T09:11:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.042+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd37c010ac0 con 0x7fd3941048a0 2024-03-21T09:11:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.043+0000 7fd392000640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3780a2fb0 0x7fd3780a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.043+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd37c0cfd00 con 0x7fd3941048a0 2024-03-21T09:11:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.043+0000 7fd398e00640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3780a2fb0 0x7fd3780a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.044+0000 7fd398e00640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3780a2fb0 0x7fd3780a5470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd384006010 tx=0x7fd384004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.047+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd37c09bf90 con 0x7fd3941048a0 2024-03-21T09:11:57.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:57 smithi037 ceph-mon[31830]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T09:11:57.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:57 smithi037 ceph-mon[31830]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:57.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:57 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/397562550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.301+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd39402d270 con 0x7fd3941048a0 2024-03-21T09:11:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.301+0000 7fd392000640 1 -- 172.21.15.37:0/2347065660 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd37c0a0030 con 0x7fd3941048a0 2024-03-21T09:11:57.295 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:11:57.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.304+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3780a2fb0 msgr2=0x7fd3780a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.304+0000 7fd39a200640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3780a2fb0 0x7fd3780a5470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd384006010 tx=0x7fd384004a40 comp rx=0 tx=0).stop 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.304+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 msgr2=0x7fd394197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.304+0000 7fd39a200640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 0x7fd394197d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9c0 tx=0x7fd37c00ee80 comp rx=0 tx=0).stop 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 shutdown_connections 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fd3780a2fb0 0x7fd3780a5470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3941071d0 0x7fd39419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941048a0 0x7fd394197d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 --2- 172.21.15.37:0/2347065660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd3941977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 >> 172.21.15.37:0/2347065660 conn(0x7fd3940fb980 msgr2=0x7fd3941028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 shutdown_connections 2024-03-21T09:11:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.305+0000 7fd39a200640 1 -- 172.21.15.37:0/2347065660 wait complete. 2024-03-21T09:11:57.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:57.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:57 smithi071 ceph-mon[36597]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T09:11:57.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:57 smithi071 ceph-mon[36597]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:57.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:57 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/397562550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:57.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:57 smithi053 ceph-mon[36169]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T09:11:57.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:57 smithi053 ceph-mon[36169]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:11:57.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:57 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/397562550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 -- 172.21.15.37:0/1256093858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 msgr2=0x7fef6c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 --2- 172.21.15.37:0/1256093858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c108a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef64009ca0 tx=0x7fef6402c100 comp rx=0 tx=0).stop 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 -- 172.21.15.37:0/1256093858 shutdown_connections 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 --2- 172.21.15.37:0/1256093858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c108a00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 --2- 172.21.15.37:0/1256093858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1017b0 0x7fef6c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 --2- 172.21.15.37:0/1256093858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1005c0 0x7fef6c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 -- 172.21.15.37:0/1256093858 >> 172.21.15.37:0/1256093858 conn(0x7fef6c0fbb30 msgr2=0x7fef6c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 -- 172.21.15.37:0/1256093858 shutdown_connections 2024-03-21T09:11:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.550+0000 7fef72400640 1 -- 172.21.15.37:0/1256093858 wait complete. 2024-03-21T09:11:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.551+0000 7fef72400640 1 Processor -- start 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.551+0000 7fef72400640 1 -- start start 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef72400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1005c0 0x7fef6c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef72400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 0x7fef6c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef72400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef72400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c10c220 con 0x7fef6c1005c0 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef72400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c10c0a0 con 0x7fef6c1017b0 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef72400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c10c3a0 con 0x7fef6c102170 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 0x7fef6c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1005c0 0x7fef6c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 0x7fef6c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50480/0 (socket says 172.21.15.37:50480) 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 -- 172.21.15.37:0/2273652317 learned_addr learned my addr 172.21.15.37:0/2273652317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6be00640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 -- 172.21.15.37:0/2273652317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 msgr2=0x7fef6c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 -- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1005c0 msgr2=0x7fef6c197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.552+0000 7fef6aa00640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1005c0 0x7fef6c197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef6aa00640 1 -- 172.21.15.37:0/2273652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c19f560 con 0x7fef6c1017b0 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef6be00640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef6aa00640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 0x7fef6c197ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef5400d950 tx=0x7fef5400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54017070 con 0x7fef6c1017b0 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c19f850 con 0x7fef6c1017b0 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef54009c30 con 0x7fef6c1017b0 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5400f6b0 con 0x7fef6c1017b0 2024-03-21T09:11:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.553+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c19fda0 con 0x7fef6c1017b0 2024-03-21T09:11:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.555+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c1a0000 con 0x7fef6c1017b0 2024-03-21T09:11:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.556+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fef540109d0 con 0x7fef6c1017b0 2024-03-21T09:11:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.558+0000 7fef63e00640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fef500a2fb0 0x7fef500a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.558+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef540cf570 con 0x7fef6c1017b0 2024-03-21T09:11:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.558+0000 7fef6b400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fef500a2fb0 0x7fef500a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.559+0000 7fef6b400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fef500a2fb0 0x7fef500a5470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef6c101610 tx=0x7fef5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.562+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef54009de0 con 0x7fef6c1017b0 2024-03-21T09:11:57.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.765+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef6c1a01d0 con 0x7fef6c1017b0 2024-03-21T09:11:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.765+0000 7fef63e00640 1 -- 172.21.15.37:0/2273652317 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef5409b800 con 0x7fef6c1017b0 2024-03-21T09:11:57.759 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fef500a2fb0 msgr2=0x7fef500a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fef500a2fb0 0x7fef500a5470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fef6c101610 tx=0x7fef5c005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 msgr2=0x7fef6c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 0x7fef6c197ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef5400d950 tx=0x7fef5400de10 comp rx=0 tx=0).stop 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 shutdown_connections 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fef500a2fb0 0x7fef500a5470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c102170 0x7fef6c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef6c1017b0 0x7fef6c197ed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 --2- 172.21.15.37:0/2273652317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1005c0 0x7fef6c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 >> 172.21.15.37:0/2273652317 conn(0x7fef6c0fbb30 msgr2=0x7fef6c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 shutdown_connections 2024-03-21T09:11:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:57.768+0000 7fef72400640 1 -- 172.21.15.37:0/2273652317 wait complete. 2024-03-21T09:11:57.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:58.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 -- 172.21.15.37:0/2839036514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 msgr2=0x7fb044073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 --2- 172.21.15.37:0/2839036514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb044073b30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb034009860 tx=0x7fb03402bb70 comp rx=0 tx=0).stop 2024-03-21T09:11:58.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 -- 172.21.15.37:0/2839036514 shutdown_connections 2024-03-21T09:11:58.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 --2- 172.21.15.37:0/2839036514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044109570 0x7fb04410ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 --2- 172.21.15.37:0/2839036514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb044074100 0x7fb044074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 --2- 172.21.15.37:0/2839036514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb044073b30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.014+0000 7fb04ae00640 1 -- 172.21.15.37:0/2839036514 >> 172.21.15.37:0/2839036514 conn(0x7fb0440fcc20 msgr2=0x7fb0440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:58.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.015+0000 7fb04ae00640 1 -- 172.21.15.37:0/2839036514 shutdown_connections 2024-03-21T09:11:58.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.015+0000 7fb04ae00640 1 -- 172.21.15.37:0/2839036514 wait complete. 2024-03-21T09:11:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 Processor -- start 2024-03-21T09:11:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 -- start start 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044074100 0x7fb044199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb04419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb044109570 0x7fb0441a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410f4f0 con 0x7fb044074100 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410f370 con 0x7fb044109570 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb04ae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410f670 con 0x7fb0440756e0 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb043e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044074100 0x7fb044199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb043400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb04419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb048e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb044109570 0x7fb0441a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb043400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb04419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57246/0 (socket says 172.21.15.37:57246) 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.016+0000 7fb043400640 1 -- 172.21.15.37:0/705192270 learned_addr learned my addr 172.21.15.37:0/705192270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb043400640 1 -- 172.21.15.37:0/705192270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb044109570 msgr2=0x7fb0441a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb043400640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb044109570 0x7fb0441a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb043400640 1 -- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044074100 msgr2=0x7fb044199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb043400640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044074100 0x7fb044199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb043400640 1 -- 172.21.15.37:0/705192270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0441a17e0 con 0x7fb0440756e0 2024-03-21T09:11:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb043400640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb04419a100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9c0 tx=0x7fb03000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000cde0 con 0x7fb0440756e0 2024-03-21T09:11:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0300035f0 con 0x7fb0440756e0 2024-03-21T09:11:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0441a1ad0 con 0x7fb0440756e0 2024-03-21T09:11:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.017+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030010690 con 0x7fb0440756e0 2024-03-21T09:11:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.018+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0441a2070 con 0x7fb0440756e0 2024-03-21T09:11:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.019+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0441082a0 con 0x7fb0440756e0 2024-03-21T09:11:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.024+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb0300037a0 con 0x7fb0440756e0 2024-03-21T09:11:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.025+0000 7fb040c00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb0280a3030 0x7fb0280a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.026+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb0300cfb80 con 0x7fb0440756e0 2024-03-21T09:11:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.026+0000 7fb043e00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb0280a3030 0x7fb0280a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.026+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0300fd7e0 con 0x7fb0440756e0 2024-03-21T09:11:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.027+0000 7fb043e00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb0280a3030 0x7fb0280a54f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb034009830 tx=0x7fb034034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:58.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:58 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2347065660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:58.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:58 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2273652317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.234+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb04402d070 con 0x7fb0440756e0 2024-03-21T09:11:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.234+0000 7fb040c00640 1 -- 172.21.15.37:0/705192270 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb03009c4c0 con 0x7fb0440756e0 2024-03-21T09:11:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:11:58.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.237+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb0280a3030 msgr2=0x7fb0280a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.237+0000 7fb04ae00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb0280a3030 0x7fb0280a54f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb034009830 tx=0x7fb034034ab0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.237+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 msgr2=0x7fb04419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.237+0000 7fb04ae00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb04419a100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9c0 tx=0x7fb03000ee80 comp rx=0 tx=0).stop 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 shutdown_connections 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb0280a3030 0x7fb0280a54f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb044109570 0x7fb0441a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0440756e0 0x7fb04419a100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 --2- 172.21.15.37:0/705192270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044074100 0x7fb044199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 >> 172.21.15.37:0/705192270 conn(0x7fb0440fcc20 msgr2=0x7fb044105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 shutdown_connections 2024-03-21T09:11:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.238+0000 7fb04ae00640 1 -- 172.21.15.37:0/705192270 wait complete. 2024-03-21T09:11:58.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:11:58.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:58 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2347065660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:58.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:58 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2273652317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:58.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:58 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2347065660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:58.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:58 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2273652317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:58.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 -- 172.21.15.37:0/921084611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c075660 msgr2=0x7fb52c073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 --2- 172.21.15.37:0/921084611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c075660 0x7fb52c073ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 -- 172.21.15.37:0/921084611 shutdown_connections 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 --2- 172.21.15.37:0/921084611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52c1093c0 0x7fb52c10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 --2- 172.21.15.37:0/921084611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c074080 0x7fb52c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 --2- 172.21.15.37:0/921084611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c075660 0x7fb52c073ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 -- 172.21.15.37:0/921084611 >> 172.21.15.37:0/921084611 conn(0x7fb52c0fca80 msgr2=0x7fb52c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 -- 172.21.15.37:0/921084611 shutdown_connections 2024-03-21T09:11:58.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.469+0000 7fb532e00640 1 -- 172.21.15.37:0/921084611 wait complete. 2024-03-21T09:11:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 Processor -- start 2024-03-21T09:11:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 -- start start 2024-03-21T09:11:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 0x7fb52c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52c075660 0x7fb52c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c1093c0 0x7fb52c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c10f3c0 con 0x7fb52c075660 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c10f240 con 0x7fb52c1093c0 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.471+0000 7fb532e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c10f540 con 0x7fb52c074080 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 0x7fb52c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 0x7fb52c199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57272/0 (socket says 172.21.15.37:57272) 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 -- 172.21.15.37:0/3618654563 learned_addr learned my addr 172.21.15.37:0/3618654563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb532400640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c1093c0 0x7fb52c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531000640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52c075660 0x7fb52c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 -- 172.21.15.37:0/3618654563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c1093c0 msgr2=0x7fb52c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c1093c0 0x7fb52c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 -- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52c075660 msgr2=0x7fb52c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52c075660 0x7fb52c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb531a00640 1 -- 172.21.15.37:0/3618654563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c1a1620 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.472+0000 7fb532400640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c1093c0 0x7fb52c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.473+0000 7fb531a00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 0x7fb52c199a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb520005ab0 tx=0x7fb52002c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.473+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520043020 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.473+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5200054e0 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.473+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c1a18b0 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.473+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520003cc0 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.473+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1a1e50 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.474+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c108180 con 0x7fb52c074080 2024-03-21T09:11:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.476+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb520046020 con 0x7fb52c074080 2024-03-21T09:11:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.477+0000 7fb52a000640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5140a3030 0x7fb5140a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.478+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5200f1bb0 con 0x7fb52c074080 2024-03-21T09:11:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.478+0000 7fb531000640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5140a3030 0x7fb5140a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.478+0000 7fb531000640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5140a3030 0x7fb5140a54f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb52c19af10 tx=0x7fb518005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:58.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.480+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb52004d020 con 0x7fb52c074080 2024-03-21T09:11:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.681+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb52c02d2c0 con 0x7fb52c074080 2024-03-21T09:11:58.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.682+0000 7fb52a000640 1 -- 172.21.15.37:0/3618654563 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb5200be5a0 con 0x7fb52c074080 2024-03-21T09:11:58.676 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:11:58.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.685+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5140a3030 msgr2=0x7fb5140a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.685+0000 7fb532e00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5140a3030 0x7fb5140a54f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb52c19af10 tx=0x7fb518005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:58.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.685+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 msgr2=0x7fb52c199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.685+0000 7fb532e00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 0x7fb52c199a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb520005ab0 tx=0x7fb52002c750 comp rx=0 tx=0).stop 2024-03-21T09:11:58.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 shutdown_connections 2024-03-21T09:11:58.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb5140a3030 0x7fb5140a54f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52c1093c0 0x7fb52c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52c075660 0x7fb52c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 --2- 172.21.15.37:0/3618654563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb52c074080 0x7fb52c199a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 >> 172.21.15.37:0/3618654563 conn(0x7fb52c0fca80 msgr2=0x7fb52c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:58.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 shutdown_connections 2024-03-21T09:11:58.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.686+0000 7fb532e00640 1 -- 172.21.15.37:0/3618654563 wait complete. 2024-03-21T09:11:58.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:58.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 -- 172.21.15.37:0/1451842737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520109450 msgr2=0x7f652010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 --2- 172.21.15.37:0/1451842737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520109450 0x7f652010b8e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6518009ca0 tx=0x7f651802c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 -- 172.21.15.37:0/1451842737 shutdown_connections 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 --2- 172.21.15.37:0/1451842737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520109450 0x7f652010b8e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 --2- 172.21.15.37:0/1451842737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520074100 0x7f6520074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 --2- 172.21.15.37:0/1451842737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200756e0 0x7f6520073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 -- 172.21.15.37:0/1451842737 >> 172.21.15.37:0/1451842737 conn(0x7f65200fcb00 msgr2=0x7f65200fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 -- 172.21.15.37:0/1451842737 shutdown_connections 2024-03-21T09:11:58.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.960+0000 7f6524a00640 1 -- 172.21.15.37:0/1451842737 wait complete. 2024-03-21T09:11:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.962+0000 7f6524a00640 1 Processor -- start 2024-03-21T09:11:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.962+0000 7f6524a00640 1 -- start start 2024-03-21T09:11:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.962+0000 7f6524a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520074100 0x7f6520199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.962+0000 7f6524a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200756e0 0x7f6520199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.962+0000 7f6524a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520109450 0x7f65201a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f6524a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652010f3d0 con 0x7f65200756e0 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f6524a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652010f250 con 0x7f6520109450 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f6524a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652010f550 con 0x7f6520074100 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520074100 0x7f6520199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520109450 0x7f65201a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520074100 0x7f6520199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57294/0 (socket says 172.21.15.37:57294) 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f6515e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200756e0 0x7f6520199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651e800640 1 -- 172.21.15.37:0/3328837696 learned_addr learned my addr 172.21.15.37:0/3328837696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651f200640 1 -- 172.21.15.37:0/3328837696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520074100 msgr2=0x7f6520199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651f200640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520074100 0x7f6520199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651f200640 1 -- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200756e0 msgr2=0x7f6520199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651f200640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200756e0 0x7f6520199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.963+0000 7f651f200640 1 -- 172.21.15.37:0/3328837696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65201a1620 con 0x7f6520109450 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.964+0000 7f651f200640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520109450 0x7f65201a0fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6518037040 tx=0x7f6518032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.964+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518041020 con 0x7f6520109450 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.964+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6518034cd0 con 0x7f6520109450 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.964+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518039040 con 0x7f6520109450 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.964+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65201a18b0 con 0x7f6520109450 2024-03-21T09:11:58.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.964+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65201a1d20 con 0x7f6520109450 2024-03-21T09:11:58.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.965+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6520108180 con 0x7f6520109450 2024-03-21T09:11:58.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.967+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6518044020 con 0x7f6520109450 2024-03-21T09:11:58.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.968+0000 7f6517e00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f65040a3020 0x7f65040a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:58.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.968+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f65180f1620 con 0x7f6520109450 2024-03-21T09:11:58.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.968+0000 7f651e800640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f65040a3020 0x7f65040a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.969+0000 7f651e800640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f65040a3020 0x7f65040a54e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6510009c90 tx=0x7f6510005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:58.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:58.977+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f651804b020 con 0x7f6520109450 2024-03-21T09:11:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.243+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f652002d070 con 0x7f6520109450 2024-03-21T09:11:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.244+0000 7f6517e00640 1 -- 172.21.15.37:0/3328837696 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f65180be0a0 con 0x7f6520109450 2024-03-21T09:11:59.238 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:59.238 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:11:48.827236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:11:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f65040a3020 msgr2=0x7f65040a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f65040a3020 0x7f65040a54e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6510009c90 tx=0x7f6510005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520109450 msgr2=0x7f65201a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520109450 0x7f65201a0fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6518037040 tx=0x7f6518032ea0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 shutdown_connections 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f65040a3020 0x7f65040a54e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520109450 0x7f65201a0fb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200756e0 0x7f6520199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 --2- 172.21.15.37:0/3328837696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6520074100 0x7f6520199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 >> 172.21.15.37:0/3328837696 conn(0x7f65200fcb00 msgr2=0x7f6520105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.247+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 shutdown_connections 2024-03-21T09:11:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.248+0000 7f6524a00640 1 -- 172.21.15.37:0/3328837696 wait complete. 2024-03-21T09:11:59.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:59.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:59 smithi037 ceph-mon[31830]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:11:59.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:59 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/705192270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:59.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:11:59 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3618654563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:59.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:59 smithi071 ceph-mon[36597]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:11:59.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:59 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/705192270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:59.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:11:59 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3618654563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:59.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:59 smithi053 ceph-mon[36169]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:11:59.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:59 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/705192270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:59.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:11:59 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3618654563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 -- 172.21.15.37:0/2877980635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2201027b0 msgr2=0x7fa220102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 --2- 172.21.15.37:0/2877980635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2201027b0 0x7fa220102c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009ca0 tx=0x7fa20c02c100 comp rx=0 tx=0).stop 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 -- 172.21.15.37:0/2877980635 shutdown_connections 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 --2- 172.21.15.37:0/2877980635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa220103170 0x7fa220109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 --2- 172.21.15.37:0/2877980635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2201027b0 0x7fa220102c30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 --2- 172.21.15.37:0/2877980635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa2201019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 -- 172.21.15.37:0/2877980635 >> 172.21.15.37:0/2877980635 conn(0x7fa2200fcb30 msgr2=0x7fa2200fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 -- 172.21.15.37:0/2877980635 shutdown_connections 2024-03-21T09:11:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.490+0000 7fa226000640 1 -- 172.21.15.37:0/2877980635 wait complete. 2024-03-21T09:11:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.491+0000 7fa226000640 1 Processor -- start 2024-03-21T09:11:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 -- start start 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa220197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2201027b0 0x7fa220197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa220103170 0x7fa22019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22010d220 con 0x7fa2201015c0 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22010d0a0 con 0x7fa2201027b0 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa226000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22010d3a0 con 0x7fa220103170 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa224c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa220197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa224c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa220197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37644/0 (socket says 172.21.15.37:37644) 2024-03-21T09:11:59.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa224c00640 1 -- 172.21.15.37:0/3106570010 learned_addr learned my addr 172.21.15.37:0/3106570010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:11:59.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa21fe00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2201027b0 0x7fa220197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:59.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.492+0000 7fa225600640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa220103170 0x7fa22019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21fe00640 1 -- 172.21.15.37:0/3106570010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa220103170 msgr2=0x7fa22019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21fe00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa220103170 0x7fa22019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21fe00640 1 -- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 msgr2=0x7fa220197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21fe00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa220197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21fe00640 1 -- 172.21.15.37:0/3106570010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22019f480 con 0x7fa2201027b0 2024-03-21T09:11:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa224c00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa220197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21fe00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2201027b0 0x7fa220197da0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009bf0 tx=0x7fa20c035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c039040 con 0x7fa2201027b0 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.493+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa20c003970 con 0x7fa2201027b0 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.494+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22019f710 con 0x7fa2201027b0 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.494+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004360 con 0x7fa2201027b0 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.494+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22019fc20 con 0x7fa2201027b0 2024-03-21T09:11:59.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.495+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22004f470 con 0x7fa2201027b0 2024-03-21T09:11:59.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.497+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa20c046030 con 0x7fa2201027b0 2024-03-21T09:11:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.499+0000 7fa21d600640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2040a3020 0x7fa2040a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.499+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa20c0f1310 con 0x7fa2201027b0 2024-03-21T09:11:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.499+0000 7fa224c00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2040a3020 0x7fa2040a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.500+0000 7fa224c00640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2040a3020 0x7fa2040a54e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa2100058c0 tx=0x7fa210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:59.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.502+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa20c04d060 con 0x7fa2201027b0 2024-03-21T09:11:59.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.753+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2200622a0 con 0x7fa2201027b0 2024-03-21T09:11:59.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.754+0000 7fa21d600640 1 -- 172.21.15.37:0/3106570010 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24664 (secure 0 0 0) 0x7fa20c0bdd90 con 0x7fa2201027b0 2024-03-21T09:11:59.749 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:11:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:10:47.397472+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:59.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2040a3020 msgr2=0x7fa2040a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2040a3020 0x7fa2040a54e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa2100058c0 tx=0x7fa210005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2201027b0 msgr2=0x7fa220197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2201027b0 0x7fa220197da0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009bf0 tx=0x7fa20c035d10 comp rx=0 tx=0).stop 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 shutdown_connections 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fa2040a3020 0x7fa2040a54e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa220103170 0x7fa22019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2201027b0 0x7fa220197da0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 --2- 172.21.15.37:0/3106570010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2201015c0 0x7fa220197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 >> 172.21.15.37:0/3106570010 conn(0x7fa2200fcb30 msgr2=0x7fa220105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:59.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 shutdown_connections 2024-03-21T09:11:59.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:11:59.757+0000 7fa226000640 1 -- 172.21.15.37:0/3106570010 wait complete. 2024-03-21T09:11:59.770 DEBUG:tasks.fwd_scrub:Yielding 2024-03-21T09:11:59.770 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T09:11:59.780 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T09:11:59.780 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:11:59.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-21T09:11:59.781 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-21T09:11:59.781 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:11:59.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:59.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:11:59.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 -- 172.21.15.37:0/3105554216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b8105250 msgr2=0x7f20b81056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 --2- 172.21.15.37:0/3105554216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b8105250 0x7f20b81056d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009be0 tx=0x7f20b002bef0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 -- 172.21.15.37:0/3105554216 shutdown_connections 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 --2- 172.21.15.37:0/3105554216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b8105250 0x7f20b81056d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 --2- 172.21.15.37:0/3105554216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b81028a0 0x7f20b8104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 --2- 172.21.15.37:0/3105554216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b80fff30 0x7f20b8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.004+0000 7f20bee00640 1 -- 172.21.15.37:0/3105554216 >> 172.21.15.37:0/3105554216 conn(0x7f20b80fbb30 msgr2=0x7f20b80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.005+0000 7f20bee00640 1 -- 172.21.15.37:0/3105554216 shutdown_connections 2024-03-21T09:11:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.005+0000 7f20bee00640 1 -- 172.21.15.37:0/3105554216 wait complete. 2024-03-21T09:12:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 Processor -- start 2024-03-21T09:12:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 -- 172.21.15.37:0/1657256241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 msgr2=0x7f164c102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 --2- 172.21.15.37:0/1657256241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c102b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1638009900 tx=0x7f163802bc40 comp rx=0 tx=0).stop 2024-03-21T09:12:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 -- start start 2024-03-21T09:12:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 -- 172.21.15.37:0/1657256241 shutdown_connections 2024-03-21T09:12:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 --2- 172.21.15.37:0/1657256241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164c1030d0 0x7f164c109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 --2- 172.21.15.37:0/1657256241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c102b90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 --2- 172.21.15.37:0/1657256241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 0x7f164c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 -- 172.21.15.37:0/1657256241 >> 172.21.15.37:0/1657256241 conn(0x7f164c0fcab0 msgr2=0x7f164c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 -- 172.21.15.37:0/1657256241 shutdown_connections 2024-03-21T09:12:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f1652600640 1 -- 172.21.15.37:0/1657256241 wait complete. 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 0x7f20b819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b81028a0 0x7f20b819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b8105250 0x7f20b81a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b8113960 con 0x7f20b8105250 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81137e0 con 0x7f20b80fff30 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20bee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b8113ae0 con 0x7f20b81028a0 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 0x7f20b819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 0x7f20b819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50598/0 (socket says 172.21.15.37:50598) 2024-03-21T09:12:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20b7e00640 1 -- 172.21.15.37:0/3619769579 learned_addr learned my addr 172.21.15.37:0/3619769579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.006+0000 7f20b7400640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b81028a0 0x7f20b819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20bce00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b8105250 0x7f20b81a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b7e00640 1 -- 172.21.15.37:0/3619769579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b81028a0 msgr2=0x7f20b819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b7e00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b81028a0 0x7f20b819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b7e00640 1 -- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b8105250 msgr2=0x7f20b81a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b7e00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b8105250 0x7f20b81a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b7e00640 1 -- 172.21.15.37:0/3619769579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b81a5b80 con 0x7f20b80fff30 2024-03-21T09:12:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f1652600640 1 Processor -- start 2024-03-21T09:12:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b7e00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 0x7f20b819df60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e9f0 tx=0x7f20a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f1652600640 1 -- start start 2024-03-21T09:12:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b4c00640 1 -- 172.21.15.37:0/3619769579 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a800cd80 con 0x7f20b80fff30 2024-03-21T09:12:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.007+0000 7f20b4c00640 1 -- 172.21.15.37:0/3619769579 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20a8010910 con 0x7f20b80fff30 2024-03-21T09:12:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b81a5e70 con 0x7f20b80fff30 2024-03-21T09:12:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f20b4c00640 1 -- 172.21.15.37:0/3619769579 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8018990 con 0x7f20b80fff30 2024-03-21T09:12:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b81a6330 con 0x7f20b80fff30 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1652600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 0x7f164c075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1652600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1652600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164c1030d0 0x7f164c073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1652600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c10d260 con 0x7f164c1030d0 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1652600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c10d0e0 con 0x7f164c101520 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1652600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c10d3e0 con 0x7f164c102710 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1651200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 0x7f164c075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f164be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f164be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57316/0 (socket says 172.21.15.37:57316) 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1651200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 0x7f164c075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50602/0 (socket says 172.21.15.37:50602) 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f164be00640 1 -- 172.21.15.37:0/2973151206 learned_addr learned my addr 172.21.15.37:0/2973151206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f1651c00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164c1030d0 0x7f164c073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.008+0000 7f164be00640 1 -- 172.21.15.37:0/2973151206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 msgr2=0x7f164c075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f164be00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 0x7f164c075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f164be00640 1 -- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164c1030d0 msgr2=0x7f164c073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f164be00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164c1030d0 0x7f164c073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f164be00640 1 -- 172.21.15.37:0/2973151206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c074860 con 0x7f164c102710 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f164be00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c073a90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f163802c8f0 tx=0x7f1638005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638041070 con 0x7f164c102710 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16380044d0 con 0x7f164c102710 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638034440 con 0x7f164c102710 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f1652600640 1 -- 172.21.15.37:0/2973151206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c1a1930 con 0x7f164c102710 2024-03-21T09:12:00.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.009+0000 7f1652600640 1 -- 172.21.15.37:0/2973151206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c1a1ed0 con 0x7f164c102710 2024-03-21T09:12:00.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.011+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1628005290 con 0x7f164c102710 2024-03-21T09:12:00.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 Processor -- start 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 -- start start 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d6c00 0x557bb25d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb2536eb0 con 0x557bb25d10a0 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb24dfa60 con 0x557bb25d1a90 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.012+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb2532f50 con 0x557bb25d6c00 2024-03-21T09:12:00.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f1638044070 con 0x7f164c102710 2024-03-21T09:12:00.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37682/0 (socket says 172.21.15.37:37682) 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 -- 172.21.15.37:0/207293856 learned_addr learned my addr 172.21.15.37:0/207293856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20b4c00640 1 -- 172.21.15.37:0/3619769579 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f20a8010ac0 con 0x7f20b80fff30 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20b7400640 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d6c00 0x557bb25d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 -- 172.21.15.37:0/207293856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d6c00 msgr2=0x557bb25d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d6c00 0x557bb25d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 -- 172.21.15.37:0/207293856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 msgr2=0x557bb25d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 -- 172.21.15.37:0/207293856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bb23137e0 con 0x557bb25d10a0 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.013+0000 7f20bce00640 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25d14c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20b002c500 tx=0x7f20b0009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bacea073be4313 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.014+0000 7f20ad600640 1 -- 172.21.15.37:0/207293856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0043030 con 0x557bb25d10a0 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.014+0000 7f20ad600640 1 -- 172.21.15.37:0/207293856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b0033580 con 0x557bb25d10a0 2024-03-21T09:12:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.014+0000 7f20ad600640 1 -- 172.21.15.37:0/207293856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b002c890 con 0x557bb25d10a0 2024-03-21T09:12:00.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.014+0000 7f1649600640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16340a3010 0x7f16340a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.014+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f16380f1450 con 0x7f164c102710 2024-03-21T09:12:00.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.014+0000 7f20b4c00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f209c0a3030 0x7f209c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f1651200640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16340a3010 0x7f16340a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 -- 172.21.15.37:0/207293856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 msgr2=0x557bb25d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25d14c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20b002c500 tx=0x7f20b0009330 comp rx=0 tx=0).stop 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20b4c00640 1 -- 172.21.15.37:0/3619769579 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f20a80cfb80 con 0x7f20b80fff30 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 -- 172.21.15.37:0/207293856 shutdown_connections 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20b7e00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f209c0a3030 0x7f209c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d6c00 0x557bb25d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/207293856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25d14c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.015+0000 7f20c1ba6740 1 -- 172.21.15.37:0/207293856 >> 172.21.15.37:0/207293856 conn(0x557bb25cc610 msgr2=0x557bb25cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.016+0000 7f20c1ba6740 1 -- 172.21.15.37:0/207293856 shutdown_connections 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.016+0000 7f20b7e00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f209c0a3030 0x7f209c0a54f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x557bb25d2500 tx=0x7f20a80cfe40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.016+0000 7f20c1ba6740 1 -- 172.21.15.37:0/207293856 wait complete. 2024-03-21T09:12:00.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.016+0000 7f1651200640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16340a3010 0x7f16340a54d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f164c102570 tx=0x7f1640009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.016+0000 7f20c1ba6740 1 -- start start 2024-03-21T09:12:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 Processor -- start 2024-03-21T09:12:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d10a0 0x557bb25db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d6c00 0x557bb25e2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb25d9b50 con 0x557bb25d6c00 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb24f2eb0 con 0x557bb25d1a90 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb25d9cd0 con 0x557bb25d10a0 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20c1ba6740 10 client.0 _ll_register_callbacks cb 0x557bb24e7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20bce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d10a0 0x557bb25db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20b7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20bce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d10a0 0x557bb25db1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57328/0 (socket says 172.21.15.37:57328) 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20bce00640 1 -- 172.21.15.37:0/2734726619 learned_addr learned my addr 172.21.15.37:0/2734726619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20b7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50608/0 (socket says 172.21.15.37:50608) 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.017+0000 7f20bce00640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d6c00 0x557bb25e2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16380bde40 con 0x7f164c102710 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f20b7400640 1 -- 172.21.15.37:0/2734726619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d10a0 msgr2=0x557bb25db1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f20b7400640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d10a0 0x557bb25db1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f20b7400640 1 -- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d6c00 msgr2=0x557bb25e2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f20b7400640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d6c00 0x557bb25e2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f20b7400640 1 -- 172.21.15.37:0/2734726619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bb2373f70 con 0x557bb25d1a90 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.018+0000 7f20bce00640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d6c00 0x557bb25e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20b7400640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25db6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x557bb25cd580 tx=0x7f20a4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39908fce1d60fb8a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20a2000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20a2000640 1 -- 172.21.15.37:0/2734726619 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004020 con 0x557bb25d1a90 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b0033580 con 0x557bb25d1a90 2024-03-21T09:12:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20a2000640 1 -- 172.21.15.37:0/2734726619 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f20a40041c0 con 0x557bb25d1a90 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20a2000640 1 -- 172.21.15.37:0/2734726619 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4025050 con 0x557bb25d1a90 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20c1ba6740 10 client.25514 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557bb220b170 con 0x557bb25d1a90 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.019+0000 7f20a2000640 1 -- 172.21.15.37:0/2734726619 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b0033580 con 0x557bb25d1a90 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20a2000640 1 -- 172.21.15.37:0/2734726619 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f20a40037b0 con 0x557bb25d1a90 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20c1ba6740 10 client.25514 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20c1ba6740 10 client.25514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20c1ba6740 20 client.25514 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25e7050 0x557bb25e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20c1ba6740 4 client.25514 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.020+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bb25443f0 con 0x557bb25e7050 2024-03-21T09:12:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.021+0000 7f20bce00640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25e7050 0x557bb25e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.021+0000 7f20bce00640 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25e7050 0x557bb25e9440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5459908436d8f0ec server_cookie=20920f1227ef35e9 in_seq=0 out_seq=0 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.021+0000 7f20a2000640 10 client.25514 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.022+0000 7f20a2000640 1 -- 172.21.15.37:0/2734726619 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557bb25443f0 con 0x557bb25e7050 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.022+0000 7f20a2000640 10 client.25514 handle_command_reply: tid=0 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 client.25514 shutdown 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 msgr2=0x557bb25db6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25db6f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x557bb25cd580 tx=0x7f20a4009f90 comp rx=0 tx=0).stop 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 shutdown_connections 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d6c00 0x557bb25e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25e7050 0x557bb25e9440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25db6f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2734726619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d10a0 0x557bb25db1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 >> 172.21.15.37:0/2734726619 conn(0x557bb25cc610 msgr2=0x557bb25cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20a2000640 0 client.25514 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 shutdown_connections 2024-03-21T09:12:00.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2734726619 wait complete. 2024-03-21T09:12:00.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.023+0000 7f20c1ba6740 20 client.25514 trim_cache size 0 max 16384 2024-03-21T09:12:00.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.042+0000 7f20c1ba6740 1 Processor -- start 2024-03-21T09:12:00.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 -- start start 2024-03-21T09:12:00.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 0x557bb25d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81b2bc0 con 0x557bb25d10a0 2024-03-21T09:12:00.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81b06c0 con 0x557bb25d1a90 2024-03-21T09:12:00.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81ae760 con 0x557bb25cc610 2024-03-21T09:12:00.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20bce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 0x557bb25d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20bce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 0x557bb25d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57344/0 (socket says 172.21.15.37:57344) 2024-03-21T09:12:00.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20bce00640 1 -- 172.21.15.37:0/2094105197 learned_addr learned my addr 172.21.15.37:0/2094105197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20b7400640 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20bce00640 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20bce00640 1 -- 172.21.15.37:0/2094105197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 msgr2=0x557bb25d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.043+0000 7f20bce00640 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20bce00640 1 -- 172.21.15.37:0/2094105197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 msgr2=0x557bb25e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20bce00640 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20bce00640 1 -- 172.21.15.37:0/2094105197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bb25dc2e0 con 0x557bb25cc610 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20bce00640 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 0x557bb25d2530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f20b00099a0 tx=0x7f20b0034f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd65d91c7d8dbaea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20acc00640 1 -- 172.21.15.37:0/2094105197 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0043040 con 0x557bb25cc610 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20acc00640 1 -- 172.21.15.37:0/2094105197 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b0003cb0 con 0x557bb25cc610 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.044+0000 7f20acc00640 1 -- 172.21.15.37:0/2094105197 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0056b60 con 0x557bb25cc610 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2094105197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 msgr2=0x557bb25d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 0x557bb25d2530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f20b00099a0 tx=0x7f20b0034f90 comp rx=0 tx=0).stop 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2094105197 shutdown_connections 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25d1a90 0x557bb25d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/2094105197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25cc610 0x557bb25d2530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2094105197 >> 172.21.15.37:0/2094105197 conn(0x557bb25d04a0 msgr2=0x557bb25da960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2094105197 shutdown_connections 2024-03-21T09:12:00.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.045+0000 7f20c1ba6740 1 -- 172.21.15.37:0/2094105197 wait complete. 2024-03-21T09:12:00.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 -- start start 2024-03-21T09:12:00.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 Processor -- start 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 0x557bb25e2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d1a90 0x557bb25e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81aed60 con 0x557bb25d10a0 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81b0060 con 0x557bb25cc610 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.046+0000 7f20c1ba6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81af3a0 con 0x557bb25d1a90 2024-03-21T09:12:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20c1ba6740 10 client.0 _ll_register_callbacks cb 0x7f20a4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 0x557bb25e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 0x557bb25e2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50622/0 (socket says 172.21.15.37:50622) 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 -- 172.21.15.37:0/1766654409 learned_addr learned my addr 172.21.15.37:0/1766654409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20b7400640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d1a90 0x557bb25e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 -- 172.21.15.37:0/1766654409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d1a90 msgr2=0x557bb25e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d1a90 0x557bb25e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 -- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 msgr2=0x557bb25e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 -- 172.21.15.37:0/1766654409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bb25cd080 con 0x557bb25cc610 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20bce00640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 0x557bb25e2380 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009b30 tx=0x7f20b0033640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8b0f60dd11659c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.047+0000 7f20a2000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20a2000640 1 -- 172.21.15.37:0/1766654409 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0043070 con 0x557bb25cc610 2024-03-21T09:12:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b0003cb0 con 0x557bb25cc610 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20a2000640 1 -- 172.21.15.37:0/1766654409 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f20b0056ae0 con 0x557bb25cc610 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20a2000640 1 -- 172.21.15.37:0/1766654409 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20b0056ce0 con 0x557bb25cc610 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 10 client.25520 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557bb2643550 con 0x557bb25cc610 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20a2000640 1 -- 172.21.15.37:0/1766654409 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b0003cb0 con 0x557bb25cc610 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20a2000640 1 -- 172.21.15.37:0/1766654409 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f20b0004470 con 0x557bb25cc610 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 10 client.25520 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 10 client.25520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 20 client.25520 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25d6c00 0x557bb2645910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 4 client.25520 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.048+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557bb253e740 con 0x557bb25d6c00 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.049+0000 7f20b7400640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25d6c00 0x557bb2645910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.049+0000 7f20b7400640 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25d6c00 0x557bb2645910 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70dec1b91d87667f server_cookie=7d20d332f791ee56 in_seq=0 out_seq=0 2024-03-21T09:12:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.049+0000 7f20a2000640 10 client.25520 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20a2000640 1 -- 172.21.15.37:0/1766654409 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557bb253e740 con 0x557bb25d6c00 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20a2000640 10 client.25520 handle_command_reply: tid=0 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 client.25520 shutdown 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 msgr2=0x557bb25e2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 0x557bb25e2380 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009b30 tx=0x7f20b0033640 comp rx=0 tx=0).stop 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 shutdown_connections 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bb25d1a90 0x557bb25e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bb25d10a0 0x557bb25e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557bb25d6c00 0x557bb2645910 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 --2- 172.21.15.37:0/1766654409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557bb25cc610 0x557bb25e2380 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.051+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 >> 172.21.15.37:0/1766654409 conn(0x557bb25d04a0 msgr2=0x557bb25cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.052+0000 7f20a2000640 0 client.25520 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.052+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 shutdown_connections 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.052+0000 7f20c1ba6740 1 -- 172.21.15.37:0/1766654409 wait complete. 2024-03-21T09:12:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.052+0000 7f20c1ba6740 20 client.25520 trim_cache size 0 max 16384 2024-03-21T09:12:00.049 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:12:00.049 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:12:00.049 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "1633c1c2-df2d-471c-a2ef-e928faedc85d", 2024-03-21T09:12:00.049 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:12:00.049 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f209c0a3030 msgr2=0x7f209c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f209c0a3030 0x7f209c0a54f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x557bb25d2500 tx=0x7f20a80cfe40 comp rx=0 tx=0).stop 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 msgr2=0x7f20b819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 0x7f20b819df60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e9f0 tx=0x7f20a800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 shutdown_connections 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f209c0a3030 0x7f209c0a54f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b8105250 0x7f20b81a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b81028a0 0x7f20b819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 --2- 172.21.15.37:0/3619769579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b80fff30 0x7f20b819df60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.053+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 >> 172.21.15.37:0/3619769579 conn(0x7f20b80fbb30 msgr2=0x7f20b81008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.054+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 shutdown_connections 2024-03-21T09:12:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.054+0000 7f20bee00640 1 -- 172.21.15.37:0/3619769579 wait complete. 2024-03-21T09:12:00.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:12:00.063 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 -- 172.21.15.37:0/981319277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c101500 msgr2=0x7f1f7c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 --2- 172.21.15.37:0/981319277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c101500 0x7f1f7c101920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 -- 172.21.15.37:0/981319277 shutdown_connections 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 --2- 172.21.15.37:0/981319277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c1030b0 0x7f1f7c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 --2- 172.21.15.37:0/981319277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c1026f0 0x7f1f7c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 --2- 172.21.15.37:0/981319277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c101500 0x7f1f7c101920 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 -- 172.21.15.37:0/981319277 >> 172.21.15.37:0/981319277 conn(0x7f1f7c0fcab0 msgr2=0x7f1f7c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 -- 172.21.15.37:0/981319277 shutdown_connections 2024-03-21T09:12:00.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.255+0000 7f1f81000640 1 -- 172.21.15.37:0/981319277 wait complete. 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.256+0000 7f1f81000640 1 Processor -- start 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.256+0000 7f1f81000640 1 -- start start 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.256+0000 7f1f81000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 0x7f1f7c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.256+0000 7f1f81000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c1026f0 0x7f1f7c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f81000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c1030b0 0x7f1f7c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f81000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c10d1e0 con 0x7f1f7c1030b0 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f81000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c10d060 con 0x7f1f7c1026f0 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f81000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c10d360 con 0x7f1f7c101500 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 0x7f1f7c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 0x7f1f7c199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57382/0 (socket says 172.21.15.37:57382) 2024-03-21T09:12:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 -- 172.21.15.37:0/2018957032 learned_addr learned my addr 172.21.15.37:0/2018957032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c1026f0 0x7f1f7c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7be00640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c1030b0 0x7f1f7c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 -- 172.21.15.37:0/2018957032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c1026f0 msgr2=0x7f1f7c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c1026f0 0x7f1f7c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 -- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c1030b0 msgr2=0x7f1f7c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c1030b0 0x7f1f7c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.257+0000 7f1f7b400640 1 -- 172.21.15.37:0/2018957032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c1a1620 con 0x7f1f7c101500 2024-03-21T09:12:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.258+0000 7f1f7b400640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 0x7f1f7c199a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000c00 tx=0x7f1f6c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.258+0000 7f1f73e00640 1 -- 172.21.15.37:0/2018957032 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c043020 con 0x7f1f7c101500 2024-03-21T09:12:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.258+0000 7f1f73e00640 1 -- 172.21.15.37:0/2018957032 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c0054e0 con 0x7f1f7c101500 2024-03-21T09:12:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.258+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c1a18b0 con 0x7f1f7c101500 2024-03-21T09:12:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.258+0000 7f1f73e00640 1 -- 172.21.15.37:0/2018957032 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c003d80 con 0x7f1f7c101500 2024-03-21T09:12:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.258+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c1a1e50 con 0x7f1f7c101500 2024-03-21T09:12:00.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.261+0000 7f1f73e00640 1 -- 172.21.15.37:0/2018957032 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f1f6c046020 con 0x7f1f7c101500 2024-03-21T09:12:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.262+0000 7f1f8539a740 1 Processor -- start 2024-03-21T09:12:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 -- start start 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f73e00640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f600a3030 0x7f1f600a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f51470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f56540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f80eb6250 con 0x557f80f56bb0 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f80e5ea30 con 0x557f80f51a40 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f8539a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f80eb22f0 con 0x557f80f51050 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f73e00640 1 -- 172.21.15.37:0/2018957032 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1f6c0f1a40 con 0x7f1f7c101500 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.263+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f51470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f51470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57386/0 (socket says 172.21.15.37:57386) 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 -- 172.21.15.37:0/1071443468 learned_addr learned my addr 172.21.15.37:0/1071443468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f56540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7be00640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f600a3030 0x7f1f600a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7be00640 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 -- 172.21.15.37:0/1071443468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 msgr2=0x557f80f56540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f56540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 -- 172.21.15.37:0/1071443468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 msgr2=0x557f80f59040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f59040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 -- 172.21.15.37:0/1071443468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f80e49690 con 0x557f80f51050 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7be00640 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f7b400640 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f51470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c02c170 tx=0x7f1f6c0ba100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c198478ee2a5f1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f70c00640 1 -- 172.21.15.37:0/1071443468 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0c3010 con 0x557f80f51050 2024-03-21T09:12:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f70c00640 1 -- 172.21.15.37:0/1071443468 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c005690 con 0x557f80f51050 2024-03-21T09:12:00.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.264+0000 7f1f70c00640 1 -- 172.21.15.37:0/1071443468 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0c37f0 con 0x557f80f51050 2024-03-21T09:12:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 -- 172.21.15.37:0/1071443468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 msgr2=0x557f80f51470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f7be00640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f600a3030 0x7f1f600a54f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005fd0 tx=0x7f1f74009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f51470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c02c170 tx=0x7f1f6c0ba100 comp rx=0 tx=0).stop 2024-03-21T09:12:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 -- 172.21.15.37:0/1071443468 shutdown_connections 2024-03-21T09:12:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f56540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 --2- 172.21.15.37:0/1071443468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f51470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.265+0000 7f1f8539a740 1 -- 172.21.15.37:0/1071443468 >> 172.21.15.37:0/1071443468 conn(0x557f80f19200 msgr2=0x557f80f4c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.266+0000 7f1f8539a740 1 -- 172.21.15.37:0/1071443468 shutdown_connections 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.266+0000 7f1f8539a740 1 -- 172.21.15.37:0/1071443468 wait complete. 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.266+0000 7f1f8539a740 1 -- start start 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.266+0000 7f1f8539a740 1 Processor -- start 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f5ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f80f59920 con 0x557f80f56bb0 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f80e71e80 con 0x557f80f51a40 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f80f59aa0 con 0x557f80f51050 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f8539a740 10 client.0 _ll_register_callbacks cb 0x557f80e84130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f5ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.267+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f5b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50664/0 (socket says 172.21.15.37:50664) 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 -- 172.21.15.37:0/587184876 learned_addr learned my addr 172.21.15.37:0/587184876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7be00640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 -- 172.21.15.37:0/587184876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 msgr2=0x557f80f5ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f5ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 -- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 msgr2=0x557f80f622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 -- 172.21.15.37:0/587184876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f80c947d0 con 0x557f80f51a40 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7be00640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f5b270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f68010840 tx=0x7f1f68010d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870c16a1b9158540 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f65600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f65600640 1 -- 172.21.15.37:0/587184876 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6801e060 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.268+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f80ec3790 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f65600640 1 -- 172.21.15.37:0/587184876 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1f6800fb90 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f65600640 1 -- 172.21.15.37:0/587184876 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f68025050 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 10 client.25532 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557f80bca3f0 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f65600640 1 -- 172.21.15.37:0/587184876 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557f80ec3790 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f65600640 1 -- 172.21.15.37:0/587184876 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1f6800e420 con 0x557f80f51a40 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 10 client.25532 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 10 client.25532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 20 client.25532 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80f66af0 0x557f80f68f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 4 client.25532 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.269+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f80ebdae0 con 0x557f80f66af0 2024-03-21T09:12:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.270+0000 7f1f7b400640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80f66af0 0x557f80f68f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.270+0000 7f1f7b400640 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80f66af0 0x557f80f68f30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c433e6d149b1f2 server_cookie=c885a6a25aea6f00 in_seq=0 out_seq=0 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.270+0000 7f1f65600640 10 client.25532 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f65600640 1 -- 172.21.15.37:0/587184876 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557f80ebdae0 con 0x557f80f66af0 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f65600640 10 client.25532 handle_command_reply: tid=0 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 client.25532 shutdown 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 msgr2=0x557f80f5b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f5b270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f68010840 tx=0x7f1f68010d00 comp rx=0 tx=0).stop 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 shutdown_connections 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80f66af0 0x557f80f68f30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51a40 0x557f80f5b270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 --2- 172.21.15.37:0/587184876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51050 0x557f80f5ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 >> 172.21.15.37:0/587184876 conn(0x557f80f19200 msgr2=0x557f80f4bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.271+0000 7f1f65600640 0 client.25532 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.272+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 shutdown_connections 2024-03-21T09:12:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.272+0000 7f1f8539a740 1 -- 172.21.15.37:0/587184876 wait complete. 2024-03-21T09:12:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.272+0000 7f1f8539a740 20 client.25532 trim_cache size 0 max 16384 2024-03-21T09:12:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.288+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1628005850 con 0x7f164c102710 2024-03-21T09:12:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 Processor -- start 2024-03-21T09:12:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 -- start start 2024-03-21T09:12:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f51050 0x557f80f4ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f4bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f56bb0 0x557f80f62270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1ae750 con 0x557f80f51050 2024-03-21T09:12:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1ac250 con 0x557f80f56bb0 2024-03-21T09:12:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.290+0000 7f1f8539a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1aa2f0 con 0x557f80f51a40 2024-03-21T09:12:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f4bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f51050 0x557f80f4ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f4bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57404/0 (socket says 172.21.15.37:57404) 2024-03-21T09:12:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 -- 172.21.15.37:0/86165046 learned_addr learned my addr 172.21.15.37:0/86165046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7be00640 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f56bb0 0x557f80f62270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 -- 172.21.15.37:0/86165046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f56bb0 msgr2=0x557f80f62270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f56bb0 0x557f80f62270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 -- 172.21.15.37:0/86165046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f51050 msgr2=0x557f80f4ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f51050 0x557f80f4ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 -- 172.21.15.37:0/86165046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f80b8a0f0 con 0x557f80f51a40 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7b400640 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f51050 0x557f80f4ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f4bf80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800e840 tx=0x7f1f6800efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2a13398003d1f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f67e00640 1 -- 172.21.15.37:0/86165046 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6802f070 con 0x557f80f51a40 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.291+0000 7f1f67e00640 1 -- 172.21.15.37:0/86165046 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f6800a340 con 0x557f80f51a40 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f67e00640 1 -- 172.21.15.37:0/86165046 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800fdf0 con 0x557f80f51a40 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 -- 172.21.15.37:0/86165046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 msgr2=0x557f80f4bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f4bf80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800e840 tx=0x7f1f6800efa0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 -- 172.21.15.37:0/86165046 shutdown_connections 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f56bb0 0x557f80f62270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f4bf80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 --2- 172.21.15.37:0/86165046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f51050 0x557f80f4ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 -- 172.21.15.37:0/86165046 >> 172.21.15.37:0/86165046 conn(0x557f80f19200 msgr2=0x557f80f4b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 -- 172.21.15.37:0/86165046 shutdown_connections 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.292+0000 7f1f8539a740 1 -- 172.21.15.37:0/86165046 wait complete. 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 -- start start 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 Processor -- start 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 0x557f80f5fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1aa8f0 con 0x557f80f56bb0 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1abbf0 con 0x557f80f51050 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1aaf30 con 0x557f80f51a40 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f8539a740 10 client.0 _ll_register_callbacks cb 0x7f1f68010450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 0x557f80f5fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.293+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 0x557f80f5fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50686/0 (socket says 172.21.15.37:50686) 2024-03-21T09:12:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57416/0 (socket says 172.21.15.37:57416) 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7b400640 1 -- 172.21.15.37:0/3268278284 learned_addr learned my addr 172.21.15.37:0/3268278284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 -- 172.21.15.37:0/3268278284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 msgr2=0x557f80f5fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 0x557f80f5fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 -- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 msgr2=0x557f80f612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 -- 172.21.15.37:0/3268278284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f80f618c0 con 0x557f80f51a40 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7b400640 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 0x557f80f5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f7aa00640 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f602c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1f68010840 tx=0x7f1f6800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6525f87dab6041d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f65600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f65600640 1 -- 172.21.15.37:0/3268278284 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6802f040 con 0x557f80f51a40 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f6800a340 con 0x557f80f51a40 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.294+0000 7f1f65600640 1 -- 172.21.15.37:0/3268278284 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1f6800fb90 con 0x557f80f51a40 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f65600640 1 -- 172.21.15.37:0/3268278284 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f6800fd90 con 0x557f80f51a40 2024-03-21T09:12:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 10 client.25570 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557f80fc26f0 con 0x557f80f51a40 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f65600640 1 -- 172.21.15.37:0/3268278284 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f6800a340 con 0x557f80f51a40 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f65600640 1 -- 172.21.15.37:0/3268278284 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1f680193e0 con 0x557f80f51a40 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 10 client.25570 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 20 client.25570 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80fc2fe0 0x557f80fc5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 4 client.25570 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f80e3b3a0 con 0x557f80fc2fe0 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.295+0000 7f1f7b400640 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80fc2fe0 0x557f80fc5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f7b400640 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80fc2fe0 0x557f80fc5400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ac5dce19d4e2f2 server_cookie=de27726266f2ec56 in_seq=0 out_seq=0 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f65600640 10 client.25570 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f65600640 1 -- 172.21.15.37:0/3268278284 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557f80e3b3a0 con 0x557f80fc2fe0 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f65600640 10 client.25570 handle_command_reply: tid=0 2024-03-21T09:12:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 client.25570 shutdown 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 msgr2=0x557f80f602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f602c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1f68010840 tx=0x7f1f6800ec90 comp rx=0 tx=0).stop 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 shutdown_connections 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f80f56bb0 0x557f80f612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f80f51a40 0x557f80f602c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557f80fc2fe0 0x557f80fc5400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 --2- 172.21.15.37:0/3268278284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f80f51050 0x557f80f5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f65600640 0 client.25570 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.296+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 >> 172.21.15.37:0/3268278284 conn(0x557f80f19200 msgr2=0x557f80f4bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.297+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 shutdown_connections 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.297+0000 7f1f8539a740 1 -- 172.21.15.37:0/3268278284 wait complete. 2024-03-21T09:12:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.297+0000 7f1f8539a740 20 client.25570 trim_cache size 0 max 16384 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: "1633c1c2-df2d-471c-a2ef-e928faedc85d": { 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "1633c1c2-df2d-471c-a2ef-e928faedc85d", 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:12:00.301 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:12:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f600a3030 msgr2=0x7f1f600a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f600a3030 0x7f1f600a54f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005fd0 tx=0x7f1f74009590 comp rx=0 tx=0).stop 2024-03-21T09:12:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 msgr2=0x7f1f7c199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 0x7f1f7c199a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000c00 tx=0x7f1f6c005dd0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 shutdown_connections 2024-03-21T09:12:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f1f600a3030 0x7f1f600a54f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c1030b0 0x7f1f7c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f7c1026f0 0x7f1f7c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 --2- 172.21.15.37:0/2018957032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f7c101500 0x7f1f7c199a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 >> 172.21.15.37:0/2018957032 conn(0x7f1f7c0fcab0 msgr2=0x7f1f7c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 shutdown_connections 2024-03-21T09:12:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:00.298+0000 7f1f81000640 1 -- 172.21.15.37:0/2018957032 wait complete. 2024-03-21T09:12:00.305 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3328837696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:12:00.305 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3106570010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:12:00.308 INFO:tasks.cephfs.filesystem:scrub status for tag:1633c1c2-df2d-471c-a2ef-e928faedc85d - {'path': '/', 'tag': '1633c1c2-df2d-471c-a2ef-e928faedc85d', 'options': 'recursive,force'} 2024-03-21T09:12:00.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3328837696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:12:00.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3106570010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:12:00.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3328837696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:12:00.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3106570010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:12:01.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.069+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 20) v1 ==== 137922+0+0 (secure 0 0 0) 0x7f16380b9db0 con 0x7f164c102710 2024-03-21T09:12:01.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.071+0000 7f1649600640 1 -- 172.21.15.37:0/2973151206 <== mon.1 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1638098f90 con 0x7f164c102710 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16340a3010 msgr2=0x7f16340a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16340a3010 0x7f16340a54d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f164c102570 tx=0x7f1640009110 comp rx=0 tx=0).stop 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 msgr2=0x7f164c073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c073a90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f163802c8f0 tx=0x7f1638005d50 comp rx=0 tx=0).stop 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 shutdown_connections 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f16340a3010 0x7f16340a54d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164c1030d0 0x7f164c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c102710 0x7f164c073a90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 --2- 172.21.15.37:0/2973151206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c101520 0x7f164c075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.076+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 >> 172.21.15.37:0/2973151206 conn(0x7f164c0fcab0 msgr2=0x7f164c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:01.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.077+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 shutdown_connections 2024-03-21T09:12:01.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.077+0000 7f163ea00640 1 -- 172.21.15.37:0/2973151206 wait complete. 2024-03-21T09:12:01.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-21T09:12:01.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 -- 172.21.15.37:0/931761120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 msgr2=0x7f60a010ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 --2- 172.21.15.37:0/931761120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a010ba10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6098009be0 tx=0x7f609802bef0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 -- 172.21.15.37:0/931761120 shutdown_connections 2024-03-21T09:12:01.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 --2- 172.21.15.37:0/931761120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a010ba10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 --2- 172.21.15.37:0/931761120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 --2- 172.21.15.37:0/931761120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00756a0 0x7f60a0073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 -- 172.21.15.37:0/931761120 >> 172.21.15.37:0/931761120 conn(0x7f60a00fcc60 msgr2=0x7f60a00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:01.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 -- 172.21.15.37:0/931761120 shutdown_connections 2024-03-21T09:12:01.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.294+0000 7f60a6a00640 1 -- 172.21.15.37:0/931761120 wait complete. 2024-03-21T09:12:01.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.295+0000 7f60a6a00640 1 Processor -- start 2024-03-21T09:12:01.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 -- start start 2024-03-21T09:12:01.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00756a0 0x7f60a019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a01a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a010f530 con 0x7f60a00740c0 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a010f3b0 con 0x7f60a01095a0 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a6a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a010f6b0 con 0x7f60a00756a0 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f609fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f609fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37782/0 (socket says 172.21.15.37:37782) 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f609fe00640 1 -- 172.21.15.37:0/1361215311 learned_addr learned my addr 172.21.15.37:0/1361215311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f60a4a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a01a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.296+0000 7f609f400640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00756a0 0x7f60a019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f60a4a00640 1 -- 172.21.15.37:0/1361215311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00756a0 msgr2=0x7f60a019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f60a4a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00756a0 0x7f60a019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f60a4a00640 1 -- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 msgr2=0x7f60a0199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f60a4a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f60a4a00640 1 -- 172.21.15.37:0/1361215311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a01a1760 con 0x7f60a01095a0 2024-03-21T09:12:01.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f609fe00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:01.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.297+0000 7f60a4a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a01a10f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f609802c430 tx=0x7f609802cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:01.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.298+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098043020 con 0x7f60a01095a0 2024-03-21T09:12:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.298+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6098003e50 con 0x7f60a01095a0 2024-03-21T09:12:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.298+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098034690 con 0x7f60a01095a0 2024-03-21T09:12:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.298+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a01a19f0 con 0x7f60a01095a0 2024-03-21T09:12:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.298+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a01a1ef0 con 0x7f60a01095a0 2024-03-21T09:12:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.299+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a0108320 con 0x7f60a01095a0 2024-03-21T09:12:01.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.301+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) v1 ==== 137922+0+0 (secure 0 0 0) 0x7f6098046020 con 0x7f60a01095a0 2024-03-21T09:12:01.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.303+0000 7f609cc00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 0x7f60840a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.303+0000 7f609fe00640 1 -- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 msgr2=0x7f60840a5530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:01.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.303+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60980f1d80 con 0x7f60a01095a0 2024-03-21T09:12:01.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.303+0000 7f609fe00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 0x7f60840a5530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:12:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.307+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f609804d020 con 0x7f60a01095a0 2024-03-21T09:12:01.352 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: ignoring --setuser ceph since I am not root 2024-03-21T09:12:01.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: ignoring --setgroup ceph since I am not root 2024-03-21T09:12:01.353 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mgr[38483]: -- 172.21.15.71:0/2100454646 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632405a45a0 con 0x563240579000 2024-03-21T09:12:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:12:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:12:01.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:01.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:12:01.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:01.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2973151206' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T09:12:01.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T09:12:01.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: ignoring --setuser ceph since I am not root 2024-03-21T09:12:01.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: ignoring --setgroup ceph since I am not root 2024-03-21T09:12:01.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:12:01.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:12:01.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:01.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:12:01.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:01.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2973151206' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T09:12:01.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:01 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T09:12:01.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-21T09:12:01.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:12:01.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:01.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:12:01.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:01.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2973151206' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T09:12:01.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T09:12:01.443 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: ignoring --setuser ceph since I am not root 2024-03-21T09:12:01.443 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: ignoring --setgroup ceph since I am not root 2024-03-21T09:12:01.443 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-mgr[37552]: -- 172.21.15.53:0/2059679744 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591c5aa05a0 con 0x5591c5a74c00 2024-03-21T09:12:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.504+0000 7f609fe00640 1 -- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 msgr2=0x7f60840a5530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.504+0000 7f609fe00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 0x7f60840a5530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:12:01.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.575+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f60a002d070 con 0x7f60a01095a0 2024-03-21T09:12:01.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.581+0000 7f609cc00640 1 -- 172.21.15.37:0/1361215311 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f60980be770 con 0x7f60a01095a0 2024-03-21T09:12:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 msgr2=0x7f60840a5530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 0x7f60840a5530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 msgr2=0x7f60a01a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a01a10f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f609802c430 tx=0x7f609802cb90 comp rx=0 tx=0).stop 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 shutdown_connections 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f60840a3070 0x7f60840a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a01095a0 0x7f60a01a10f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00756a0 0x7f60a019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 --2- 172.21.15.37:0/1361215311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00740c0 0x7f60a0199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 >> 172.21.15.37:0/1361215311 conn(0x7f60a00fcc60 msgr2=0x7f60a01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 shutdown_connections 2024-03-21T09:12:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.584+0000 7f60a6a00640 1 -- 172.21.15.37:0/1361215311 wait complete. 2024-03-21T09:12:01.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-21T09:12:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 -- 172.21.15.37:0/4263927876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c1071f0 msgr2=0x7fb90c109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 --2- 172.21.15.37:0/4263927876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c1071f0 0x7fb90c109640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb904009be0 tx=0x7fb90402bef0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 -- 172.21.15.37:0/4263927876 shutdown_connections 2024-03-21T09:12:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 --2- 172.21.15.37:0/4263927876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c1071f0 0x7fb90c109640 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 --2- 172.21.15.37:0/4263927876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1048c0 0x7fb90c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 --2- 172.21.15.37:0/4263927876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb90c101f90 0x7fb90c104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 -- 172.21.15.37:0/4263927876 >> 172.21.15.37:0/4263927876 conn(0x7fb90c0fb960 msgr2=0x7fb90c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:01.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 -- 172.21.15.37:0/4263927876 shutdown_connections 2024-03-21T09:12:01.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.817+0000 7fb912600640 1 -- 172.21.15.37:0/4263927876 wait complete. 2024-03-21T09:12:01.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.818+0000 7fb912600640 1 Processor -- start 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.818+0000 7fb912600640 1 -- start start 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb912600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 0x7fb90c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb912600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb90c1048c0 0x7fb90c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb912600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1071f0 0x7fb90c1a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb912600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c10d1a0 con 0x7fb90c1048c0 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb912600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c10d020 con 0x7fb90c101f90 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb912600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c10d320 con 0x7fb90c1071f0 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 0x7fb90c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 0x7fb90c1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50730/0 (socket says 172.21.15.37:50730) 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 -- 172.21.15.37:0/1067262104 learned_addr learned my addr 172.21.15.37:0/1067262104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:01.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb90be00640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb90c1048c0 0x7fb90c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911c00640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1071f0 0x7fb90c1a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 -- 172.21.15.37:0/1067262104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1071f0 msgr2=0x7fb90c1a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1071f0 0x7fb90c1a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 -- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb90c1048c0 msgr2=0x7fb90c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.819+0000 7fb911200640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb90c1048c0 0x7fb90c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb911200640 1 -- 172.21.15.37:0/1067262104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c1a15d0 con 0x7fb90c101f90 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb911c00640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1071f0 0x7fb90c1a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb911200640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 0x7fb90c1999b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb90000e9f0 tx=0x7fb90000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000cd80 con 0x7fb90c101f90 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb900010910 con 0x7fb90c101f90 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900018990 con 0x7fb90c101f90 2024-03-21T09:12:01.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90c1a18c0 con 0x7fb90c101f90 2024-03-21T09:12:01.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.820+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c1a1dc0 con 0x7fb90c101f90 2024-03-21T09:12:01.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.822+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90c108ce0 con 0x7fb90c101f90 2024-03-21T09:12:01.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.823+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 20) v1 ==== 137922+0+0 (secure 0 0 0) 0x7fb900010ac0 con 0x7fb90c101f90 2024-03-21T09:12:01.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.824+0000 7fb909600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 0x7fb8f40a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:01.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.825+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9000cfcd0 con 0x7fb90c101f90 2024-03-21T09:12:01.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.825+0000 7fb90be00640 1 -- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 msgr2=0x7fb8f40a54a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:01.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.825+0000 7fb90be00640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 0x7fb8f40a54a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:12:01.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:01.829+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb90009bf60 con 0x7fb90c101f90 2024-03-21T09:12:01.941 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:01.637+0000 7fde1f436200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:12:01.941 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:01.734+0000 7fde1f436200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:12:02.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.026+0000 7fb90be00640 1 -- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 msgr2=0x7fb8f40a54a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.026+0000 7fb90be00640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 0x7fb8f40a54a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:12:02.047 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:01.638+0000 7ff28928d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:12:02.047 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:01.732+0000 7ff28928d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:12:02.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.083+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb90c062340 con 0x7fb90c101f90 2024-03-21T09:12:02.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.091+0000 7fb909600640 1 -- 172.21.15.37:0/1067262104 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb9000a0000 con 0x7fb90c101f90 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 msgr2=0x7fb8f40a54a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 0x7fb8f40a54a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 msgr2=0x7fb90c1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 0x7fb90c1999b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb90000e9f0 tx=0x7fb90000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 shutdown_connections 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7fb8f40a2fe0 0x7fb8f40a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb90c1071f0 0x7fb90c1a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb90c1048c0 0x7fb90c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 --2- 172.21.15.37:0/1067262104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb90c101f90 0x7fb90c1999b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 >> 172.21.15.37:0/1067262104 conn(0x7fb90c0fb960 msgr2=0x7fb90c101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 shutdown_connections 2024-03-21T09:12:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.094+0000 7fb912600640 1 -- 172.21.15.37:0/1067262104 wait complete. 2024-03-21T09:12:02.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.311+0000 7f6e62200640 1 -- 172.21.15.37:0/1125033435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 msgr2=0x7f6e5c102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.311+0000 7f6e62200640 1 --2- 172.21.15.37:0/1125033435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c102d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802bc30 comp rx=0 tx=0).stop 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 -- 172.21.15.37:0/1125033435 shutdown_connections 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 --2- 172.21.15.37:0/1125033435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1032c0 0x7f6e5c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 --2- 172.21.15.37:0/1125033435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c102d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 --2- 172.21.15.37:0/1125033435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 0x7f6e5c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 -- 172.21.15.37:0/1125033435 >> 172.21.15.37:0/1125033435 conn(0x7f6e5c0fcc80 msgr2=0x7f6e5c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 -- 172.21.15.37:0/1125033435 shutdown_connections 2024-03-21T09:12:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.312+0000 7f6e62200640 1 -- 172.21.15.37:0/1125033435 wait complete. 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.313+0000 7f6e62200640 1 Processor -- start 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.313+0000 7f6e62200640 1 -- start start 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.313+0000 7f6e62200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 0x7f6e5c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e62200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e62200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1032c0 0x7f6e5c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e62200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c10d370 con 0x7f6e5c1032c0 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e62200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c10d1f0 con 0x7f6e5c102900 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e62200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c10d4f0 con 0x7f6e5c101710 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 0x7f6e5c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50746/0 (socket says 172.21.15.37:50746) 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 0x7f6e5c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57492/0 (socket says 172.21.15.37:57492) 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 learned_addr learned my addr 172.21.15.37:0/1191607464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5be00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1032c0 0x7f6e5c1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 msgr2=0x7f6e5c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1032c0 msgr2=0x7f6e5c1a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1032c0 0x7f6e5c1a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5b400640 1 -- 172.21.15.37:0/1191607464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c1a17d0 con 0x7f6e5c101710 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.314+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e5c102900 0x7f6e5c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.315+0000 7f6e5b400640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 0x7f6e5c199bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00e9c0 tx=0x7f6e4c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.315+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00ccc0 con 0x7f6e5c101710 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.315+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c1a1ac0 con 0x7f6e5c101710 2024-03-21T09:12:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.315+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c0035f0 con 0x7f6e5c101710 2024-03-21T09:12:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.315+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00ce60 con 0x7f6e5c101710 2024-03-21T09:12:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.315+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c1a1fe0 con 0x7f6e5c101710 2024-03-21T09:12:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.316+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c1a2270 con 0x7f6e5c101710 2024-03-21T09:12:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.318+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) v1 ==== 137922+0+0 (secure 0 0 0) 0x7f6e4c0037a0 con 0x7f6e5c101710 2024-03-21T09:12:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.319+0000 7f6e53e00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.319+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.319+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6e4c0cfdf0 con 0x7f6e5c101710 2024-03-21T09:12:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.319+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:12:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.324+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6e4c09c6d0 con 0x7f6e5c101710 2024-03-21T09:12:02.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T09:12:02.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-mon[36597]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-21T09:12:02.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' 2024-03-21T09:12:02.398 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T09:12:02.399 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-mon[36169]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-21T09:12:02.399 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' 2024-03-21T09:12:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:02 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T09:12:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:02 smithi037 ceph-mon[31830]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-21T09:12:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:02 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' 2024-03-21T09:12:02.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:02 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:02.068+0000 7f035d770200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T09:12:02.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:02 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:02.236+0000 7f035d770200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T09:12:02.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.520+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:02.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.520+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:12:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.575+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 --> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e5c074ff0 con 0x7f6e400a2ff0 2024-03-21T09:12:02.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:02.405+0000 7fde1f436200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:12:02.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:02.501+0000 7fde1f436200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:12:02.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:02.648+0000 7fde1f436200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:12:02.713 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:02.390+0000 7ff28928d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:12:02.713 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:02.485+0000 7ff28928d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:12:02.713 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:02.637+0000 7ff28928d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:12:02.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.921+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:02.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:02.921+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T09:12:03.083 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:02.753+0000 7ff28928d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:12:03.083 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:02.840+0000 7ff28928d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:12:03.083 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:02.747+0000 7fde1f436200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:12:03.084 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:02.835+0000 7fde1f436200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:12:03.351 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:03 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' 2024-03-21T09:12:03.351 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:03.124+0000 7fde1f436200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:12:03.351 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:03.255+0000 7fde1f436200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:12:03.351 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:03.357+0000 7fde1f436200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:12:03.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:03 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' 2024-03-21T09:12:03.352 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:03.116+0000 7ff28928d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:12:03.352 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:03.240+0000 7ff28928d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:12:03.352 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:03.328+0000 7ff28928d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:12:03.404 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:03 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' 2024-03-21T09:12:03.404 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:03.280+0000 7f035d770200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T09:12:03.632 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:03.456+0000 7fde1f436200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:12:03.672 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:03.410+0000 7f035d770200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T09:12:03.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:03.606+0000 7f035d770200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T09:12:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:03.722+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:03.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:03.722+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T09:12:03.853 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:03.424+0000 7ff28928d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:12:03.853 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:03.602+0000 7ff28928d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:12:03.941 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:03.638+0000 7fde1f436200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:12:04.172 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:03.741+0000 7f035d770200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T09:12:04.173 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:03 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:03.857+0000 7f035d770200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T09:12:04.465 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:04.206+0000 7f035d770200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T09:12:04.465 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:04.354+0000 7f035d770200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T09:12:04.817 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:04.470+0000 7f035d770200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T09:12:04.817 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:04.599+0000 7f035d770200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T09:12:04.943 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:04.667+0000 7fde1f436200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:12:04.943 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:04.802+0000 7fde1f436200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:12:05.083 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:04.622+0000 7ff28928d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:12:05.084 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:04.753+0000 7ff28928d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:12:05.084 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:05.089+0000 7ff28928d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:12:05.172 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:04.822+0000 7f035d770200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T09:12:05.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:05.324+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:05.324+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T09:12:05.352 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:05.179+0000 7ff28928d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:12:05.438 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:05.149+0000 7fde1f436200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:12:05.438 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:05.241+0000 7fde1f436200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:12:05.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:05.444+0000 7fde1f436200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:12:05.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:05.636+0000 7fde1f436200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:12:05.852 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:05.376+0000 7ff28928d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:12:05.852 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:05.561+0000 7ff28928d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:12:06.142 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:05.859+0000 7ff28928d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:12:06.191 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:05.933+0000 7fde1f436200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:12:06.602 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:06.148+0000 7ff28928d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:12:06.603 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:06.386+0000 7ff28928d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:12:06.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:06 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:06.228+0000 7f035d770200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T09:12:06.673 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:06 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:06.408+0000 7f035d770200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T09:12:06.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:06.201+0000 7fde1f436200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:12:06.691 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:06.447+0000 7fde1f436200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:12:07.173 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:06 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:06.853+0000 7f035d770200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T09:12:07.173 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:06 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:06.981+0000 7f035d770200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T09:12:07.509 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:07.256+0000 7f035d770200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:12:07.592 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:07.293+0000 7fde1f436200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:12:07.602 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:07.199+0000 7ff28928d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:12:07.905 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:07.514+0000 7f035d770200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T09:12:08.081 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:07.773+0000 7ff28928d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:12:08.081 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:07.871+0000 7ff28928d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:12:08.082 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z[38479]: 2024-03-21T09:12:08.087+0000 7ff28928d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:12:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:08.171+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 21) v1 ==== 137922+0+0 (secure 0 0 0) 0x7f6e4c098640 con 0x7f6e5c101710 2024-03-21T09:12:08.166 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:07.887+0000 7fde1f436200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:12:08.166 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:07.988+0000 7fde1f436200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:12:08.166 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:07.910+0000 7f035d770200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T09:12:08.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-mon[36597]: Standby manager daemon z restarted 2024-03-21T09:12:08.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-mon[36597]: Standby manager daemon z started 2024-03-21T09:12:08.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T09:12:08.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:12:08.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T09:12:08.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:08 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:12:08.422 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:08.265+0000 7f035d770200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T09:12:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-mon[31830]: Standby manager daemon z restarted 2024-03-21T09:12:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-mon[31830]: Standby manager daemon z started 2024-03-21T09:12:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T09:12:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:12:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T09:12:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y[37548]: 2024-03-21T09:12:08.232+0000 7fde1f436200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-mon[36169]: Standby manager daemon z restarted 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-mon[36169]: Standby manager daemon z started 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T09:12:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:08 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.71:0/76608945' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:12:08.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:08.527+0000 7f6e5aa00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1300592626 2024-03-21T09:12:08.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:08.527+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-21T09:12:08.922 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:08.587+0000 7f035d770200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T09:12:09.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:09.192+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 22) v1 ==== 137922+0+0 (secure 0 0 0) 0x7f6e4c0cd600 con 0x7f6e5c101710 2024-03-21T09:12:09.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-21T09:12:09.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: Standby manager daemon y restarted 2024-03-21T09:12:09.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: Standby manager daemon y started 2024-03-21T09:12:09.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T09:12:09.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:12:09.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T09:12:09.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-mon[31830]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: Standby manager daemon y restarted 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: Standby manager daemon y started 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T09:12:09.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:09 smithi053 ceph-mon[36169]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:12:09.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-21T09:12:09.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: Standby manager daemon y restarted 2024-03-21T09:12:09.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: Standby manager daemon y started 2024-03-21T09:12:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T09:12:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:12:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T09:12:09.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:09 smithi071 ceph-mon[36597]: from='mgr.? 172.21.15.53:0/2303160991' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:12:10.173 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:09 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:09.792+0000 7f035d770200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T09:12:10.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:10 smithi053 ceph-mon[36169]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-21T09:12:10.536 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:10 smithi037 ceph-mon[31830]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-21T09:12:10.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:10 smithi071 ceph-mon[36597]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-21T09:12:10.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:10 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:10.542+0000 7f035d770200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T09:12:10.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:10 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:10.679+0000 7f035d770200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T09:12:11.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:11.016+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 9 ==== mgrmap(e 23) v1 ==== 137539+0+0 (secure 0 0 0) 0x7f6e4c00ccc0 con 0x7f6e5c101710 2024-03-21T09:12:11.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:11.017+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 msgr2=0x7f6e400a54b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:12:11.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:11.018+0000 7f6e53e00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1300592626,v1:172.21.15.37:6801/1300592626] conn(0x7f6e400a2ff0 0x7f6e400a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: Active manager daemon x restarted 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: Activating manager daemon x 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: mgrmap e23: x(active, starting, since 0.0301068s), standbys: y, z 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:12:11.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:12:11.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: Manager daemon x is now available 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:12:11.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:12:11.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:12:11.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:12:11.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:12:11.251 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:10 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:10.980+0000 7f035d770200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T09:12:11.433 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: Active manager daemon x restarted 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: Activating manager daemon x 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: mgrmap e23: x(active, starting, since 0.0301068s), standbys: y, z 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:12:11.434 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:12:11.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:12:11.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: Manager daemon x is now available 2024-03-21T09:12:11.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:12:11.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:12:11.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:12:11.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:12:11.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:11 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:12:11.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: Active manager daemon x restarted 2024-03-21T09:12:11.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: Activating manager daemon x 2024-03-21T09:12:11.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T09:12:11.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: mgrmap e23: x(active, starting, since 0.0301068s), standbys: y, z 2024-03-21T09:12:11.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:12:11.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T09:12:11.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: Manager daemon x is now available 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:12:11.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:11 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T09:12:11.500 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.343+0000 7f02b2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.343+0000 7f02b2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.343+0000 7f02b2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.343+0000 7f02b2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.343+0000 7f02b2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.344+0000 7f02ae200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.344+0000 7f02ae200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.344+0000 7f02ae200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.344+0000 7f02ae200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:11.501 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:12:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:12:11.344+0000 7f02ae200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:12:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.018+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mon.1 v2:172.21.15.71:3300/0 10 ==== mgrmap(e 24) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f6e4c098f10 con 0x7f6e5c101710 2024-03-21T09:12:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.018+0000 7f6e53e00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e40068960 0x7f6e40068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.018+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 --> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e5c074ff0 con 0x7f6e40068960 2024-03-21T09:12:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.022+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e40068960 0x7f6e40068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.030+0000 7f6e5aa00640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e40068960 0x7f6e40068d40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6e48000c00 tx=0x7f6e4802c880 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:12.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.113+0000 7f6e53e00640 1 -- 172.21.15.37:0/1191607464 <== mgr.25553 v2:172.21.15.37:6800/1212038665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6e5c074ff0 con 0x7f6e40068960 2024-03-21T09:12:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-03-21T09:12:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.115+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e40068960 msgr2=0x7f6e40068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.115+0000 7f6e62200640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e40068960 0x7f6e40068d40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6e48000c00 tx=0x7f6e4802c880 comp rx=0 tx=0).stop 2024-03-21T09:12:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.115+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 msgr2=0x7f6e5c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.115+0000 7f6e62200640 1 --2- 172.21.15.37:0/1191607464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e5c101710 0x7f6e5c199bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00e9c0 tx=0x7f6e4c00ee80 comp rx=0 tx=0).stop 2024-03-21T09:12:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.115+0000 7f6e5be00640 1 -- 172.21.15.37:0/1191607464 reap_dead start 2024-03-21T09:12:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.116+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 shutdown_connections 2024-03-21T09:12:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.116+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 >> 172.21.15.37:0/1191607464 conn(0x7f6e5c0fcc80 msgr2=0x7f6e5c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.116+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 shutdown_connections 2024-03-21T09:12:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.116+0000 7f6e62200640 1 -- 172.21.15.37:0/1191607464 wait complete. 2024-03-21T09:12:12.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-21T09:12:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.319+0000 7f574ce00640 1 -- 172.21.15.37:0/2708779253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481028e0 msgr2=0x7f5748102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.319+0000 7f574ce00640 1 --2- 172.21.15.37:0/2708779253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481028e0 0x7f5748102d60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5734009900 tx=0x7f573402bc30 comp rx=0 tx=0).stop 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 -- 172.21.15.37:0/2708779253 shutdown_connections 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 --2- 172.21.15.37:0/2708779253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57481032a0 0x7f5748109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 --2- 172.21.15.37:0/2708779253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481028e0 0x7f5748102d60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 --2- 172.21.15.37:0/2708779253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57481016f0 0x7f5748101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 -- 172.21.15.37:0/2708779253 >> 172.21.15.37:0/2708779253 conn(0x7f57480fcc60 msgr2=0x7f57480ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 -- 172.21.15.37:0/2708779253 shutdown_connections 2024-03-21T09:12:12.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.320+0000 7f574ce00640 1 -- 172.21.15.37:0/2708779253 wait complete. 2024-03-21T09:12:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 Processor -- start 2024-03-21T09:12:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 -- start start 2024-03-21T09:12:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 0x7f5748199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57481028e0 0x7f574819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57481032a0 0x7f57481a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574810d370 con 0x7f57481032a0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574810d1f0 con 0x7f57481016f0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f574ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574810d4f0 con 0x7f57481028e0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f5745e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 0x7f5748199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.321+0000 7f5745e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 0x7f5748199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46934/0 (socket says 172.21.15.37:46934) 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 -- 172.21.15.37:0/3617767985 learned_addr learned my addr 172.21.15.37:0/3617767985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5746800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57481032a0 0x7f57481a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57481028e0 0x7f574819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 -- 172.21.15.37:0/3617767985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57481028e0 msgr2=0x7f574819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57481028e0 0x7f574819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 -- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57481032a0 msgr2=0x7f57481a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57481032a0 0x7f57481a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 -- 172.21.15.37:0/3617767985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57481a1860 con 0x7f57481016f0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.322+0000 7f5745e00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 0x7f5748199c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f573800e9c0 tx=0x7f573800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.323+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800cd80 con 0x7f57481016f0 2024-03-21T09:12:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.323+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57380035f0 con 0x7f57481016f0 2024-03-21T09:12:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.323+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738010690 con 0x7f57481016f0 2024-03-21T09:12:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.323+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57481a1b50 con 0x7f57481016f0 2024-03-21T09:12:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.323+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57481a2050 con 0x7f57481016f0 2024-03-21T09:12:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.324+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57481a2220 con 0x7f57481016f0 2024-03-21T09:12:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.327+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f57380037a0 con 0x7f57481016f0 2024-03-21T09:12:12.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.328+0000 7f573ea00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f572c0a2d80 0x7f572c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.329+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f57380cf870 con 0x7f57481016f0 2024-03-21T09:12:12.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.329+0000 7f5745400640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f572c0a2d80 0x7f572c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.330+0000 7f5745400640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f572c0a2d80 0x7f572c0a5240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f573402c7b0 tx=0x7f5734005fd0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:12.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.337+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f573809c490 con 0x7f57481016f0 2024-03-21T09:12:12.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:12 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:12:12.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:12 smithi037 ceph-mon[31830]: mgrmap e24: x(active, since 1.02779s), standbys: y, z 2024-03-21T09:12:12.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:12 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:12:12.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:12 smithi053 ceph-mon[36169]: mgrmap e24: x(active, since 1.02779s), standbys: y, z 2024-03-21T09:12:12.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:12 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:12:12.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:12 smithi071 ceph-mon[36597]: mgrmap e24: x(active, since 1.02779s), standbys: y, z 2024-03-21T09:12:12.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.565+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 --> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f5748074ff0 con 0x7f572c0a2d80 2024-03-21T09:12:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.639+0000 7f573ea00640 1 -- 172.21.15.37:0/3617767985 <== mgr.25553 v2:172.21.15.37:6800/1212038665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5748074ff0 con 0x7f572c0a2d80 2024-03-21T09:12:12.634 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-03-21T09:12:12.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.641+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f572c0a2d80 msgr2=0x7f572c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.641+0000 7f574ce00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f572c0a2d80 0x7f572c0a5240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f573402c7b0 tx=0x7f5734005fd0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.641+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 msgr2=0x7f5748199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.641+0000 7f574ce00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 0x7f5748199c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f573800e9c0 tx=0x7f573800ee80 comp rx=0 tx=0).stop 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 shutdown_connections 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57481032a0 0x7f57481a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57481028e0 0x7f574819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f572c0a2d80 0x7f572c0a5240 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 --2- 172.21.15.37:0/3617767985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57481016f0 0x7f5748199c50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 >> 172.21.15.37:0/3617767985 conn(0x7f57480fcc60 msgr2=0x7f57480738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 shutdown_connections 2024-03-21T09:12:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.642+0000 7f574ce00640 1 -- 172.21.15.37:0/3617767985 wait complete. 2024-03-21T09:12:12.654 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-21T09:12:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 -- 172.21.15.37:0/699782329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd610103900 msgr2=0x7fd610103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 --2- 172.21.15.37:0/699782329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd610103900 0x7fd610103d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002bc40 comp rx=0 tx=0).stop 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 -- 172.21.15.37:0/699782329 shutdown_connections 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 --2- 172.21.15.37:0/699782329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6101054b0 0x7fd61010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 --2- 172.21.15.37:0/699782329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd610104af0 0x7fd610104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 --2- 172.21.15.37:0/699782329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd610103900 0x7fd610103d20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.853+0000 7fd615400640 1 -- 172.21.15.37:0/699782329 >> 172.21.15.37:0/699782329 conn(0x7fd6100fee90 msgr2=0x7fd6101012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.854+0000 7fd615400640 1 -- 172.21.15.37:0/699782329 shutdown_connections 2024-03-21T09:12:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.854+0000 7fd615400640 1 -- 172.21.15.37:0/699782329 wait complete. 2024-03-21T09:12:12.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.854+0000 7fd615400640 1 Processor -- start 2024-03-21T09:12:12.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 -- start start 2024-03-21T09:12:12.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd610103900 0x7fd610199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd610104af0 0x7fd61019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 0x7fd6101a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61010f5a0 con 0x7fd610103900 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61010f420 con 0x7fd6101054b0 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd615400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61010f720 con 0x7fd610104af0 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd60f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 0x7fd6101a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd60de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd610104af0 0x7fd61019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd60e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd610103900 0x7fd610199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd60f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 0x7fd6101a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46950/0 (socket says 172.21.15.37:46950) 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd60f200640 1 -- 172.21.15.37:0/1316154077 learned_addr learned my addr 172.21.15.37:0/1316154077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.855+0000 7fd60de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd610104af0 0x7fd61019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56240/0 (socket says 172.21.15.37:56240) 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd60f200640 1 -- 172.21.15.37:0/1316154077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd610104af0 msgr2=0x7fd61019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd60f200640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd610104af0 0x7fd61019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd60f200640 1 -- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd610103900 msgr2=0x7fd610199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd60f200640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd610103900 0x7fd610199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:12.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd60f200640 1 -- 172.21.15.37:0/1316154077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6101a17d0 con 0x7fd6101054b0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd60f200640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 0x7fd6101a1160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd60800eb10 tx=0x7fd60800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608004b20 con 0x7fd6101054b0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd608004cc0 con 0x7fd6101054b0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6101a1ac0 con 0x7fd6101054b0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608010760 con 0x7fd6101054b0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.856+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6101a1fc0 con 0x7fd6101054b0 2024-03-21T09:12:12.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.857+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61010ac20 con 0x7fd6101054b0 2024-03-21T09:12:12.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.860+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 137666+0+0 (secure 0 0 0) 0x7fd608005020 con 0x7fd6101054b0 2024-03-21T09:12:12.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.862+0000 7fd607400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd5f40a2d00 0x7fd5f40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:12.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.862+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd6080cfa50 con 0x7fd6101054b0 2024-03-21T09:12:12.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.862+0000 7fd60e800640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd5f40a2d00 0x7fd5f40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:12.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.863+0000 7fd60e800640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd5f40a2d00 0x7fd5f40a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd60002c180 tx=0x7fd60002cf90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:12.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:12.865+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd60809c670 con 0x7fd6101054b0 2024-03-21T09:12:13.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.096+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 --> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fd610061ae0 con 0x7fd5f40a2d00 2024-03-21T09:12:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.108+0000 7fd607400640 1 -- 172.21.15.37:0/1316154077 <== mgr.25553 v2:172.21.15.37:6800/1212038665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fd610061ae0 con 0x7fd5f40a2d00 2024-03-21T09:12:13.102 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-21T00:00:00", "created": "2024-03-21T09:12:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-21T09:12:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.110+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd5f40a2d00 msgr2=0x7fd5f40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.110+0000 7fd615400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd5f40a2d00 0x7fd5f40a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd60002c180 tx=0x7fd60002cf90 comp rx=0 tx=0).stop 2024-03-21T09:12:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.110+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 msgr2=0x7fd6101a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.110+0000 7fd615400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 0x7fd6101a1160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd60800eb10 tx=0x7fd60800c2d0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 shutdown_connections 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd5f40a2d00 0x7fd5f40a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6101054b0 0x7fd6101a1160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd610104af0 0x7fd61019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 --2- 172.21.15.37:0/1316154077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd610103900 0x7fd610199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 >> 172.21.15.37:0/1316154077 conn(0x7fd6100fee90 msgr2=0x7fd610107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 shutdown_connections 2024-03-21T09:12:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.111+0000 7fd615400640 1 -- 172.21.15.37:0/1316154077 wait complete. 2024-03-21T09:12:13.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 -- 172.21.15.37:0/3091035303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 msgr2=0x7fef340744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 --2- 172.21.15.37:0/3091035303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef340744e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef20009900 tx=0x7fef2002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 -- 172.21.15.37:0/3091035303 shutdown_connections 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 --2- 172.21.15.37:0/3091035303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34109590 0x7fef3410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 --2- 172.21.15.37:0/3091035303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef340744e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 --2- 172.21.15.37:0/3091035303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef34075660 0x7fef34073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 -- 172.21.15.37:0/3091035303 >> 172.21.15.37:0/3091035303 conn(0x7fef340fcc70 msgr2=0x7fef340ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 -- 172.21.15.37:0/3091035303 shutdown_connections 2024-03-21T09:12:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.328+0000 7fef3b400640 1 -- 172.21.15.37:0/3091035303 wait complete. 2024-03-21T09:12:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.329+0000 7fef3b400640 1 Processor -- start 2024-03-21T09:12:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.329+0000 7fef3b400640 1 -- start start 2024-03-21T09:12:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef3b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef34199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef3b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef34075660 0x7fef3419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef3b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34109590 0x7fef341a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef3b400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3410f540 con 0x7fef34075660 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef3b400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3410f3c0 con 0x7fef34109590 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef3b400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3410f6c0 con 0x7fef34074080 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef38a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef34199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef38a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef34199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56260/0 (socket says 172.21.15.37:56260) 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef38a00640 1 -- 172.21.15.37:0/2429424334 learned_addr learned my addr 172.21.15.37:0/2429424334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef33e00640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef34075660 0x7fef3419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.330+0000 7fef39400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34109590 0x7fef341a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef38a00640 1 -- 172.21.15.37:0/2429424334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34109590 msgr2=0x7fef341a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef38a00640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34109590 0x7fef341a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef38a00640 1 -- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef34075660 msgr2=0x7fef3419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef38a00640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef34075660 0x7fef3419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef38a00640 1 -- 172.21.15.37:0/2429424334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef341a17f0 con 0x7fef34074080 2024-03-21T09:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef38a00640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef34199bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fef2800e9f0 tx=0x7fef2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2800cd80 con 0x7fef34074080 2024-03-21T09:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef28010910 con 0x7fef34074080 2024-03-21T09:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef341a1ae0 con 0x7fef34074080 2024-03-21T09:12:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.331+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28018990 con 0x7fef34074080 2024-03-21T09:12:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.332+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef341a2000 con 0x7fef34074080 2024-03-21T09:12:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.333+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34108350 con 0x7fef34074080 2024-03-21T09:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.334+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 137974+0+0 (secure 0 0 0) 0x7fef28010ac0 con 0x7fef34074080 2024-03-21T09:12:13.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.337+0000 7fef31600640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef1c0a3080 0x7fef1c0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.337+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fef280cfff0 con 0x7fef34074080 2024-03-21T09:12:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.337+0000 7fef33e00640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef1c0a3080 0x7fef1c0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.338+0000 7fef33e00640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef1c0a3080 0x7fef1c0a5540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef20006010 tx=0x7fef2002df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:13.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.340+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef2809c740 con 0x7fef34074080 2024-03-21T09:12:13.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:13 smithi071 ceph-mon[36597]: [21/Mar/2024:09:12:11] ENGINE Bus STARTING 2024-03-21T09:12:13.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:13 smithi071 ceph-mon[36597]: from='client.25603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:13.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:13 smithi071 ceph-mon[36597]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:13.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:13 smithi071 ceph-mon[36597]: [21/Mar/2024:09:12:12] ENGINE Serving on http://172.21.15.37:8765 2024-03-21T09:12:13.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:13 smithi071 ceph-mon[36597]: [21/Mar/2024:09:12:12] ENGINE Serving on https://172.21.15.37:7150 2024-03-21T09:12:13.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:13 smithi071 ceph-mon[36597]: [21/Mar/2024:09:12:12] ENGINE Bus STARTED 2024-03-21T09:12:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:13 smithi037 ceph-mon[31830]: [21/Mar/2024:09:12:11] ENGINE Bus STARTING 2024-03-21T09:12:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:13 smithi037 ceph-mon[31830]: from='client.25603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:13 smithi037 ceph-mon[31830]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:13 smithi037 ceph-mon[31830]: [21/Mar/2024:09:12:12] ENGINE Serving on http://172.21.15.37:8765 2024-03-21T09:12:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:13 smithi037 ceph-mon[31830]: [21/Mar/2024:09:12:12] ENGINE Serving on https://172.21.15.37:7150 2024-03-21T09:12:13.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:13 smithi037 ceph-mon[31830]: [21/Mar/2024:09:12:12] ENGINE Bus STARTED 2024-03-21T09:12:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.590+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 --> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fef34061ae0 con 0x7fef1c0a3080 2024-03-21T09:12:13.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.602+0000 7fef31600640 1 -- 172.21.15.37:0/2429424334 <== mgr.25553 v2:172.21.15.37:6800/1212038665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fef34061ae0 con 0x7fef1c0a3080 2024-03-21T09:12:13.595 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef1c0a3080 msgr2=0x7fef1c0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef1c0a3080 0x7fef1c0a5540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef20006010 tx=0x7fef2002df90 comp rx=0 tx=0).stop 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 msgr2=0x7fef34199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef34199bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fef2800e9f0 tx=0x7fef2800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 shutdown_connections 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef1c0a3080 0x7fef1c0a5540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34109590 0x7fef341a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef34075660 0x7fef3419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 --2- 172.21.15.37:0/2429424334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef34074080 0x7fef34199bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 >> 172.21.15.37:0/2429424334 conn(0x7fef340fcc70 msgr2=0x7fef34105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.604+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 shutdown_connections 2024-03-21T09:12:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:13.605+0000 7fef3b400640 1 -- 172.21.15.37:0/2429424334 wait complete. 2024-03-21T09:12:13.615 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-21T09:12:13.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:13 smithi053 ceph-mon[36169]: [21/Mar/2024:09:12:11] ENGINE Bus STARTING 2024-03-21T09:12:13.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:13 smithi053 ceph-mon[36169]: from='client.25603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:13.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:13 smithi053 ceph-mon[36169]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:13.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:13 smithi053 ceph-mon[36169]: [21/Mar/2024:09:12:12] ENGINE Serving on http://172.21.15.37:8765 2024-03-21T09:12:13.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:13 smithi053 ceph-mon[36169]: [21/Mar/2024:09:12:12] ENGINE Serving on https://172.21.15.37:7150 2024-03-21T09:12:13.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:13 smithi053 ceph-mon[36169]: [21/Mar/2024:09:12:12] ENGINE Bus STARTED 2024-03-21T09:12:13.716 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-21T09:12:13.729 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-21T09:12:13.767 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:12:13.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-21T09:12:13.776 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T09:12:13.801 INFO:tasks.workunit:Pulling workunits from ref 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:12:13.802 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T09:12:13.802 INFO:tasks.workunit:timeout=3h 2024-03-21T09:12:13.802 INFO:tasks.workunit:cleanup=True 2024-03-21T09:12:13.802 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-21 09:09:22.875930626 +0000 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-21 09:09:22.875930626 +0000 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-21 09:11:47.991670315 +0000 2024-03-21T09:12:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-21 09:09:22.875930626 +0000 2024-03-21T09:12:13.858 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-21T09:12:13.858 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-21T09:12:13.949 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:12:13.995 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T09:12:14.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: from='client.25610 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:14.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: from='client.25616 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-21T09:12:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:14 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: from='client.25610 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:14.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:14.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: from='client.25616 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-21T09:12:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:14 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: from='client.25610 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: from='client.25616 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:14 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='client.25657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:15 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='client.25657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:15 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='client.25657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:12:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:15.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:15.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:15 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:16 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi071 to 174.1M 2024-03-21T09:12:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:16 smithi071 ceph-mon[36597]: Unable to set osd_memory_target on smithi071 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-21T09:12:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:16 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi053 to 175.2M 2024-03-21T09:12:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:16 smithi071 ceph-mon[36597]: Unable to set osd_memory_target on smithi053 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-21T09:12:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:16 smithi071 ceph-mon[36597]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:16.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:16 smithi071 ceph-mon[36597]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-21T09:12:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:16 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi071 to 174.1M 2024-03-21T09:12:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:16 smithi037 ceph-mon[31830]: Unable to set osd_memory_target on smithi071 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-21T09:12:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:16 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi053 to 175.2M 2024-03-21T09:12:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:16 smithi037 ceph-mon[31830]: Unable to set osd_memory_target on smithi053 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-21T09:12:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:16 smithi037 ceph-mon[31830]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:16 smithi037 ceph-mon[31830]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-21T09:12:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:16 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi071 to 174.1M 2024-03-21T09:12:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:16 smithi053 ceph-mon[36169]: Unable to set osd_memory_target on smithi071 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-21T09:12:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:16 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi053 to 175.2M 2024-03-21T09:12:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:16 smithi053 ceph-mon[36169]: Unable to set osd_memory_target on smithi053 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-21T09:12:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:16 smithi053 ceph-mon[36169]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:12:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:16 smithi053 ceph-mon[36169]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-21T09:12:18.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 185 KiB/s wr, 107 op/s 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi037 to 177.8M 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: Unable to set osd_memory_target on smithi037 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T09:12:18.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:12:18.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:12:18.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-21T09:12:18.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: Updating smithi053:/etc/ceph/ceph.conf 2024-03-21T09:12:18.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:18 smithi037 ceph-mon[31830]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 185 KiB/s wr, 107 op/s 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi037 to 177.8M 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: Unable to set osd_memory_target on smithi037 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: Updating smithi053:/etc/ceph/ceph.conf 2024-03-21T09:12:18.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:18 smithi053 ceph-mon[36169]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-21T09:12:18.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 185 KiB/s wr, 107 op/s 2024-03-21T09:12:18.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi037 to 177.8M 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: Unable to set osd_memory_target on smithi037 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:12:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-21T09:12:18.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: Updating smithi053:/etc/ceph/ceph.conf 2024-03-21T09:12:18.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:18 smithi071 ceph-mon[36597]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-21T09:12:19.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:19 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:19 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: Updating smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:19.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:19 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:20 smithi037 ceph-mon[31830]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:20 smithi037 ceph-mon[31830]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-21T09:12:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:20 smithi037 ceph-mon[31830]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:20 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:20 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:20 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:20 smithi053 ceph-mon[36169]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:20 smithi053 ceph-mon[36169]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-21T09:12:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:20 smithi053 ceph-mon[36169]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:20 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:20 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:20 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:20 smithi071 ceph-mon[36597]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:12:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:20 smithi071 ceph-mon[36597]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-21T09:12:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:20 smithi071 ceph-mon[36597]: Updating smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:12:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:20 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:20 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:20 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:21.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:21 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:12:21.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:21 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:12:21.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:21 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:12:21.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:21 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:21 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:12:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:21 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:12:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:21 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:12:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:21 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:21 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:12:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:21 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:12:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:21 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:12:21.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:21 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:12:22.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:22 smithi053 ceph-mon[36169]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-03-21T09:12:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:22 smithi071 ceph-mon[36597]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-03-21T09:12:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:22 smithi037 ceph-mon[31830]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-03-21T09:12:24.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:24 smithi053 ceph-mon[36169]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-21T09:12:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:24 smithi071 ceph-mon[36597]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-21T09:12:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:24 smithi037 ceph-mon[31830]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-21T09:12:26.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:26 smithi053 ceph-mon[36169]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-21T09:12:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:26 smithi071 ceph-mon[36597]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-21T09:12:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:26 smithi037 ceph-mon[31830]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-21T09:12:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 -- 172.21.15.37:0/1943326795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4074360 msgr2=0x7fa7c4111f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/1943326795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4074360 0x7fa7c4111f50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009900 tx=0x7fa7b002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 -- 172.21.15.37:0/1943326795 shutdown_connections 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/1943326795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c4112490 0x7fa7c4114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/1943326795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4074360 0x7fa7c4111f50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/1943326795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4073a40 0x7fa7c4073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.071+0000 7fa7c8a00640 1 -- 172.21.15.37:0/1943326795 >> 172.21.15.37:0/1943326795 conn(0x7fa7c40fcf50 msgr2=0x7fa7c40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.072+0000 7fa7c8a00640 1 -- 172.21.15.37:0/1943326795 shutdown_connections 2024-03-21T09:12:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.072+0000 7fa7c8a00640 1 -- 172.21.15.37:0/1943326795 wait complete. 2024-03-21T09:12:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 Processor -- start 2024-03-21T09:12:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 -- start start 2024-03-21T09:12:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4073a40 0x7fa7c419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c4074360 0x7fa7c419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 0x7fa7c41a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4117ee0 con 0x7fa7c4074360 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4117d60 con 0x7fa7c4073a40 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4118060 con 0x7fa7c4112490 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 0x7fa7c41a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4073a40 0x7fa7c419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 0x7fa7c41a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34272/0 (socket says 172.21.15.37:34272) 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c3200640 1 -- 172.21.15.37:0/2645322425 learned_addr learned my addr 172.21.15.37:0/2645322425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.073+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c4074360 0x7fa7c419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c2800640 1 -- 172.21.15.37:0/2645322425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 msgr2=0x7fa7c41a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c2800640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 0x7fa7c41a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c2800640 1 -- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c4074360 msgr2=0x7fa7c419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c2800640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c4074360 0x7fa7c419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c2800640 1 -- 172.21.15.37:0/2645322425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c41a3400 con 0x7fa7c4073a40 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c3200640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 0x7fa7c41a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c2800640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4073a40 0x7fa7c419e460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9f0 tx=0x7fa7b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7bb400640 1 -- 172.21.15.37:0/2645322425 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ccc0 con 0x7fa7c4073a40 2024-03-21T09:12:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7bb400640 1 -- 172.21.15.37:0/2645322425 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4005020 con 0x7fa7c4073a40 2024-03-21T09:12:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c41a36f0 con 0x7fa7c4073a40 2024-03-21T09:12:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.074+0000 7fa7bb400640 1 -- 172.21.15.37:0/2645322425 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40049e0 con 0x7fa7c4073a40 2024-03-21T09:12:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.075+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c41aaf80 con 0x7fa7c4073a40 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.079+0000 7fa7cc1aa740 1 Processor -- start 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.079+0000 7fa7cc1aa740 1 -- start start 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.079+0000 7fa7bb400640 1 -- 172.21.15.37:0/2645322425 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fa7b4005560 con 0x7fa7c4073a40 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x5589475585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x558947561390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947563cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894751a3b0 con 0x5589475581e0 2024-03-21T09:12:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589474bc830 con 0x558947558b00 2024-03-21T09:12:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589474655d0 con 0x5589475618d0 2024-03-21T09:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x5589475585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x558947561390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x5589475585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42570/0 (socket says 172.21.15.37:42570) 2024-03-21T09:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c3200640 1 -- 172.21.15.37:0/475351190 learned_addr learned my addr 172.21.15.37:0/475351190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947563cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c1e00640 1 -- 172.21.15.37:0/475351190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 msgr2=0x558947561390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x558947561390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c1e00640 1 -- 172.21.15.37:0/475351190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 msgr2=0x5589475585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x5589475585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c1e00640 1 -- 172.21.15.37:0/475351190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894747d2c0 con 0x5589475618d0 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.080+0000 7fa7c3200640 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x5589475585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.081+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947563cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7b002c200 tx=0x7fa7b0009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca58a840dbbd45d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.081+0000 7fa7af400640 1 -- 172.21.15.37:0/475351190 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0043070 con 0x5589475618d0 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.081+0000 7fa7af400640 1 -- 172.21.15.37:0/475351190 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b002c6e0 con 0x5589475618d0 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.081+0000 7fa7af400640 1 -- 172.21.15.37:0/475351190 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00049b0 con 0x5589475618d0 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.081+0000 7fa7bb400640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa7a80a3180 0x7fa7a80a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7bb400640 1 -- 172.21.15.37:0/2645322425 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa7b40cfc50 con 0x7fa7c4073a40 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7c3200640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa7a80a3180 0x7fa7a80a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/475351190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 msgr2=0x558947563cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947563cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7b002c200 tx=0x7fa7b0009560 comp rx=0 tx=0).stop 2024-03-21T09:12:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/475351190 shutdown_connections 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947563cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x558947561390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/475351190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x5589475585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/475351190 >> 172.21.15.37:0/475351190 conn(0x558947552b00 msgr2=0x558947552ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/475351190 shutdown_connections 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.082+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/475351190 wait complete. 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7c3200640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa7a80a3180 0x7fa7a80a5640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0094d0 tx=0x7fa7bc009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 -- start start 2024-03-21T09:12:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 Processor -- start 2024-03-21T09:12:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 0x55894755e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475618d0 0x55894755f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558947478a20 con 0x5589475618d0 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589474b88d0 con 0x5589475581e0 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.083+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589475644a0 con 0x558947558b00 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7cc1aa740 10 client.0 _ll_register_callbacks cb 0x5589474b51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 0x55894755e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 0x55894755e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59274/0 (socket says 172.21.15.37:59274) 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 -- 172.21.15.37:0/1592708366 learned_addr learned my addr 172.21.15.37:0/1592708366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34280/0 (socket says 172.21.15.37:34280) 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c3200640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475618d0 0x55894755f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 -- 172.21.15.37:0/1592708366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 msgr2=0x55894755ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 -- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475618d0 msgr2=0x55894755f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475618d0 0x55894755f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 -- 172.21.15.37:0/1592708366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589471d1210 con 0x5589475581e0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c3200640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475618d0 0x55894755f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7c2800640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 0x55894755e940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c6a0 tx=0x7fa7b4005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92374dac10bdd93e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.084+0000 7fa7a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1592708366 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b409ce60 con 0x5589475581e0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1592708366 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa7b40a42b0 con 0x5589475581e0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7b002c6e0 con 0x5589475581e0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1592708366 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7b40b4e60 con 0x5589475581e0 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 10 client.25640 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55894729d620 con 0x5589475581e0 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1592708366 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b002c6e0 con 0x5589475581e0 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1592708366 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa7b4004bc0 con 0x5589475581e0 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 10 client.25640 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 20 client.25640 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.085+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558947571970 0x558947573d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.086+0000 7fa7cc1aa740 4 client.25640 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.086+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589474c9d70 con 0x558947571970 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.086+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558947571970 0x558947573d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.086+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558947571970 0x558947573d60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5dd0ee730898d28 server_cookie=dc73dba102e45228 in_seq=0 out_seq=0 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.086+0000 7fa7a3e00640 10 client.25640 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.087+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1592708366 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5589474c9d70 con 0x558947571970 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.087+0000 7fa7a3e00640 10 client.25640 handle_command_reply: tid=0 2024-03-21T09:12:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.087+0000 7fa7cc1aa740 1 client.25640 shutdown 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.087+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 msgr2=0x55894755e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 0x55894755e940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c6a0 tx=0x7fa7b4005440 comp rx=0 tx=0).stop 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 shutdown_connections 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475618d0 0x55894755f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558947571970 0x558947573d60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1592708366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475581e0 0x55894755e940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7a3e00640 0 client.25640 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 >> 172.21.15.37:0/1592708366 conn(0x558947552b00 msgr2=0x558947554850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 shutdown_connections 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1592708366 wait complete. 2024-03-21T09:12:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.088+0000 7fa7cc1aa740 20 client.25640 trim_cache size 0 max 16384 2024-03-21T09:12:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.103+0000 7fa7cc1aa740 1 Processor -- start 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.103+0000 7fa7cc1aa740 1 -- start start 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947561070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475618d0 0x55894755d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41b7250 con 0x5589475581e0 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41b4d50 con 0x5589475618d0 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41b2df0 con 0x558947558b00 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475618d0 0x55894755d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947561070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475618d0 0x55894755d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59282/0 (socket says 172.21.15.37:59282) 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c3200640 1 -- 172.21.15.37:0/3928601733 learned_addr learned my addr 172.21.15.37:0/3928601733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34286/0 (socket says 172.21.15.37:34286) 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 -- 172.21.15.37:0/3928601733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475618d0 msgr2=0x55894755d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475618d0 0x55894755d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 -- 172.21.15.37:0/3928601733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 msgr2=0x558947561070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947561070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.104+0000 7fa7c1e00640 1 -- 172.21.15.37:0/3928601733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589475740d0 con 0x558947558b00 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.105+0000 7fa7c1e00640 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755d210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7b002c8b0 tx=0x7fa7b002bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bf4b6e76e8c635 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.105+0000 7fa7aea00640 1 -- 172.21.15.37:0/3928601733 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0043040 con 0x558947558b00 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.105+0000 7fa7aea00640 1 -- 172.21.15.37:0/3928601733 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b002cd10 con 0x558947558b00 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.105+0000 7fa7aea00640 1 -- 172.21.15.37:0/3928601733 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0005140 con 0x558947558b00 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/3928601733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 msgr2=0x55894755d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755d210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7b002c8b0 tx=0x7fa7b002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/3928601733 shutdown_connections 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589475618d0 0x55894755d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558947558b00 0x55894755d210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/3928601733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947561070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/3928601733 >> 172.21.15.37:0/3928601733 conn(0x558947552b00 msgr2=0x55894754d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/3928601733 shutdown_connections 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/3928601733 wait complete. 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 -- start start 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.106+0000 7fa7cc1aa740 1 Processor -- start 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947569560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x5589475648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947564ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41b33f0 con 0x5589475581e0 2024-03-21T09:12:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41b46f0 con 0x558947558b00 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41b3a30 con 0x5589475618d0 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7cc1aa740 10 client.0 _ll_register_callbacks cb 0x7fa7b40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947564ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x5589475648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7c2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947569560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947564ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34288/0 (socket says 172.21.15.37:34288) 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7c1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x5589475648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59286/0 (socket says 172.21.15.37:59286) 2024-03-21T09:12:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.107+0000 7fa7c3200640 1 -- 172.21.15.37:0/1414750722 learned_addr learned my addr 172.21.15.37:0/1414750722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7c3200640 1 -- 172.21.15.37:0/1414750722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 msgr2=0x5589475648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7c3200640 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x5589475648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7c3200640 1 -- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 msgr2=0x558947569560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7c3200640 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947569560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7c3200640 1 -- 172.21.15.37:0/1414750722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589475656b0 con 0x5589475618d0 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7c3200640 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947564ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0041a0 tx=0x7fa7bc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c79fe7d28a042f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1414750722 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc004b20 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7b002cd10 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.108+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1414750722 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa7bc004cc0 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1414750722 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7bc034050 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 10 client.25675 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5589475cd370 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1414750722 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7b002cd10 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1414750722 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa7bc028070 con 0x5589475618d0 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 10 client.25675 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 20 client.25675 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5589475cdaa0 0x5589475cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 4 client.25675 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:12:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.109+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589474c40c0 con 0x5589475cdaa0 2024-03-21T09:12:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.110+0000 7fa7c2800640 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5589475cdaa0 0x5589475cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.110+0000 7fa7c2800640 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5589475cdaa0 0x5589475cfec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a600976c4bc12d server_cookie=a005f3c44d7873fd in_seq=0 out_seq=0 2024-03-21T09:12:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.110+0000 7fa7a3e00640 10 client.25675 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7a3e00640 1 -- 172.21.15.37:0/1414750722 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589474c40c0 con 0x5589475cdaa0 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7a3e00640 10 client.25675 handle_command_reply: tid=0 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 client.25675 shutdown 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 msgr2=0x558947564ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947564ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0041a0 tx=0x7fa7bc004ab0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 shutdown_connections 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5589475618d0 0x558947564ee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5589475cdaa0 0x5589475cfec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558947558b00 0x5589475648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7a3e00640 0 client.25675 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 --2- 172.21.15.37:0/1414750722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5589475581e0 0x558947569560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 >> 172.21.15.37:0/1414750722 conn(0x558947552b00 msgr2=0x55894755d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 shutdown_connections 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 1 -- 172.21.15.37:0/1414750722 wait complete. 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.124+0000 7fa7cc1aa740 20 client.25675 trim_cache size 0 max 16384 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:12:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:12:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:12:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa7a80a3180 msgr2=0x7fa7a80a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa7a80a3180 0x7fa7a80a5640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0094d0 tx=0x7fa7bc009500 comp rx=0 tx=0).stop 2024-03-21T09:12:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4073a40 msgr2=0x7fa7c419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4073a40 0x7fa7c419e460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9f0 tx=0x7fa7b400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 shutdown_connections 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4112490 0x7fa7c41a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c4074360 0x7fa7c419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa7a80a3180 0x7fa7a80a5640 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 --2- 172.21.15.37:0/2645322425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c4073a40 0x7fa7c419e460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 >> 172.21.15.37:0/2645322425 conn(0x7fa7c40fcf50 msgr2=0x7fa7c40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.125+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 shutdown_connections 2024-03-21T09:12:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:27.126+0000 7fa7c8a00640 1 -- 172.21.15.37:0/2645322425 wait complete. 2024-03-21T09:12:28.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:28 smithi053 ceph-mon[36169]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-21T09:12:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:28 smithi071 ceph-mon[36597]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-21T09:12:28.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:28 smithi037 ceph-mon[31830]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-21T09:12:30.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:12:30.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:12:30.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 -- 172.21.15.37:0/1653106064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b24102f50 msgr2=0x7f4b2410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1653106064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b24102f50 0x7f4b2410b430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009900 tx=0x7f4b1002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 -- 172.21.15.37:0/1653106064 shutdown_connections 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1653106064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2410ba50 0x7f4b2410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1653106064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b24102f50 0x7f4b2410b430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1653106064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.552+0000 7f4b2a200640 1 -- 172.21.15.37:0/1653106064 >> 172.21.15.37:0/1653106064 conn(0x7f4b24077130 msgr2=0x7f4b240fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.553+0000 7f4b2a200640 1 -- 172.21.15.37:0/1653106064 shutdown_connections 2024-03-21T09:12:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.553+0000 7f4b2a200640 1 -- 172.21.15.37:0/1653106064 wait complete. 2024-03-21T09:12:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 Processor -- start 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 -- start start 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b24102f50 0x7f4b2419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b2410ba50 0x7f4b2419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24111870 con 0x7f4b24102630 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241116f0 con 0x7f4b2410ba50 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b2a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241119f0 con 0x7f4b24102f50 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b28e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b28e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42622/0 (socket says 172.21.15.37:42622) 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.554+0000 7f4b28e00640 1 -- 172.21.15.37:0/1717180913 learned_addr learned my addr 172.21.15.37:0/1717180913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b29800640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b2410ba50 0x7f4b2419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b23e00640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b24102f50 0x7f4b2419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b28e00640 1 -- 172.21.15.37:0/1717180913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b24102f50 msgr2=0x7f4b2419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b28e00640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b24102f50 0x7f4b2419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b28e00640 1 -- 172.21.15.37:0/1717180913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b2410ba50 msgr2=0x7f4b2419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b28e00640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b2410ba50 0x7f4b2419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b28e00640 1 -- 172.21.15.37:0/1717180913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2419efc0 con 0x7f4b24102630 2024-03-21T09:12:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b28e00640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24199fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800e9f0 tx=0x7f4b1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.555+0000 7f4b21600640 1 -- 172.21.15.37:0/1717180913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800cd80 con 0x7f4b24102630 2024-03-21T09:12:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.556+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2419f2b0 con 0x7f4b24102630 2024-03-21T09:12:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.556+0000 7f4b21600640 1 -- 172.21.15.37:0/1717180913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b18010910 con 0x7f4b24102630 2024-03-21T09:12:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.556+0000 7f4b21600640 1 -- 172.21.15.37:0/1717180913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18018990 con 0x7f4b24102630 2024-03-21T09:12:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.556+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b241a6b40 con 0x7f4b24102630 2024-03-21T09:12:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.559+0000 7f4b21600640 1 -- 172.21.15.37:0/1717180913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f4b18010ac0 con 0x7f4b24102630 2024-03-21T09:12:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.559+0000 7f4b2e5ec740 1 Processor -- start 2024-03-21T09:12:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.559+0000 7f4b2e5ec740 1 -- start start 2024-03-21T09:12:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 0x561302c74330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 0x561302c74d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c80060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561302bd0250 con 0x561302c755e0 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561302b78a30 con 0x561302c7dc70 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561302bcc2f0 con 0x561302c74900 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b28e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 0x561302c74d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b29800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 0x561302c74330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c80060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b29800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 0x561302c74330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42624/0 (socket says 172.21.15.37:42624) 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b28e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 0x561302c74d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34316/0 (socket says 172.21.15.37:34316) 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.560+0000 7f4b29800640 1 -- 172.21.15.37:0/2431979215 learned_addr learned my addr 172.21.15.37:0/2431979215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b29800640 1 -- 172.21.15.37:0/2431979215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 msgr2=0x561302c74d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b29800640 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 0x561302c74d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b29800640 1 -- 172.21.15.37:0/2431979215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 msgr2=0x561302c80060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b29800640 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c80060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b29800640 1 -- 172.21.15.37:0/2431979215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561302b63690 con 0x561302c755e0 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b28e00640 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 0x561302c74d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b29800640 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 0x561302c74330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009f50 tx=0x7f4b1c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b37aafd5aa8f38 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b16000640 1 -- 172.21.15.37:0/2431979215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c004d20 con 0x561302c755e0 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b16000640 1 -- 172.21.15.37:0/2431979215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c004050 con 0x561302c755e0 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.561+0000 7f4b16000640 1 -- 172.21.15.37:0/2431979215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c005280 con 0x561302c755e0 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b21600640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4b0c0a3120 0x7f4b0c0a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b21600640 1 -- 172.21.15.37:0/1717180913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4b180d0090 con 0x7f4b24102630 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b28e00640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4b0c0a3120 0x7f4b0c0a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2431979215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 msgr2=0x561302c74330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 0x561302c74330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009f50 tx=0x7f4b1c0118d0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2431979215 shutdown_connections 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c74900 0x561302c74d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2431979215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c755e0 0x561302c74330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.562+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2431979215 >> 172.21.15.37:0/2431979215 conn(0x561302c33200 msgr2=0x561302c6f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.563+0000 7f4b28e00640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4b0c0a3120 0x7f4b0c0a55e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b180cd380 tx=0x7f4b180d3a90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.563+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2431979215 shutdown_connections 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.563+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2431979215 wait complete. 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.563+0000 7f4b2e5ec740 1 -- start start 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.563+0000 7f4b2e5ec740 1 Processor -- start 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c7ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561302c80ae0 con 0x561302c74900 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561302b8be80 con 0x561302c7dc70 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561302c80c60 con 0x561302c755e0 2024-03-21T09:12:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b2e5ec740 10 client.0 _ll_register_callbacks cb 0x561302b9e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34324/0 (socket says 172.21.15.37:34324) 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b23e00640 1 -- 172.21.15.37:0/2742028268 learned_addr learned my addr 172.21.15.37:0/2742028268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b29800640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.564+0000 7f4b29800640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c7ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b23e00640 1 -- 172.21.15.37:0/2742028268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 msgr2=0x561302c7ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c7ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b23e00640 1 -- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 msgr2=0x561302c7d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b23e00640 1 -- 172.21.15.37:0/2742028268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613029ae7d0 con 0x561302c755e0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b29800640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7a700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009230 tx=0x7f4b10005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85919a271dfad8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2742028268 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10043070 con 0x561302c755e0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b1c004050 con 0x561302c755e0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.565+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2742028268 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4b100046c0 con 0x561302c755e0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2742028268 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b10050050 con 0x561302c755e0 2024-03-21T09:12:30.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 10 client.25687 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5613028e43f0 con 0x561302c755e0 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2742028268 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c004050 con 0x561302c755e0 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2742028268 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4b100035f0 con 0x561302c755e0 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 10 client.25687 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 20 client.25687 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c8db30 0x561302c8ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 4 client.25687 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.566+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561302bdd790 con 0x561302c8db30 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.567+0000 7f4b29800640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c8db30 0x561302c8ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.567+0000 7f4b29800640 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c8db30 0x561302c8ff20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed38eb64b2f5796 server_cookie=a1b81bf3f58861f5 in_seq=0 out_seq=0 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.567+0000 7f4b0aa00640 10 client.25687 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:30.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2742028268 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561302bdd790 con 0x561302c8db30 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b0aa00640 10 client.25687 handle_command_reply: tid=0 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 client.25687 shutdown 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 msgr2=0x561302c7a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7a700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009230 tx=0x7f4b10005dd0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 shutdown_connections 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c8db30 0x561302c8ff20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c7dc70 0x561302c7ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7a700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2742028268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b0aa00640 0 client.25687 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.568+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 >> 172.21.15.37:0/2742028268 conn(0x561302c33200 msgr2=0x561302c710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.569+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 shutdown_connections 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.569+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2742028268 wait complete. 2024-03-21T09:12:30.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.569+0000 7f4b2e5ec740 20 client.25687 trim_cache size 0 max 16384 2024-03-21T09:12:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.589+0000 7f4b2e5ec740 1 Processor -- start 2024-03-21T09:12:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 -- start start 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c33200 0x561302c7ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241aeba0 con 0x561302c33200 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241ac6a0 con 0x561302c74900 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241aa7a0 con 0x561302c755e0 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59350/0 (socket says 172.21.15.37:59350) 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b23e00640 1 -- 172.21.15.37:0/712007678 learned_addr learned my addr 172.21.15.37:0/712007678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b29800640 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c33200 0x561302c7ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.590+0000 7f4b29800640 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b23e00640 1 -- 172.21.15.37:0/712007678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 msgr2=0x561302c7f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b23e00640 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b23e00640 1 -- 172.21.15.37:0/712007678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c33200 msgr2=0x561302c7ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b23e00640 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c33200 0x561302c7ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b23e00640 1 -- 172.21.15.37:0/712007678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561302c90290 con 0x561302c74900 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b29800640 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b23e00640 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 0x561302c7d370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b1002f020 tx=0x7f4b1002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c68d1fdb9ff877 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b15600640 1 -- 172.21.15.37:0/712007678 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10043040 con 0x561302c74900 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b15600640 1 -- 172.21.15.37:0/712007678 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b10009520 con 0x561302c74900 2024-03-21T09:12:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.591+0000 7f4b15600640 1 -- 172.21.15.37:0/712007678 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10005210 con 0x561302c74900 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/712007678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 msgr2=0x561302c7d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 0x561302c7d370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b1002f020 tx=0x7f4b1002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/712007678 shutdown_connections 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c7f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c74900 0x561302c7d370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/712007678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c33200 0x561302c7ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/712007678 >> 172.21.15.37:0/712007678 conn(0x561302c7ada0 msgr2=0x561302c6ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/712007678 shutdown_connections 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/712007678 wait complete. 2024-03-21T09:12:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.592+0000 7f4b2e5ec740 1 -- start start 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 Processor -- start 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 0x561302c76990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c77410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241aada0 con 0x561302c74900 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241ac040 con 0x561302c33200 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241ab3a0 con 0x561302c755e0 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.593+0000 7f4b2e5ec740 10 client.0 _ll_register_callbacks cb 0x7f4b10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 0x561302c76990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 0x561302c76990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59354/0 (socket says 172.21.15.37:59354) 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 -- 172.21.15.37:0/2973135822 learned_addr learned my addr 172.21.15.37:0/2973135822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c77410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 -- 172.21.15.37:0/2973135822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 msgr2=0x561302c77410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c77410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 -- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 msgr2=0x561302c76ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c76ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 -- 172.21.15.37:0/2973135822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613028a40f0 con 0x561302c33200 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b29800640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 0x561302c76990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009730 tx=0x7f4b1c028040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=558fa5523d205fb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.594+0000 7f4b0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2973135822 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c02d2f0 con 0x561302c33200 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b10009520 con 0x561302c33200 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2973135822 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4b1c003b80 con 0x561302c33200 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2973135822 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b1c040050 con 0x561302c33200 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 10 client.25664 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561302cea560 con 0x561302c33200 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2973135822 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b10009520 con 0x561302c33200 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2973135822 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4b1c02d8e0 con 0x561302c33200 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 10 client.25664 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 20 client.25664 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c7dc70 0x561302ceca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 4 client.25664 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.595+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561302bd7ae0 con 0x561302c7dc70 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.596+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c7dc70 0x561302ceca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.596+0000 7f4b23e00640 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c7dc70 0x561302ceca10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ad41dd14ca8377 server_cookie=8ff8a35ad9792887 in_seq=0 out_seq=0 2024-03-21T09:12:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.596+0000 7f4b0aa00640 10 client.25664 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.596+0000 7f4b0aa00640 1 -- 172.21.15.37:0/2973135822 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561302bd7ae0 con 0x561302c7dc70 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.596+0000 7f4b0aa00640 10 client.25664 handle_command_reply: tid=0 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.596+0000 7f4b2e5ec740 1 client.25664 shutdown 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 msgr2=0x561302c76990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 0x561302c76990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009730 tx=0x7f4b1c028040 comp rx=0 tx=0).stop 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 shutdown_connections 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561302c755e0 0x561302c77410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561302c74900 0x561302c76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561302c7dc70 0x561302ceca10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 --2- 172.21.15.37:0/2973135822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561302c33200 0x561302c76990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b0aa00640 0 client.25664 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 >> 172.21.15.37:0/2973135822 conn(0x561302c7ada0 msgr2=0x561302c65330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 shutdown_connections 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 1 -- 172.21.15.37:0/2973135822 wait complete. 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.597+0000 7f4b2e5ec740 20 client.25664 trim_cache size 0 max 16384 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:12:30.596 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:12:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4b0c0a3120 msgr2=0x7f4b0c0a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4b0c0a3120 0x7f4b0c0a55e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b180cd380 tx=0x7f4b180d3a90 comp rx=0 tx=0).stop 2024-03-21T09:12:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 msgr2=0x7f4b24199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24199fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800e9f0 tx=0x7f4b1800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 shutdown_connections 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4b0c0a3120 0x7f4b0c0a55e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b2410ba50 0x7f4b2419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b24102f50 0x7f4b2419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 --2- 172.21.15.37:0/1717180913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b24102630 0x7f4b24199fd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 >> 172.21.15.37:0/1717180913 conn(0x7f4b24077130 msgr2=0x7f4b240fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 shutdown_connections 2024-03-21T09:12:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.598+0000 7f4b2a200640 1 -- 172.21.15.37:0/1717180913 wait complete. 2024-03-21T09:12:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:30 smithi071 ceph-mon[36597]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:12:30.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:30 smithi037 ceph-mon[31830]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:12:30.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:12:30.613 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:12:30.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:30 smithi053 ceph-mon[36169]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:12:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 -- 172.21.15.37:0/318465956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb82410bc20 msgr2=0x7fb82410e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 --2- 172.21.15.37:0/318465956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb82410bc20 0x7fb82410e0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009be0 tx=0x7fb81c02bf80 comp rx=0 tx=0).stop 2024-03-21T09:12:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 -- 172.21.15.37:0/318465956 shutdown_connections 2024-03-21T09:12:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 --2- 172.21.15.37:0/318465956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb82410bc20 0x7fb82410e0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 --2- 172.21.15.37:0/318465956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb824103120 0x7fb82410b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 --2- 172.21.15.37:0/318465956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824102800 0x7fb824102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 -- 172.21.15.37:0/318465956 >> 172.21.15.37:0/318465956 conn(0x7fb8240fc020 msgr2=0x7fb8240fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 -- 172.21.15.37:0/318465956 shutdown_connections 2024-03-21T09:12:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.874+0000 7fb82a800640 1 -- 172.21.15.37:0/318465956 wait complete. 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.875+0000 7fb82a800640 1 Processor -- start 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 -- start start 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 0x7fb824100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 0x7fb824100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb82410bc20 0x7fb824100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb824111a20 con 0x7fb82410bc20 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8241118a0 con 0x7fb824103120 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb82a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb824111ba0 con 0x7fb824102800 2024-03-21T09:12:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb822a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 0x7fb824100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb823400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 0x7fb824100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb823e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb82410bc20 0x7fb824100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb823400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 0x7fb824100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34378/0 (socket says 172.21.15.37:34378) 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb822a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 0x7fb824100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59378/0 (socket says 172.21.15.37:59378) 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.876+0000 7fb823400640 1 -- 172.21.15.37:0/2104256385 learned_addr learned my addr 172.21.15.37:0/2104256385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb822a00640 1 -- 172.21.15.37:0/2104256385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 msgr2=0x7fb824100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb822a00640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 0x7fb824100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb822a00640 1 -- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb82410bc20 msgr2=0x7fb824100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb822a00640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb82410bc20 0x7fb824100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb822a00640 1 -- 172.21.15.37:0/2104256385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8241a6380 con 0x7fb824103120 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb823400640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 0x7fb824100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb822a00640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 0x7fb824100990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb81000e7a0 tx=0x7fb81000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000ccc0 con 0x7fb824103120 2024-03-21T09:12:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb810009530 con 0x7fb824103120 2024-03-21T09:12:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810018880 con 0x7fb824103120 2024-03-21T09:12:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.877+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8241a6670 con 0x7fb824103120 2024-03-21T09:12:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.878+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8241a6b70 con 0x7fb824103120 2024-03-21T09:12:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.879+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82406a600 con 0x7fb824103120 2024-03-21T09:12:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.882+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fb810018a20 con 0x7fb824103120 2024-03-21T09:12:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.883+0000 7fb81be00640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8080a31d0 0x7fb8080a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.883+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb8100cfe50 con 0x7fb824103120 2024-03-21T09:12:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.884+0000 7fb823400640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8080a31d0 0x7fb8080a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:30.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.884+0000 7fb823400640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8080a31d0 0x7fb8080a5690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8140096c0 tx=0x7fb814009260 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:30.887+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb81009bd80 con 0x7fb824103120 2024-03-21T09:12:31.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.134+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb824106cc0 con 0x7fb824103120 2024-03-21T09:12:31.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.135+0000 7fb81be00640 1 -- 172.21.15.37:0/2104256385 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fb81009fe20 con 0x7fb824103120 2024-03-21T09:12:31.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:12:31.129 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:11:48.827236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:12:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8080a31d0 msgr2=0x7fb8080a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8080a31d0 0x7fb8080a5690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8140096c0 tx=0x7fb814009260 comp rx=0 tx=0).stop 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 msgr2=0x7fb824100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 0x7fb824100990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb81000e7a0 tx=0x7fb81000ec60 comp rx=0 tx=0).stop 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 shutdown_connections 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb82410bc20 0x7fb824100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8080a31d0 0x7fb8080a5690 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb824103120 0x7fb824100990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 --2- 172.21.15.37:0/2104256385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb824102800 0x7fb824100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 >> 172.21.15.37:0/2104256385 conn(0x7fb8240fc020 msgr2=0x7fb8240fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 shutdown_connections 2024-03-21T09:12:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.138+0000 7fb82a800640 1 -- 172.21.15.37:0/2104256385 wait complete. 2024-03-21T09:12:31.151 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:12:31.152 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 -- 172.21.15.37:0/3327592804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 msgr2=0x7faee0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 --2- 172.21.15.37:0/3327592804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0102a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faed4009900 tx=0x7faed402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 -- 172.21.15.37:0/3327592804 shutdown_connections 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 --2- 172.21.15.37:0/3327592804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee010ba50 0x7faee010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 --2- 172.21.15.37:0/3327592804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee0102f50 0x7faee010b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 --2- 172.21.15.37:0/3327592804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0102a10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 -- 172.21.15.37:0/3327592804 >> 172.21.15.37:0/3327592804 conn(0x7faee0077130 msgr2=0x7faee00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 -- 172.21.15.37:0/3327592804 shutdown_connections 2024-03-21T09:12:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.380+0000 7faee5e00640 1 -- 172.21.15.37:0/3327592804 wait complete. 2024-03-21T09:12:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.381+0000 7faee5e00640 1 Processor -- start 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 -- start start 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee0102f50 0x7faee019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee010ba50 0x7faee019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0111850 con 0x7faee010ba50 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01116d0 con 0x7faee0102630 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee5e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01119d0 con 0x7faee0102f50 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faedfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee0102f50 0x7faee019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59398/0 (socket says 172.21.15.37:59398) 2024-03-21T09:12:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.382+0000 7faee4a00640 1 -- 172.21.15.37:0/1001187415 learned_addr learned my addr 172.21.15.37:0/1001187415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faee5400640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee010ba50 0x7faee019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedfe00640 1 -- 172.21.15.37:0/1001187415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 msgr2=0x7faee0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedfe00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedfe00640 1 -- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee010ba50 msgr2=0x7faee019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedfe00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee010ba50 0x7faee019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedfe00640 1 -- 172.21.15.37:0/1001187415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee019ef60 con 0x7faee0102f50 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faee4a00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedfe00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee0102f50 0x7faee019a4b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faecc00e9f0 tx=0x7faecc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedd600640 1 -- 172.21.15.37:0/1001187415 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ccc0 con 0x7faee0102f50 2024-03-21T09:12:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee019f250 con 0x7faee0102f50 2024-03-21T09:12:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedd600640 1 -- 172.21.15.37:0/1001187415 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faecc0035f0 con 0x7faee0102f50 2024-03-21T09:12:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.383+0000 7faedd600640 1 -- 172.21.15.37:0/1001187415 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00ce60 con 0x7faee0102f50 2024-03-21T09:12:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.384+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01a6ae0 con 0x7faee0102f50 2024-03-21T09:12:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.386+0000 7faedd600640 1 -- 172.21.15.37:0/1001187415 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7faecc0037a0 con 0x7faee0102f50 2024-03-21T09:12:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.387+0000 7faeea04f740 1 Processor -- start 2024-03-21T09:12:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.387+0000 7faeea04f740 1 -- start start 2024-03-21T09:12:31.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faeea04f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x559801779c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faeea04f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801782810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faeea04f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x559801785230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faeea04f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016dd2c0 con 0x559801782de0 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faeea04f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801685c90 con 0x5598017798b0 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faeea04f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016d9360 con 0x55980177a260 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faee4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801782810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faedfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x559801785230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faee5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x559801779c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faee4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801782810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34408/0 (socket says 172.21.15.37:34408) 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.388+0000 7faee4a00640 1 -- 172.21.15.37:0/273988047 learned_addr learned my addr 172.21.15.37:0/273988047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee4a00640 1 -- 172.21.15.37:0/273988047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 msgr2=0x559801779c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee4a00640 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x559801779c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee4a00640 1 -- 172.21.15.37:0/273988047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 msgr2=0x559801785230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee4a00640 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x559801785230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee4a00640 1 -- 172.21.15.37:0/273988047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598016aa080 con 0x55980177a260 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee5400640 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x559801779c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faee4a00640 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801782810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faed4005c30 tx=0x7faed4009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc07c59a546102d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faed1c00640 1 -- 172.21.15.37:0/273988047 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4043070 con 0x55980177a260 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faed1c00640 1 -- 172.21.15.37:0/273988047 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faed4004a70 con 0x55980177a260 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.389+0000 7faed1c00640 1 -- 172.21.15.37:0/273988047 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed40051a0 con 0x55980177a260 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faedd600640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7faec80a3120 0x7faec80a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faedd600640 1 -- 172.21.15.37:0/1001187415 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7faecc0cff70 con 0x7faee0102f50 2024-03-21T09:12:31.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faee5400640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7faec80a3120 0x7faec80a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 -- 172.21.15.37:0/273988047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 msgr2=0x559801782810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801782810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faed4005c30 tx=0x7faed4009560 comp rx=0 tx=0).stop 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 -- 172.21.15.37:0/273988047 shutdown_connections 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x559801785230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801782810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 --2- 172.21.15.37:0/273988047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x559801779c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.390+0000 7faeea04f740 1 -- 172.21.15.37:0/273988047 >> 172.21.15.37:0/273988047 conn(0x559801773f80 msgr2=0x559801774360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.391+0000 7faeea04f740 1 -- 172.21.15.37:0/273988047 shutdown_connections 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.391+0000 7faeea04f740 1 -- 172.21.15.37:0/273988047 wait complete. 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.391+0000 7faee5400640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7faec80a3120 0x7faec80a55e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faed8005fd0 tx=0x7faed8009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.391+0000 7faeea04f740 1 -- start start 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.391+0000 7faeea04f740 1 Processor -- start 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598017798b0 0x5598017826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x55980177fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 0x55980177fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801785cf0 con 0x5598017798b0 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016990d0 con 0x55980177a260 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801785e70 con 0x559801782de0 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faeea04f740 10 client.0 _ll_register_callbacks cb 0x55980169de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faedfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x55980177fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faee5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 0x55980177fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faee5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 0x55980177fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34412/0 (socket says 172.21.15.37:34412) 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faedfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x55980177fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59428/0 (socket says 172.21.15.37:59428) 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faee5400640 1 -- 172.21.15.37:0/1568897886 learned_addr learned my addr 172.21.15.37:0/1568897886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.392+0000 7faee4a00640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598017798b0 0x5598017826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee5400640 1 -- 172.21.15.37:0/1568897886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 msgr2=0x55980177fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee5400640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x55980177fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee5400640 1 -- 172.21.15.37:0/1568897886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598017798b0 msgr2=0x5598017826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee5400640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598017798b0 0x5598017826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee5400640 1 -- 172.21.15.37:0/1568897886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598013f13f0 con 0x559801782de0 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faedfe00640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x55980177fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee4a00640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598017798b0 0x5598017826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faee5400640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 0x55980177fff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faed8005c20 tx=0x7faed8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d46925cf34e4257 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faec6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faec6000640 1 -- 172.21.15.37:0/1568897886 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800f7d0 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faed4004a70 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.393+0000 7faec6000640 1 -- 172.21.15.37:0/1568897886 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7faed800f970 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faec6000640 1 -- 172.21.15.37:0/1568897886 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faed800fb70 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 10 client.25708 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5598014bb7d0 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faec6000640 1 -- 172.21.15.37:0/1568897886 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faed4004a70 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faec6000640 1 -- 172.21.15.37:0/1568897886 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faed8027070 con 0x559801782de0 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 10 client.25708 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 10 client.25708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 20 client.25708 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:31.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x559801792be0 0x559801794ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 4 client.25708 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.394+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598016ea800 con 0x559801792be0 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.395+0000 7faee4a00640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x559801792be0 0x559801794ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.395+0000 7faec6000640 10 client.25708 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.395+0000 7faee4a00640 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x559801792be0 0x559801794ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d4bd75ac7223e32 server_cookie=a7115e79ae60342e in_seq=0 out_seq=0 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.396+0000 7faec6000640 1 -- 172.21.15.37:0/1568897886 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5598016ea800 con 0x559801792be0 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.396+0000 7faec6000640 10 client.25708 handle_command_reply: tid=0 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.396+0000 7faeea04f740 1 client.25708 shutdown 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 msgr2=0x55980177fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 0x55980177fff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faed8005c20 tx=0x7faed8004ab0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 shutdown_connections 2024-03-21T09:12:31.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559801782de0 0x55980177fff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x55980177fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x559801792be0 0x559801794ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 --2- 172.21.15.37:0/1568897886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598017798b0 0x5598017826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 >> 172.21.15.37:0/1568897886 conn(0x559801773f80 msgr2=0x559801775cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faec6000640 0 client.25708 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 shutdown_connections 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 1 -- 172.21.15.37:0/1568897886 wait complete. 2024-03-21T09:12:31.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.397+0000 7faeea04f740 20 client.25708 trim_cache size 0 max 16384 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 Processor -- start 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 -- start start 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 0x5598017821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x5598017826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x55980177ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01b2ee0 con 0x559801782de0 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01b09e0 con 0x55980177a260 2024-03-21T09:12:31.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faeea04f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01aea80 con 0x5598017798b0 2024-03-21T09:12:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faee4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 0x5598017821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faedfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x5598017826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faee4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 0x5598017821a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34420/0 (socket says 172.21.15.37:34420) 2024-03-21T09:12:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faee4a00640 1 -- 172.21.15.37:0/4003145536 learned_addr learned my addr 172.21.15.37:0/4003145536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.411+0000 7faee5400640 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x55980177ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faee4a00640 1 -- 172.21.15.37:0/4003145536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 msgr2=0x5598017826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faee4a00640 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x5598017826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faee4a00640 1 -- 172.21.15.37:0/4003145536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 msgr2=0x55980177ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faee4a00640 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x55980177ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faee4a00640 1 -- 172.21.15.37:0/4003145536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980177a000 con 0x5598017798b0 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faee4a00640 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 0x5598017821a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faed40095d0 tx=0x7faed402cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83d5214a2be49d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faed1200640 1 -- 172.21.15.37:0/4003145536 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4043040 con 0x5598017798b0 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faed1200640 1 -- 172.21.15.37:0/4003145536 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faed4004a70 con 0x5598017798b0 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.412+0000 7faed1200640 1 -- 172.21.15.37:0/4003145536 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4003850 con 0x5598017798b0 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 -- 172.21.15.37:0/4003145536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 msgr2=0x5598017821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 0x5598017821a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faed40095d0 tx=0x7faed402cc30 comp rx=0 tx=0).stop 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 -- 172.21.15.37:0/4003145536 shutdown_connections 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x55980177ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55980177a260 0x5598017826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 --2- 172.21.15.37:0/4003145536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5598017798b0 0x5598017821a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 -- 172.21.15.37:0/4003145536 >> 172.21.15.37:0/4003145536 conn(0x559801773f80 msgr2=0x55980177e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 -- 172.21.15.37:0/4003145536 shutdown_connections 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 -- 172.21.15.37:0/4003145536 wait complete. 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.413+0000 7faeea04f740 1 -- start start 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 Processor -- start 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x55980178aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801788090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x5598017885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01af080 con 0x559801782de0 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01b0380 con 0x5598017798b0 2024-03-21T09:12:31.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01af6c0 con 0x55980177a260 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faeea04f740 10 client.0 _ll_register_callbacks cb 0x7faed8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.414+0000 7faedfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801788090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faee4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x55980178aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801788090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34434/0 (socket says 172.21.15.37:34434) 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faee4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x55980178aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59450/0 (socket says 172.21.15.37:59450) 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 -- 172.21.15.37:0/978447504 learned_addr learned my addr 172.21.15.37:0/978447504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faee5400640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x5598017885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 -- 172.21.15.37:0/978447504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 msgr2=0x55980178aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x55980178aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 -- 172.21.15.37:0/978447504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 msgr2=0x5598017885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x5598017885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 -- 172.21.15.37:0/978447504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559801788c40 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faee4a00640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x55980178aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faedfe00640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801788090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faecc00e9c0 tx=0x7faecc09b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9ba9c762e3fe64 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faec6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.415+0000 7faec6000640 1 -- 172.21.15.37:0/978447504 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0cd6b0 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faec6000640 1 -- 172.21.15.37:0/978447504 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7faecc0bab60 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faed4004a70 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faec6000640 1 -- 172.21.15.37:0/978447504 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faecc009440 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 10 client.25714 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5598017ee380 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faec6000640 1 -- 172.21.15.37:0/978447504 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faed4004a70 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faec6000640 1 -- 172.21.15.37:0/978447504 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faecc09b240 con 0x55980177a260 2024-03-21T09:12:31.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 10 client.25714 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 10 client.25714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 20 client.25714 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5598017eea80 0x5598017f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 4 client.25714 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.416+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598016e4b50 con 0x5598017eea80 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faee4a00640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5598017eea80 0x5598017f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faee4a00640 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5598017eea80 0x5598017f0ea0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0cc6acf35c295f8 server_cookie=a328adaafaa368fb in_seq=0 out_seq=0 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faec6000640 10 client.25714 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faec6000640 1 -- 172.21.15.37:0/978447504 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598016e4b50 con 0x5598017eea80 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faec6000640 10 client.25714 handle_command_reply: tid=0 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faeea04f740 1 client.25714 shutdown 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.417+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 msgr2=0x559801788090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801788090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faecc00e9c0 tx=0x7faecc09b9f0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 shutdown_connections 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5598017eea80 0x5598017f0ea0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559801782de0 0x5598017885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55980177a260 0x559801788090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faec6000640 0 client.25714 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 --2- 172.21.15.37:0/978447504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5598017798b0 0x55980178aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 >> 172.21.15.37:0/978447504 conn(0x559801773f80 msgr2=0x5598017728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 shutdown_connections 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 1 -- 172.21.15.37:0/978447504 wait complete. 2024-03-21T09:12:31.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.418+0000 7faeea04f740 20 client.25714 trim_cache size 0 max 16384 2024-03-21T09:12:31.419 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7faec80a3120 msgr2=0x7faec80a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7faec80a3120 0x7faec80a55e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faed8005fd0 tx=0x7faed8009500 comp rx=0 tx=0).stop 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee0102f50 msgr2=0x7faee019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee0102f50 0x7faee019a4b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faecc00e9f0 tx=0x7faecc00edf0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 shutdown_connections 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faee010ba50 0x7faee019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faee0102f50 0x7faee019a4b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7faec80a3120 0x7faec80a55e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 --2- 172.21.15.37:0/1001187415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faee0102630 0x7faee0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 >> 172.21.15.37:0/1001187415 conn(0x7faee0077130 msgr2=0x7faee01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 shutdown_connections 2024-03-21T09:12:31.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.419+0000 7faee5e00640 1 -- 172.21.15.37:0/1001187415 wait complete. 2024-03-21T09:12:31.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:31 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2104256385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:12:31.431 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:12:31.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:12:31.503 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:31 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2104256385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 -- 172.21.15.37:0/2160038624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0102630 msgr2=0x7f31f0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 --2- 172.21.15.37:0/2160038624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0102630 0x7f31f0102a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009ca0 tx=0x7f31e002c190 comp rx=0 tx=0).stop 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 -- 172.21.15.37:0/2160038624 shutdown_connections 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 --2- 172.21.15.37:0/2160038624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f010ba50 0x7f31f010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 --2- 172.21.15.37:0/2160038624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f010b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 --2- 172.21.15.37:0/2160038624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0102630 0x7f31f0102a10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 -- 172.21.15.37:0/2160038624 >> 172.21.15.37:0/2160038624 conn(0x7f31f0077130 msgr2=0x7f31f00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 -- 172.21.15.37:0/2160038624 shutdown_connections 2024-03-21T09:12:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.657+0000 7f31f5200640 1 -- 172.21.15.37:0/2160038624 wait complete. 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.658+0000 7f31f5200640 1 Processor -- start 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.658+0000 7f31f5200640 1 -- start start 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31f5200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 0x7f31f0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31f5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31f5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f010ba50 0x7f31f019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31f5200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0111800 con 0x7f31f0102630 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31f5200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0111680 con 0x7f31f010ba50 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31f5200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0111980 con 0x7f31f0102f50 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 0x7f31f0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34446/0 (socket says 172.21.15.37:34446) 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 0x7f31f0199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42722/0 (socket says 172.21.15.37:42722) 2024-03-21T09:12:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31e6a00640 1 -- 172.21.15.37:0/1900916044 learned_addr learned my addr 172.21.15.37:0/1900916044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.659+0000 7f31efe00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f010ba50 0x7f31f019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31e6a00640 1 -- 172.21.15.37:0/1900916044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f010ba50 msgr2=0x7f31f019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31e6a00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f010ba50 0x7f31f019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31e6a00640 1 -- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 msgr2=0x7f31f0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31e6a00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 0x7f31f0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31e6a00640 1 -- 172.21.15.37:0/1900916044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f019ef40 con 0x7f31f0102f50 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31ef400640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 0x7f31f0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31e6a00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f019a490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31d800ea60 tx=0x7f31d800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31ecc00640 1 -- 172.21.15.37:0/1900916044 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800cde0 con 0x7f31f0102f50 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31ecc00640 1 -- 172.21.15.37:0/1900916044 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31d80035f0 con 0x7f31f0102f50 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f019f230 con 0x7f31f0102f50 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31ecc00640 1 -- 172.21.15.37:0/1900916044 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8010670 con 0x7f31f0102f50 2024-03-21T09:12:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.660+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f01a6ac0 con 0x7f31f0102f50 2024-03-21T09:12:31.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.663+0000 7f31ecc00640 1 -- 172.21.15.37:0/1900916044 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f31d80037a0 con 0x7f31f0102f50 2024-03-21T09:12:31.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 Processor -- start 2024-03-21T09:12:31.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 -- start start 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd5175c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd517e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd5181230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd50d92c0 con 0x55bcd51758b0 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd5081c90 con 0x55bcd517ede0 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.664+0000 7f31f959c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd50d5360 con 0x55bcd5176260 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd517e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd5181230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd5175c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd517e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34460/0 (socket says 172.21.15.37:34460) 2024-03-21T09:12:31.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 -- 172.21.15.37:0/1847264235 learned_addr learned my addr 172.21.15.37:0/1847264235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 -- 172.21.15.37:0/1847264235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 msgr2=0x55bcd5181230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd5181230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 -- 172.21.15.37:0/1847264235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 msgr2=0x55bcd5175c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd5175c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 -- 172.21.15.37:0/1847264235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcd50a6080 con 0x55bcd5176260 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.665+0000 7f31ef400640 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd517e810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f31e0037040 tx=0x7f31e00098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d64b9b34a24a26fa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.666+0000 7f31e4c00640 1 -- 172.21.15.37:0/1847264235 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0043070 con 0x55bcd5176260 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.666+0000 7f31e4c00640 1 -- 172.21.15.37:0/1847264235 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31e0003e50 con 0x55bcd5176260 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.666+0000 7f31e4c00640 1 -- 172.21.15.37:0/1847264235 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0039040 con 0x55bcd5176260 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.666+0000 7f31ecc00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f31d40a31a0 0x7f31d40a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.666+0000 7f31ecc00640 1 -- 172.21.15.37:0/1900916044 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f31d80cffd0 con 0x7f31f0102f50 2024-03-21T09:12:31.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.666+0000 7f31efe00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f31d40a31a0 0x7f31d40a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 -- 172.21.15.37:0/1847264235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 msgr2=0x55bcd517e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd517e810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f31e0037040 tx=0x7f31e00098f0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 -- 172.21.15.37:0/1847264235 shutdown_connections 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd5181230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd517e810 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 --2- 172.21.15.37:0/1847264235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd5175c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 -- 172.21.15.37:0/1847264235 >> 172.21.15.37:0/1847264235 conn(0x55bcd516ff80 msgr2=0x55bcd5170360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31efe00640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f31d40a31a0 0x7f31d40a5660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f31e800b6c0 tx=0x7f31e800b2e0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 -- 172.21.15.37:0/1847264235 shutdown_connections 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.667+0000 7f31f959c740 1 -- 172.21.15.37:0/1847264235 wait complete. 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 -- start start 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 Processor -- start 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 0x55bcd517e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd5176260 0x55bcd517baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd517c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd5181cf0 con 0x55bcd5176260 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd50950d0 con 0x55bcd517ede0 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd5181e70 con 0x55bcd51758b0 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.668+0000 7f31f959c740 10 client.0 _ll_register_callbacks cb 0x55bcd5099e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 0x55bcd517e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31efe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd517c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd5176260 0x55bcd517baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 0x55bcd517e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34476/0 (socket says 172.21.15.37:34476) 2024-03-21T09:12:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 -- 172.21.15.37:0/2986751490 learned_addr learned my addr 172.21.15.37:0/2986751490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 -- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 msgr2=0x55bcd517c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd517c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 -- 172.21.15.37:0/2986751490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd5176260 msgr2=0x55bcd517baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd5176260 0x55bcd517baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 -- 172.21.15.37:0/2986751490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcd4ded3f0 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31ef400640 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 0x55bcd517e760 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009bf0 tx=0x7f31e002c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29853fb2b01d6efc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.669+0000 7f31dd600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31dd600640 1 -- 172.21.15.37:0/2986751490 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e004c030 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31dd600640 1 -- 172.21.15.37:0/2986751490 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f31e00347f0 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31e0003e50 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31dd600640 1 -- 172.21.15.37:0/2986751490 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e0059050 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31f959c740 10 client.25726 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bcd4eb77d0 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.670+0000 7f31dd600640 1 -- 172.21.15.37:0/2986751490 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31e0003e50 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31dd600640 1 -- 172.21.15.37:0/2986751490 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f31e0032770 con 0x55bcd51758b0 2024-03-21T09:12:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31f959c740 10 client.25726 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31f959c740 10 client.25726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31f959c740 20 client.25726 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31f959c740 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd518eb10 0x55bcd5190f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31f959c740 4 client.25726 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcd50e6800 con 0x55bcd518eb10 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.671+0000 7f31e6a00640 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd518eb10 0x55bcd5190f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.672+0000 7f31e6a00640 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd518eb10 0x55bcd5190f00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adc1d8dc6c0444a9 server_cookie=5a0328c9a2cc989 in_seq=0 out_seq=0 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.672+0000 7f31dd600640 10 client.25726 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.672+0000 7f31dd600640 1 -- 172.21.15.37:0/2986751490 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bcd50e6800 con 0x55bcd518eb10 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.672+0000 7f31dd600640 10 client.25726 handle_command_reply: tid=0 2024-03-21T09:12:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.672+0000 7f31f959c740 1 client.25726 shutdown 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 msgr2=0x55bcd517e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 0x55bcd517e760 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009bf0 tx=0x7f31e002c190 comp rx=0 tx=0).stop 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 shutdown_connections 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd518eb10 0x55bcd5190f00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd517ede0 0x55bcd517c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd5176260 0x55bcd517baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 --2- 172.21.15.37:0/2986751490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd51758b0 0x55bcd517e760 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 >> 172.21.15.37:0/2986751490 conn(0x55bcd516ff80 msgr2=0x55bcd5171cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31dd600640 0 client.25726 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:12:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 shutdown_connections 2024-03-21T09:12:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 1 -- 172.21.15.37:0/2986751490 wait complete. 2024-03-21T09:12:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.673+0000 7f31f959c740 20 client.25726 trim_cache size 0 max 16384 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 Processor -- start 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 -- start start 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd518b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd5176260 0x55bcd5171ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd517ede0 0x55bcd517e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01b2ef0 con 0x55bcd51758b0 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01b09f0 con 0x55bcd5176260 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31f959c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01aea90 con 0x55bcd517ede0 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd517ede0 0x55bcd517e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd5176260 0x55bcd5171ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd517ede0 0x55bcd517e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34486/0 (socket says 172.21.15.37:34486) 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31efe00640 1 -- 172.21.15.37:0/211410520 learned_addr learned my addr 172.21.15.37:0/211410520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.687+0000 7f31ef400640 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd518b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31e6a00640 1 -- 172.21.15.37:0/211410520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd517ede0 msgr2=0x55bcd517e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31e6a00640 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd517ede0 0x55bcd517e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31e6a00640 1 -- 172.21.15.37:0/211410520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 msgr2=0x55bcd518b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31e6a00640 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd518b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31e6a00640 1 -- 172.21.15.37:0/211410520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcd5176000 con 0x55bcd5176260 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31e6a00640 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd5176260 0x55bcd5171ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31d80cd260 tx=0x7f31d80d2420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6f51e665180484 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31dfe00640 1 -- 172.21.15.37:0/211410520 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d809b990 con 0x55bcd5176260 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31dfe00640 1 -- 172.21.15.37:0/211410520 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31d809e030 con 0x55bcd5176260 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.688+0000 7f31dfe00640 1 -- 172.21.15.37:0/211410520 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d809b7b0 con 0x55bcd5176260 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 -- 172.21.15.37:0/211410520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd5176260 msgr2=0x55bcd5171ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd5176260 0x55bcd5171ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31d80cd260 tx=0x7f31d80d2420 comp rx=0 tx=0).stop 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 -- 172.21.15.37:0/211410520 shutdown_connections 2024-03-21T09:12:31.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd517ede0 0x55bcd517e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd5176260 0x55bcd5171ae0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 --2- 172.21.15.37:0/211410520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd51758b0 0x55bcd518b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 -- 172.21.15.37:0/211410520 >> 172.21.15.37:0/211410520 conn(0x55bcd516ff80 msgr2=0x55bcd51713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 -- 172.21.15.37:0/211410520 shutdown_connections 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 -- 172.21.15.37:0/211410520 wait complete. 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.689+0000 7f31f959c740 1 -- start start 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 Processor -- start 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 0x55bcd5191420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd5191960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd517ede0 0x55bcd5191ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01af090 con 0x55bcd517ede0 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01b0390 con 0x55bcd51758b0 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01af6d0 con 0x55bcd5176260 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31f959c740 10 client.0 _ll_register_callbacks cb 0x7f31e0034f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:31.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 0x55bcd5191420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd5191960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31ef400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 0x55bcd5191420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59522/0 (socket says 172.21.15.37:59522) 2024-03-21T09:12:31.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31ef400640 1 -- 172.21.15.37:0/2699709776 learned_addr learned my addr 172.21.15.37:0/2699709776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.690+0000 7f31efe00640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd517ede0 0x55bcd5191ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31ef400640 1 -- 172.21.15.37:0/2699709776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 msgr2=0x55bcd5191960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31ef400640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd5191960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31ef400640 1 -- 172.21.15.37:0/2699709776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd517ede0 msgr2=0x55bcd5191ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31ef400640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd517ede0 0x55bcd5191ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31ef400640 1 -- 172.21.15.37:0/2699709776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcd51e9110 con 0x55bcd51758b0 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31efe00640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd517ede0 0x55bcd5191ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31ef400640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 0x55bcd5191420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31e0032b40 tx=0x7f31e00329a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a4fe6187ddc06f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31dd600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31dd600640 1 -- 172.21.15.37:0/2699709776 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0056060 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31dd600640 1 -- 172.21.15.37:0/2699709776 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f31e0034a00 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31d809e030 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31dd600640 1 -- 172.21.15.37:0/2699709776 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e0063050 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.691+0000 7f31f959c740 10 client.25706 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bcd51ea410 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31dd600640 1 -- 172.21.15.37:0/2699709776 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31d809e030 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31dd600640 1 -- 172.21.15.37:0/2699709776 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f31e0048af0 con 0x55bcd51758b0 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 10 client.25706 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 10 client.25706 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 20 client.25706 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd51ea9c0 0x55bcd51ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 4 client.25706 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcd50e0b50 con 0x55bcd51ea9c0 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.692+0000 7f31e6a00640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd51ea9c0 0x55bcd51ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31e6a00640 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd51ea9c0 0x55bcd51ecde0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8303a0a4ba69e99 server_cookie=edccce088aa5ae04 in_seq=0 out_seq=0 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31dd600640 10 client.25706 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31dd600640 1 -- 172.21.15.37:0/2699709776 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcd50e0b50 con 0x55bcd51ea9c0 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31dd600640 10 client.25706 handle_command_reply: tid=0 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 client.25706 shutdown 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 msgr2=0x55bcd5191420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 0x55bcd5191420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31e0032b40 tx=0x7f31e00329a0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 shutdown_connections 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55bcd51ea9c0 0x55bcd51ecde0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcd517ede0 0x55bcd5191ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcd5176260 0x55bcd5191960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 --2- 172.21.15.37:0/2699709776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bcd51758b0 0x55bcd5191420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31dd600640 0 client.25706 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:12:31.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 >> 172.21.15.37:0/2699709776 conn(0x55bcd516ff80 msgr2=0x55bcd516f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 shutdown_connections 2024-03-21T09:12:31.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.693+0000 7f31f959c740 1 -- 172.21.15.37:0/2699709776 wait complete. 2024-03-21T09:12:31.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f959c740 20 client.25706 trim_cache size 0 max 16384 2024-03-21T09:12:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:12:31.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:31 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2104256385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f31d40a31a0 msgr2=0x7f31d40a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f31d40a31a0 0x7f31d40a5660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f31e800b6c0 tx=0x7f31e800b2e0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 msgr2=0x7f31f019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f019a490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31d800ea60 tx=0x7f31d800ef20 comp rx=0 tx=0).stop 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 shutdown_connections 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f31d40a31a0 0x7f31d40a5660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f010ba50 0x7f31f019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31f0102f50 0x7f31f019a490 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 --2- 172.21.15.37:0/1900916044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31f0102630 0x7f31f0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 >> 172.21.15.37:0/1900916044 conn(0x7f31f0077130 msgr2=0x7f31f00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 shutdown_connections 2024-03-21T09:12:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.694+0000 7f31f5200640 1 -- 172.21.15.37:0/1900916044 wait complete. 2024-03-21T09:12:31.707 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:12:31.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:12:31.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 -- 172.21.15.37:0/4128433572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 msgr2=0x7f3688102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 --2- 172.21.15.37:0/4128433572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f3688102ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f367c009ca0 tx=0x7f367c02c180 comp rx=0 tx=0).stop 2024-03-21T09:12:31.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 -- 172.21.15.37:0/4128433572 shutdown_connections 2024-03-21T09:12:31.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 --2- 172.21.15.37:0/4128433572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f368810bc90 0x7f368810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 --2- 172.21.15.37:0/4128433572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368810b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 --2- 172.21.15.37:0/4128433572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f3688102ca0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 -- 172.21.15.37:0/4128433572 >> 172.21.15.37:0/4128433572 conn(0x7f36880776c0 msgr2=0x7f3688077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 -- 172.21.15.37:0/4128433572 shutdown_connections 2024-03-21T09:12:31.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.906+0000 7f3690400640 1 -- 172.21.15.37:0/4128433572 wait complete. 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.907+0000 7f3690400640 1 Processor -- start 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 -- start start 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f368819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f368810bc90 0x7f368819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3688111a70 con 0x7f368810bc90 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881118f0 con 0x7f36881028c0 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f3690400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3688111bf0 con 0x7f36881031e0 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f368d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f368da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f368819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f368d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34522/0 (socket says 172.21.15.37:34522) 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f368d000640 1 -- 172.21.15.37:0/3861647999 learned_addr learned my addr 172.21.15.37:0/3861647999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.908+0000 7f368e400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f368810bc90 0x7f368819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368d000640 1 -- 172.21.15.37:0/3861647999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 msgr2=0x7f368819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368d000640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f368819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368d000640 1 -- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f368810bc90 msgr2=0x7f368819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368d000640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f368810bc90 0x7f368819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368d000640 1 -- 172.21.15.37:0/3861647999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368819f160 con 0x7f36881031e0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368da00640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f368819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f368d000640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368819a750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f367400ea60 tx=0x7f367400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f3686000640 1 -- 172.21.15.37:0/3861647999 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367400cde0 con 0x7f36881031e0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f3686000640 1 -- 172.21.15.37:0/3861647999 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36740035f0 con 0x7f36881031e0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f3686000640 1 -- 172.21.15.37:0/3861647999 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674010670 con 0x7f36881031e0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368819f450 con 0x7f36881031e0 2024-03-21T09:12:31.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.909+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36881a6ce0 con 0x7f36881031e0 2024-03-21T09:12:31.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.912+0000 7f3686000640 1 -- 172.21.15.37:0/3861647999 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f36740037a0 con 0x7f36881031e0 2024-03-21T09:12:31.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.912+0000 7f3693170740 1 Processor -- start 2024-03-21T09:12:31.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 -- start start 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x564175865ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x564175871050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641757c92c0 con 0x56417586ec00 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564175771c90 con 0x5641758656d0 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f3693170740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641757c5360 con 0x564175866080 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f368e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x564175865ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f368e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x564175865ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59538/0 (socket says 172.21.15.37:59538) 2024-03-21T09:12:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f368e400640 1 -- 172.21.15.37:0/177446646 learned_addr learned my addr 172.21.15.37:0/177446646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f368da00640 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.913+0000 7f368d000640 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x564175871050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368e400640 1 -- 172.21.15.37:0/177446646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 msgr2=0x56417586e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368e400640 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368e400640 1 -- 172.21.15.37:0/177446646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 msgr2=0x564175871050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368e400640 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x564175871050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368e400640 1 -- 172.21.15.37:0/177446646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564175796080 con 0x5641758656d0 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368da00640 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f3686000640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f36700a31e0 0x7f36700a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368e400640 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x564175865ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3680009b30 tx=0x7f36800113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c151452377e23be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f3686000640 1 -- 172.21.15.37:0/3861647999 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f36740d0090 con 0x7f36881031e0 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f368da00640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f36700a31e0 0x7f36700a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.914+0000 7f367aa00640 1 -- 172.21.15.37:0/177446646 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680011e10 con 0x5641758656d0 2024-03-21T09:12:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.915+0000 7f367aa00640 1 -- 172.21.15.37:0/177446646 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3680012930 con 0x5641758656d0 2024-03-21T09:12:31.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.915+0000 7f367aa00640 1 -- 172.21.15.37:0/177446646 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368000fa40 con 0x5641758656d0 2024-03-21T09:12:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.915+0000 7f3693170740 1 -- 172.21.15.37:0/177446646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 msgr2=0x564175865ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.915+0000 7f368da00640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f36700a31e0 0x7f36700a56a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f367c006010 tx=0x7f367c005ef0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.915+0000 7f3693170740 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x564175865ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3680009b30 tx=0x7f36800113e0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 -- 172.21.15.37:0/177446646 shutdown_connections 2024-03-21T09:12:31.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x564175871050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 --2- 172.21.15.37:0/177446646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x564175865ab0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 -- 172.21.15.37:0/177446646 >> 172.21.15.37:0/177446646 conn(0x56417585fda0 msgr2=0x564175860180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 -- 172.21.15.37:0/177446646 shutdown_connections 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 -- 172.21.15.37:0/177446646 wait complete. 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.916+0000 7f3693170740 1 -- start start 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 Processor -- start 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x56417586e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x56417586be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564175871b70 con 0x56417586ec00 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641757850d0 con 0x5641758656d0 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564175871cf0 con 0x564175866080 2024-03-21T09:12:31.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.917+0000 7f3693170740 10 client.0 _ll_register_callbacks cb 0x564175789e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x56417586e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x56417586e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59548/0 (socket says 172.21.15.37:59548) 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368da00640 1 -- 172.21.15.37:0/2840713070 learned_addr learned my addr 172.21.15.37:0/2840713070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368e400640 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x56417586be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 -- 172.21.15.37:0/2840713070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 msgr2=0x56417586e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x56417586e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 -- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 msgr2=0x56417586be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x56417586be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 -- 172.21.15.37:0/2840713070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641754dd3f0 con 0x564175866080 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f368d000640 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586b8a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36740cd320 tx=0x7f36740981b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7714a8cdfe82b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f366f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.918+0000 7f366f400640 1 -- 172.21.15.37:0/2840713070 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367409b870 con 0x564175866080 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f366f400640 1 -- 172.21.15.37:0/2840713070 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3674098580 con 0x564175866080 2024-03-21T09:12:31.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3680012930 con 0x564175866080 2024-03-21T09:12:31.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f366f400640 1 -- 172.21.15.37:0/2840713070 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f367409c610 con 0x564175866080 2024-03-21T09:12:31.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f3693170740 10 client.25744 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:31.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5641755a77d0 con 0x564175866080 2024-03-21T09:12:31.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f366f400640 1 -- 172.21.15.37:0/2840713070 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3680012930 con 0x564175866080 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f366f400640 1 -- 172.21.15.37:0/2840713070 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f36740cd5d0 con 0x564175866080 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f3693170740 10 client.25744 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.919+0000 7f3693170740 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.920+0000 7f3693170740 20 client.25744 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.920+0000 7f3693170740 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56417587ea80 0x564175880e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.920+0000 7f3693170740 4 client.25744 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.920+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641757d6800 con 0x56417587ea80 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.920+0000 7f368da00640 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56417587ea80 0x564175880e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.921+0000 7f368da00640 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56417587ea80 0x564175880e70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b48329b34c5459f server_cookie=fcdc8b7e7325e393 in_seq=0 out_seq=0 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.921+0000 7f366f400640 10 client.25744 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.921+0000 7f366f400640 1 -- 172.21.15.37:0/2840713070 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5641757d6800 con 0x56417587ea80 2024-03-21T09:12:31.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.921+0000 7f366f400640 10 client.25744 handle_command_reply: tid=0 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.921+0000 7f3693170740 1 client.25744 shutdown 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 msgr2=0x56417586b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586b8a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36740cd320 tx=0x7f36740981b0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 shutdown_connections 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56417586ec00 0x56417586be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564175866080 0x56417586b8a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56417587ea80 0x564175880e70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 --2- 172.21.15.37:0/2840713070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5641758656d0 0x56417586e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f366f400640 0 client.25744 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 >> 172.21.15.37:0/2840713070 conn(0x56417585fda0 msgr2=0x564175861af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 shutdown_connections 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 1 -- 172.21.15.37:0/2840713070 wait complete. 2024-03-21T09:12:31.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.922+0000 7f3693170740 20 client.25744 trim_cache size 0 max 16384 2024-03-21T09:12:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 Processor -- start 2024-03-21T09:12:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 -- start start 2024-03-21T09:12:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 0x56417586dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564175866080 0x56417586e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 0x564175868b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881b31f0 con 0x564175866080 2024-03-21T09:12:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881b0cf0 con 0x56417586ec00 2024-03-21T09:12:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f3693170740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881aed90 con 0x5641758656d0 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f368e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 0x564175868b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f368d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564175866080 0x56417586e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f368da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 0x56417586dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f368e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 0x564175868b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59554/0 (socket says 172.21.15.37:59554) 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f368da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 0x56417586dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34542/0 (socket says 172.21.15.37:34542) 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.936+0000 7f368e400640 1 -- 172.21.15.37:0/545838832 learned_addr learned my addr 172.21.15.37:0/545838832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368da00640 1 -- 172.21.15.37:0/545838832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 msgr2=0x564175868b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368da00640 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 0x564175868b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368da00640 1 -- 172.21.15.37:0/545838832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564175866080 msgr2=0x56417586e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368da00640 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564175866080 0x56417586e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368da00640 1 -- 172.21.15.37:0/545838832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564175867470 con 0x5641758656d0 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368e400640 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 0x564175868b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f368da00640 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 0x56417586dfd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5641758816c0 tx=0x7f367c041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ab0fc42086a43e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f367a000640 1 -- 172.21.15.37:0/545838832 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c039040 con 0x5641758656d0 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f367a000640 1 -- 172.21.15.37:0/545838832 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f367c004040 con 0x5641758656d0 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.937+0000 7f367a000640 1 -- 172.21.15.37:0/545838832 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c032670 con 0x5641758656d0 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 -- 172.21.15.37:0/545838832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 msgr2=0x56417586dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 0x56417586dfd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5641758816c0 tx=0x7f367c041000 comp rx=0 tx=0).stop 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 -- 172.21.15.37:0/545838832 shutdown_connections 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56417586ec00 0x564175868b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564175866080 0x56417586e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 --2- 172.21.15.37:0/545838832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641758656d0 0x56417586dfd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 -- 172.21.15.37:0/545838832 >> 172.21.15.37:0/545838832 conn(0x56417585fda0 msgr2=0x56417585eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 -- 172.21.15.37:0/545838832 shutdown_connections 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 -- 172.21.15.37:0/545838832 wait complete. 2024-03-21T09:12:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.938+0000 7f3693170740 1 -- start start 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 Processor -- start 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5641758656d0 0x5641758766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 0x564175876c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56417586ec00 0x564175874af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881af390 con 0x5641758656d0 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881b0690 con 0x564175866080 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881af9d0 con 0x56417586ec00 2024-03-21T09:12:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f3693170740 10 client.0 _ll_register_callbacks cb 0x7f36740cd1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f368da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5641758656d0 0x5641758766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f368d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 0x564175876c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f368e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56417586ec00 0x564175874af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f368d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 0x564175876c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59570/0 (socket says 172.21.15.37:59570) 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.939+0000 7f368d000640 1 -- 172.21.15.37:0/2003793291 learned_addr learned my addr 172.21.15.37:0/2003793291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368e400640 1 -- 172.21.15.37:0/2003793291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 msgr2=0x564175876c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368e400640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 0x564175876c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368e400640 1 -- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5641758656d0 msgr2=0x5641758766c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368e400640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5641758656d0 0x5641758766c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368e400640 1 -- 172.21.15.37:0/2003793291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641758752f0 con 0x56417586ec00 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368d000640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 0x564175876c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368da00640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5641758656d0 0x5641758766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f368e400640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56417586ec00 0x564175874af0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ad0 tx=0x7f3680011ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d43ff00e12a057 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f366f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.940+0000 7f366f400640 1 -- 172.21.15.37:0/2003793291 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680028070 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f366f400640 1 -- 172.21.15.37:0/2003793291 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f36800054c0 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f367c004040 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f366f400640 1 -- 172.21.15.37:0/2003793291 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36800056c0 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 10 client.25750 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5641758dad80 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f366f400640 1 -- 172.21.15.37:0/2003793291 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f367c004040 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f366f400640 1 -- 172.21.15.37:0/2003793291 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f368000fa70 con 0x56417586ec00 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 10 client.25750 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 20 client.25750 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5641758db120 0x5641758dd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 4 client.25750 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.941+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641757d0b50 con 0x5641758db120 2024-03-21T09:12:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.942+0000 7f368da00640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5641758db120 0x5641758dd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.942+0000 7f368da00640 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5641758db120 0x5641758dd510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffbe5a48554e7b1 server_cookie=3f1b344b29a0845b in_seq=0 out_seq=0 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.942+0000 7f366f400640 10 client.25750 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f366f400640 1 -- 172.21.15.37:0/2003793291 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641757d0b50 con 0x5641758db120 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f366f400640 10 client.25750 handle_command_reply: tid=0 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 client.25750 shutdown 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56417586ec00 msgr2=0x564175874af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56417586ec00 0x564175874af0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ad0 tx=0x7f3680011ed0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 shutdown_connections 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56417586ec00 0x564175874af0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5641758db120 0x5641758dd510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564175866080 0x564175876c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 --2- 172.21.15.37:0/2003793291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5641758656d0 0x5641758766c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f366f400640 0 client.25750 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 >> 172.21.15.37:0/2003793291 conn(0x56417585fda0 msgr2=0x56417585e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 shutdown_connections 2024-03-21T09:12:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 1 -- 172.21.15.37:0/2003793291 wait complete. 2024-03-21T09:12:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.943+0000 7f3693170740 20 client.25750 trim_cache size 0 max 16384 2024-03-21T09:12:31.942 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f36700a31e0 msgr2=0x7f36700a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f36700a31e0 0x7f36700a56a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f367c006010 tx=0x7f367c005ef0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 msgr2=0x7f368819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368819a750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f367400ea60 tx=0x7f367400ef20 comp rx=0 tx=0).stop 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 shutdown_connections 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f368810bc90 0x7f368819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881031e0 0x7f368819a750 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f36700a31e0 0x7f36700a56a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 --2- 172.21.15.37:0/3861647999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36881028c0 0x7f368819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 >> 172.21.15.37:0/3861647999 conn(0x7f36880776c0 msgr2=0x7f3688107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 shutdown_connections 2024-03-21T09:12:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:31.944+0000 7f3690400640 1 -- 172.21.15.37:0/3861647999 wait complete. 2024-03-21T09:12:32.517 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:32 smithi071 ceph-mon[36597]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:12:32.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:32 smithi037 ceph-mon[31830]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:12:32.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:32 smithi053 ceph-mon[36169]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:12:32.957 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:12:32.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:12:32.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:12:33.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.230+0000 7f2a45000640 1 -- 172.21.15.37:0/3864998149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40105820 msgr2=0x7f2a40105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.230+0000 7f2a45000640 1 --2- 172.21.15.37:0/3864998149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40105820 0x7f2a40105c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009ca0 tx=0x7f2a3802c190 comp rx=0 tx=0).stop 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 -- 172.21.15.37:0/3864998149 shutdown_connections 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 --2- 172.21.15.37:0/3864998149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40105820 0x7f2a40105c90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 --2- 172.21.15.37:0/3864998149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 0x7f2a40105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 --2- 172.21.15.37:0/3864998149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40100420 0x7f2a40100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 -- 172.21.15.37:0/3864998149 >> 172.21.15.37:0/3864998149 conn(0x7f2a40077130 msgr2=0x7f2a400fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 -- 172.21.15.37:0/3864998149 shutdown_connections 2024-03-21T09:12:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.231+0000 7f2a45000640 1 -- 172.21.15.37:0/3864998149 wait complete. 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 Processor -- start 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 -- start start 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 0x7f2a4019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 0x7f2a4019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40105820 0x7f2a401a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40117e30 con 0x7f2a40100d40 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40117cb0 con 0x7f2a40100420 2024-03-21T09:12:33.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a45000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40117fb0 con 0x7f2a40105820 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 0x7f2a4019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 0x7f2a4019e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59584/0 (socket says 172.21.15.37:59584) 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a3f400640 1 -- 172.21.15.37:0/307024961 learned_addr learned my addr 172.21.15.37:0/307024961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40105820 0x7f2a401a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.232+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 0x7f2a4019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3f400640 1 -- 172.21.15.37:0/307024961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40105820 msgr2=0x7f2a401a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3f400640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40105820 0x7f2a401a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3f400640 1 -- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 msgr2=0x7f2a4019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3f400640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 0x7f2a4019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3f400640 1 -- 172.21.15.37:0/307024961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a401a3280 con 0x7f2a40100420 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 0x7f2a4019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40105820 0x7f2a401a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a3f400640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 0x7f2a4019e290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000ea60 tx=0x7f2a3000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a37e00640 1 -- 172.21.15.37:0/307024961 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000cde0 con 0x7f2a40100420 2024-03-21T09:12:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a37e00640 1 -- 172.21.15.37:0/307024961 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a300035f0 con 0x7f2a40100420 2024-03-21T09:12:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a401a3570 con 0x7f2a40100420 2024-03-21T09:12:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.233+0000 7f2a37e00640 1 -- 172.21.15.37:0/307024961 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30010670 con 0x7f2a40100420 2024-03-21T09:12:33.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.234+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401aae00 con 0x7f2a40100420 2024-03-21T09:12:33.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.236+0000 7f2a37e00640 1 -- 172.21.15.37:0/307024961 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f2a300037a0 con 0x7f2a40100420 2024-03-21T09:12:33.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.237+0000 7f2a493db740 1 Processor -- start 2024-03-21T09:12:33.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.237+0000 7f2a493db740 1 -- start start 2024-03-21T09:12:33.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a493db740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537290 0x55e165537670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a493db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165540250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a493db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165542b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a493db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16549beb0 con 0x55e165540790 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a493db740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e165444a60 con 0x55e165537290 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a493db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e165497f50 con 0x55e165537bb0 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a37e00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2a240a31a0 0x7f2a240a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165540250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165540250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34582/0 (socket says 172.21.15.37:34582) 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a3f400640 1 -- 172.21.15.37:0/308403110 learned_addr learned my addr 172.21.15.37:0/308403110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537290 0x55e165537670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165542b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a37e00640 1 -- 172.21.15.37:0/307024961 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2a300d0080 con 0x7f2a40100420 2024-03-21T09:12:33.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.238+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2a240a31a0 0x7f2a240a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3f400640 1 -- 172.21.15.37:0/308403110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537290 msgr2=0x55e165537670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3f400640 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537290 0x55e165537670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3f400640 1 -- 172.21.15.37:0/308403110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 msgr2=0x55e165542b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3f400640 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165542b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3f400640 1 -- 172.21.15.37:0/308403110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1652787e0 con 0x55e165537bb0 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537290 0x55e165537670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165542b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a3f400640 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165540250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a300988d0 tx=0x7f2a3006ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b96d70ca9ceb33 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a34c00640 1 -- 172.21.15.37:0/308403110 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30066e60 con 0x55e165537bb0 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a34c00640 1 -- 172.21.15.37:0/308403110 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a3009bf00 con 0x55e165537bb0 2024-03-21T09:12:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.239+0000 7f2a34c00640 1 -- 172.21.15.37:0/308403110 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300a4480 con 0x55e165537bb0 2024-03-21T09:12:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 -- 172.21.15.37:0/308403110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 msgr2=0x55e165540250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2a240a31a0 0x7f2a240a5660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005e30 tx=0x7f2a38005d80 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165540250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a300988d0 tx=0x7f2a3006ebe0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 -- 172.21.15.37:0/308403110 shutdown_connections 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165542b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165540250 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 --2- 172.21.15.37:0/308403110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537290 0x55e165537670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 -- 172.21.15.37:0/308403110 >> 172.21.15.37:0/308403110 conn(0x55e1655319c0 msgr2=0x55e165531da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 -- 172.21.15.37:0/308403110 shutdown_connections 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.240+0000 7f2a493db740 1 -- 172.21.15.37:0/308403110 wait complete. 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 -- start start 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 Processor -- start 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165540130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e16553d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e16553da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1655434c0 con 0x55e165537290 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e165457eb0 con 0x55e165540790 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e165543640 con 0x55e165537bb0 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.241+0000 7f2a493db740 10 client.0 _ll_register_callbacks cb 0x55e16544c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e16553d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165540130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e16553da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e16553d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34596/0 (socket says 172.21.15.37:34596) 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 -- 172.21.15.37:0/1562118082 learned_addr learned my addr 172.21.15.37:0/1562118082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 -- 172.21.15.37:0/1562118082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 msgr2=0x55e16553da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e16553da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 -- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 msgr2=0x55e165540130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165540130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 -- 172.21.15.37:0/1562118082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1652d8f70 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e16553d4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c013790 tx=0x7f2a2c013c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c0dfdd944f31265 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.242+0000 7f2a29600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a29600640 1 -- 172.21.15.37:0/1562118082 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c020060 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a3009bf00 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a29600640 1 -- 172.21.15.37:0/1562118082 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2a2c011dd0 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a29600640 1 -- 172.21.15.37:0/1562118082 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2c027050 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a493db740 10 client.25762 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e165170170 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a29600640 1 -- 172.21.15.37:0/1562118082 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a3009bf00 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.243+0000 7f2a29600640 1 -- 172.21.15.37:0/1562118082 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2a2c01a030 con 0x55e165537bb0 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a493db740 10 client.25762 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a493db740 10 client.25762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a493db740 20 client.25762 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a493db740 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e165550790 0x55e165552b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a493db740 4 client.25762 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:33.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1654a93f0 con 0x55e165550790 2024-03-21T09:12:33.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a3f400640 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e165550790 0x55e165552b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a3f400640 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e165550790 0x55e165552b80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31727e2de1e19a2 server_cookie=87913029fb9fc0be in_seq=0 out_seq=0 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.244+0000 7f2a29600640 10 client.25762 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.245+0000 7f2a29600640 1 -- 172.21.15.37:0/1562118082 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e1654a93f0 con 0x55e165550790 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.245+0000 7f2a29600640 10 client.25762 handle_command_reply: tid=0 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.245+0000 7f2a493db740 1 client.25762 shutdown 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.245+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 msgr2=0x55e16553d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.245+0000 7f2a493db740 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e16553d4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c013790 tx=0x7f2a2c013c50 comp rx=0 tx=0).stop 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 shutdown_connections 2024-03-21T09:12:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e165550790 0x55e165552b80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e16553da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e16553d4c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 --2- 172.21.15.37:0/1562118082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165540130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a29600640 0 client.25762 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 >> 172.21.15.37:0/1562118082 conn(0x55e1655319c0 msgr2=0x55e165533710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 shutdown_connections 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 1 -- 172.21.15.37:0/1562118082 wait complete. 2024-03-21T09:12:33.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.246+0000 7f2a493db740 20 client.25762 trim_cache size 0 max 16384 2024-03-21T09:12:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 Processor -- start 2024-03-21T09:12:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 -- start start 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 0x55e16553fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 0x55e165540130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165538700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401b72f0 con 0x55e165540790 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401b4df0 con 0x55e165537bb0 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.264+0000 7f2a493db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401b2e90 con 0x55e165537290 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 0x55e165540130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 0x55e16553fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 0x55e165540130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59610/0 (socket says 172.21.15.37:59610) 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 -- 172.21.15.37:0/4033555676 learned_addr learned my addr 172.21.15.37:0/4033555676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 0x55e16553fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34600/0 (socket says 172.21.15.37:34600) 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165538700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 -- 172.21.15.37:0/4033555676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 msgr2=0x55e16553fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 0x55e16553fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 -- 172.21.15.37:0/4033555676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 msgr2=0x55e165538700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165538700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 -- 172.21.15.37:0/4033555676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1655308d0 con 0x55e165537bb0 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3f400640 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 0x55e16553fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165538700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 0x55e165540130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c000c00 tx=0x7f2a2c009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e683c00a0d742ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.265+0000 7f2a2be00640 1 -- 172.21.15.37:0/4033555676 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c030070 con 0x55e165537bb0 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a2be00640 1 -- 172.21.15.37:0/4033555676 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a2c0055f0 con 0x55e165537bb0 2024-03-21T09:12:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a2be00640 1 -- 172.21.15.37:0/4033555676 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c000e60 con 0x55e165537bb0 2024-03-21T09:12:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 -- 172.21.15.37:0/4033555676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 msgr2=0x55e165540130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 0x55e165540130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c000c00 tx=0x7f2a2c009530 comp rx=0 tx=0).stop 2024-03-21T09:12:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 -- 172.21.15.37:0/4033555676 shutdown_connections 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165540790 0x55e165538700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165537bb0 0x55e165540130 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 --2- 172.21.15.37:0/4033555676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537290 0x55e16553fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.266+0000 7f2a493db740 1 -- 172.21.15.37:0/4033555676 >> 172.21.15.37:0/4033555676 conn(0x55e1655319c0 msgr2=0x55e165537ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.267+0000 7f2a493db740 1 -- 172.21.15.37:0/4033555676 shutdown_connections 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.267+0000 7f2a493db740 1 -- 172.21.15.37:0/4033555676 wait complete. 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.267+0000 7f2a493db740 1 -- start start 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.267+0000 7f2a493db740 1 Processor -- start 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165544ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165545420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e165545960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401b3490 con 0x55e165537290 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401b4790 con 0x55e165540790 2024-03-21T09:12:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401b3ad0 con 0x55e165537bb0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a493db740 10 client.0 _ll_register_callbacks cb 0x7f2a2c009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165544ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a3f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165544ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42880/0 (socket says 172.21.15.37:42880) 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a3f400640 1 -- 172.21.15.37:0/2337151965 learned_addr learned my addr 172.21.15.37:0/2337151965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165545420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.268+0000 7f2a3fe00640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e165545960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3ea00640 1 -- 172.21.15.37:0/2337151965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 msgr2=0x55e165545960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e165545960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3ea00640 1 -- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 msgr2=0x55e165544ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165544ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3ea00640 1 -- 172.21.15.37:0/2337151965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1655ac270 con 0x55e165537bb0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3f400640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165544ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a3ea00640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165545420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e6e0 tx=0x7f2a2c00eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5474bb2225b62b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a29600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a29600640 1 -- 172.21.15.37:0/2337151965 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c030040 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a2c0055f0 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.269+0000 7f2a29600640 1 -- 172.21.15.37:0/2337151965 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2a2c00ec40 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a29600640 1 -- 172.21.15.37:0/2337151965 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2c000e60 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 10 client.25768 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e1655ad570 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a29600640 1 -- 172.21.15.37:0/2337151965 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a2c0055f0 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a29600640 1 -- 172.21.15.37:0/2337151965 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2a2c033070 con 0x55e165537bb0 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 10 client.25768 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 20 client.25768 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1655adb20 0x55e1655aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 4 client.25768 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:12:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e1654a3740 con 0x55e1655adb20 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.270+0000 7f2a3f400640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1655adb20 0x55e1655aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.271+0000 7f2a3f400640 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1655adb20 0x55e1655aff10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5592a6bac89a76 server_cookie=f81a471ba9ffb2b9 in_seq=0 out_seq=0 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.271+0000 7f2a29600640 10 client.25768 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.272+0000 7f2a29600640 1 -- 172.21.15.37:0/2337151965 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e1654a3740 con 0x55e1655adb20 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a29600640 10 client.25768 handle_command_reply: tid=0 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 client.25768 shutdown 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 msgr2=0x55e165545420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165545420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e6e0 tx=0x7f2a2c00eb00 comp rx=0 tx=0).stop 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 shutdown_connections 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1655adb20 0x55e1655aff10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e165540790 0x55e165545960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e165537bb0 0x55e165545420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 --2- 172.21.15.37:0/2337151965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e165537290 0x55e165544ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 >> 172.21.15.37:0/2337151965 conn(0x55e1655319c0 msgr2=0x55e165538820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a29600640 0 client.25768 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 shutdown_connections 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.273+0000 7f2a493db740 1 -- 172.21.15.37:0/2337151965 wait complete. 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.274+0000 7f2a493db740 20 client.25768 trim_cache size 0 max 16384 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:12:33.270 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "5336d1b3-64a2-4ebd-b37d-264806b2ce31", 2024-03-21T09:12:33.271 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:12:33.271 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:12:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.274+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2a240a31a0 msgr2=0x7f2a240a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.274+0000 7f2a45000640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2a240a31a0 0x7f2a240a5660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005e30 tx=0x7f2a38005d80 comp rx=0 tx=0).stop 2024-03-21T09:12:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.274+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 msgr2=0x7f2a4019e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.274+0000 7f2a45000640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 0x7f2a4019e290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000ea60 tx=0x7f2a3000ef20 comp rx=0 tx=0).stop 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 shutdown_connections 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40105820 0x7f2a401a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a40100d40 0x7f2a4019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2a240a31a0 0x7f2a240a5660 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 --2- 172.21.15.37:0/307024961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a40100420 0x7f2a4019e290 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 >> 172.21.15.37:0/307024961 conn(0x7f2a40077130 msgr2=0x7f2a401031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 shutdown_connections 2024-03-21T09:12:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.275+0000 7f2a45000640 1 -- 172.21.15.37:0/307024961 wait complete. 2024-03-21T09:12:33.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:12:33.288 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 -- 172.21.15.37:0/3160664630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 msgr2=0x7fcf4410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3160664630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf4410ded0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009ca0 tx=0x7fcf3c02c180 comp rx=0 tx=0).stop 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 -- 172.21.15.37:0/3160664630 shutdown_connections 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3160664630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf4410ded0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3160664630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102f50 0x7fcf4410b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3160664630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102630 0x7fcf44102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 -- 172.21.15.37:0/3160664630 >> 172.21.15.37:0/3160664630 conn(0x7fcf44077130 msgr2=0x7fcf440fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 -- 172.21.15.37:0/3160664630 shutdown_connections 2024-03-21T09:12:33.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.496+0000 7fcf43e00640 1 -- 172.21.15.37:0/3160664630 wait complete. 2024-03-21T09:12:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.497+0000 7fcf43e00640 1 Processor -- start 2024-03-21T09:12:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.497+0000 7fcf43e00640 1 -- start start 2024-03-21T09:12:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102630 0x7fcf4419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 0x7fcf4419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf44197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44111810 con 0x7fcf44102630 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44111690 con 0x7fcf4410ba50 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44111990 con 0x7fcf44102f50 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102630 0x7fcf4419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf42000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 0x7fcf4419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf44197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf42000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 0x7fcf4419de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34620/0 (socket says 172.21.15.37:34620) 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf43400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf44197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59644/0 (socket says 172.21.15.37:59644) 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf42000640 1 -- 172.21.15.37:0/3843860320 learned_addr learned my addr 172.21.15.37:0/3843860320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102630 0x7fcf4419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42886/0 (socket says 172.21.15.37:42886) 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.498+0000 7fcf42000640 1 -- 172.21.15.37:0/3843860320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 msgr2=0x7fcf44197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf42000640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf44197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf42000640 1 -- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102630 msgr2=0x7fcf4419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf42000640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102630 0x7fcf4419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf42000640 1 -- 172.21.15.37:0/3843860320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf44198290 con 0x7fcf44102f50 2024-03-21T09:12:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf43400640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf44197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf42000640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 0x7fcf4419de50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000d950 tx=0x7fcf3000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf3b400640 1 -- 172.21.15.37:0/3843860320 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30017070 con 0x7fcf44102f50 2024-03-21T09:12:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf3b400640 1 -- 172.21.15.37:0/3843860320 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf30009c30 con 0x7fcf44102f50 2024-03-21T09:12:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf3b400640 1 -- 172.21.15.37:0/3843860320 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000f6b0 con 0x7fcf44102f50 2024-03-21T09:12:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf44198580 con 0x7fcf44102f50 2024-03-21T09:12:33.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.499+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf441a47d0 con 0x7fcf44102f50 2024-03-21T09:12:33.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 Processor -- start 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 -- start start 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf3b400640 1 -- 172.21.15.37:0/3843860320 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fcf300109d0 con 0x7fcf44102f50 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed47db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47e0f0 0x55e1ed47e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ed3e2250 con 0x55e1ed48b870 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ed38aa30 con 0x55e1ed47edd0 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.503+0000 7fcf4c644740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ed3de2f0 con 0x55e1ed47e0f0 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed47db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed47db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59652/0 (socket says 172.21.15.37:59652) 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 -- 172.21.15.37:0/786743322 learned_addr learned my addr 172.21.15.37:0/786743322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf42a00640 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47e0f0 0x55e1ed47e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf42000640 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 -- 172.21.15.37:0/786743322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47e0f0 msgr2=0x55e1ed47e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47e0f0 0x55e1ed47e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 -- 172.21.15.37:0/786743322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 msgr2=0x55e1ed48dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 -- 172.21.15.37:0/786743322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1ed375690 con 0x55e1ed47edd0 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.504+0000 7fcf43400640 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed47db20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c02c790 tx=0x7fcf3c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fecc0e761269bd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.505+0000 7fcf2fe00640 1 -- 172.21.15.37:0/786743322 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c043070 con 0x55e1ed47edd0 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.505+0000 7fcf2fe00640 1 -- 172.21.15.37:0/786743322 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c003e50 con 0x55e1ed47edd0 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.505+0000 7fcf2fe00640 1 -- 172.21.15.37:0/786743322 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c039040 con 0x55e1ed47edd0 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.505+0000 7fcf3b400640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf280a31a0 0x7fcf280a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.505+0000 7fcf4c644740 1 -- 172.21.15.37:0/786743322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 msgr2=0x55e1ed47db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.505+0000 7fcf4c644740 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed47db20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c02c790 tx=0x7fcf3c005f80 comp rx=0 tx=0).stop 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf3b400640 1 -- 172.21.15.37:0/3843860320 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcf300cf810 con 0x7fcf44102f50 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf42a00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf280a31a0 0x7fcf280a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 -- 172.21.15.37:0/786743322 shutdown_connections 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47e0f0 0x55e1ed47e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 --2- 172.21.15.37:0/786743322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed47db20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 -- 172.21.15.37:0/786743322 >> 172.21.15.37:0/786743322 conn(0x55e1ed445200 msgr2=0x55e1ed478b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 -- 172.21.15.37:0/786743322 shutdown_connections 2024-03-21T09:12:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf4c644740 1 -- 172.21.15.37:0/786743322 wait complete. 2024-03-21T09:12:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.506+0000 7fcf42a00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf280a31a0 0x7fcf280a5660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x55e1ed47f320 tx=0x7fcf34005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.507+0000 7fcf4c644740 1 -- start start 2024-03-21T09:12:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.507+0000 7fcf4c644740 1 Processor -- start 2024-03-21T09:12:33.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed4882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed488820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ed48e680 con 0x55e1ed47e0f0 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ed39de80 con 0x55e1ed47edd0 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ed48e800 con 0x55e1ed48b870 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf4c644740 10 client.0 _ll_register_callbacks cb 0x55e1ed3b0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed4882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed4882e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59654/0 (socket says 172.21.15.37:59654) 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 -- 172.21.15.37:0/1607493636 learned_addr learned my addr 172.21.15.37:0/1607493636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf43400640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed488820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 -- 172.21.15.37:0/1607493636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 msgr2=0x55e1ed488820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed488820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 -- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47e0f0 msgr2=0x55e1ed48af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf42000640 1 -- 172.21.15.37:0/1607493636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1ed1c07d0 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.508+0000 7fcf43400640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed488820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf42000640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed4882e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf30010fd0 tx=0x7fcf3009b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763d8c2fac88d768 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf2cc00640 1 -- 172.21.15.37:0/1607493636 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30020050 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf2cc00640 1 -- 172.21.15.37:0/1607493636 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcf30097db0 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf3c003e50 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf2cc00640 1 -- 172.21.15.37:0/1607493636 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf300b4050 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf4c644740 10 client.25754 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e1ed0f63f0 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.509+0000 7fcf2cc00640 1 -- 172.21.15.37:0/1607493636 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c003e50 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf2cc00640 1 -- 172.21.15.37:0/1607493636 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcf300afce0 con 0x55e1ed47edd0 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf4c644740 10 client.25754 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf4c644740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf4c644740 20 client.25754 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf4c644740 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed49b6a0 0x55e1ed49da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf4c644740 4 client.25754 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1ed3ef790 con 0x55e1ed49b6a0 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.510+0000 7fcf42a00640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed49b6a0 0x55e1ed49da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.511+0000 7fcf42a00640 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed49b6a0 0x55e1ed49da90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be1d36afe7938f5 server_cookie=dc8e50dc948afd7 in_seq=0 out_seq=0 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.511+0000 7fcf2cc00640 10 client.25754 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.511+0000 7fcf2cc00640 1 -- 172.21.15.37:0/1607493636 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e1ed3ef790 con 0x55e1ed49b6a0 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.511+0000 7fcf2cc00640 10 client.25754 handle_command_reply: tid=0 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.511+0000 7fcf4c644740 1 client.25754 shutdown 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 msgr2=0x55e1ed4882e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed4882e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf30010fd0 tx=0x7fcf3009b280 comp rx=0 tx=0).stop 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 shutdown_connections 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed488820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed49b6a0 0x55e1ed49da90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47edd0 0x55e1ed4882e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 --2- 172.21.15.37:0/1607493636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf2cc00640 0 client.25754 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 >> 172.21.15.37:0/1607493636 conn(0x55e1ed445200 msgr2=0x55e1ed47a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 shutdown_connections 2024-03-21T09:12:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 1 -- 172.21.15.37:0/1607493636 wait complete. 2024-03-21T09:12:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.512+0000 7fcf4c644740 20 client.25754 trim_cache size 0 max 16384 2024-03-21T09:12:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 Processor -- start 2024-03-21T09:12:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 -- start start 2024-03-21T09:12:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47edd0 0x55e1ed48af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441aea70 con 0x55e1ed48b870 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441ac570 con 0x55e1ed47e0f0 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf4c644740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441aa610 con 0x55e1ed47edd0 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59668/0 (socket says 172.21.15.37:59668) 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf42a00640 1 -- 172.21.15.37:0/954743140 learned_addr learned my addr 172.21.15.37:0/954743140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf42000640 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47edd0 0x55e1ed48af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.530+0000 7fcf43400640 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42a00640 1 -- 172.21.15.37:0/954743140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47edd0 msgr2=0x55e1ed48af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42a00640 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47edd0 0x55e1ed48af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42a00640 1 -- 172.21.15.37:0/954743140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 msgr2=0x55e1ed48cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42a00640 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42a00640 1 -- 172.21.15.37:0/954743140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1ed0b60f0 con 0x55e1ed47e0f0 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42000640 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47edd0 0x55e1ed48af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf42a00640 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48aa10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55e1ed49e280 tx=0x7fcf3401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fb12b3d92590526 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf2f400640 1 -- 172.21.15.37:0/954743140 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34003d90 con 0x55e1ed47e0f0 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf2f400640 1 -- 172.21.15.37:0/954743140 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf34030070 con 0x55e1ed47e0f0 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.531+0000 7fcf2f400640 1 -- 172.21.15.37:0/954743140 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3401fa00 con 0x55e1ed47e0f0 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 -- 172.21.15.37:0/954743140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 msgr2=0x55e1ed48aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48aa10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55e1ed49e280 tx=0x7fcf3401b000 comp rx=0 tx=0).stop 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 -- 172.21.15.37:0/954743140 shutdown_connections 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed48b870 0x55e1ed48cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed47edd0 0x55e1ed48af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 --2- 172.21.15.37:0/954743140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed48aa10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 -- 172.21.15.37:0/954743140 >> 172.21.15.37:0/954743140 conn(0x55e1ed445200 msgr2=0x55e1ed478670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 -- 172.21.15.37:0/954743140 shutdown_connections 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.532+0000 7fcf4c644740 1 -- 172.21.15.37:0/954743140 wait complete. 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 -- start start 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 Processor -- start 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed4847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47edd0 0x55e1ed484cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed485230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441aac10 con 0x55e1ed47edd0 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441abf10 con 0x55e1ed47e0f0 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441ab250 con 0x55e1ed48b870 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf4c644740 10 client.0 _ll_register_callbacks cb 0x7fcf300cc930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.533+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed4847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed4847b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59670/0 (socket says 172.21.15.37:59670) 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 -- 172.21.15.37:0/338427098 learned_addr learned my addr 172.21.15.37:0/338427098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42000640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47edd0 0x55e1ed484cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf43400640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed485230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 -- 172.21.15.37:0/338427098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 msgr2=0x55e1ed485230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed485230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 -- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47edd0 msgr2=0x55e1ed484cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47edd0 0x55e1ed484cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 -- 172.21.15.37:0/338427098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1ed486380 con 0x55e1ed47e0f0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42000640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47edd0 0x55e1ed484cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf42a00640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed4847b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf3402bfa0 tx=0x7fcf340043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb74f630ea00025 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.534+0000 7fcf2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf2cc00640 1 -- 172.21.15.37:0/338427098 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34039070 con 0x55e1ed47e0f0 2024-03-21T09:12:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf2cc00640 1 -- 172.21.15.37:0/338427098 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcf34003d90 con 0x55e1ed47e0f0 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf34030070 con 0x55e1ed47e0f0 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf2cc00640 1 -- 172.21.15.37:0/338427098 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf34045050 con 0x55e1ed47e0f0 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf4c644740 10 client.25760 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e1ed4f6fe0 con 0x55e1ed47e0f0 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf2cc00640 1 -- 172.21.15.37:0/338427098 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf34030070 con 0x55e1ed47e0f0 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.535+0000 7fcf2cc00640 1 -- 172.21.15.37:0/338427098 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcf3402bb30 con 0x55e1ed47e0f0 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf4c644740 10 client.25760 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf4c644740 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf4c644740 20 client.25760 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf4c644740 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed4f7680 0x55e1ed4f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf4c644740 4 client.25760 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e1ed3e9ae0 con 0x55e1ed4f7680 2024-03-21T09:12:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf42000640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed4f7680 0x55e1ed4f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf42000640 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed4f7680 0x55e1ed4f9aa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328b5419e85054a9 server_cookie=a0f7dae55cab277f in_seq=0 out_seq=0 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.536+0000 7fcf2cc00640 10 client.25760 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf2cc00640 1 -- 172.21.15.37:0/338427098 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e1ed3e9ae0 con 0x55e1ed4f7680 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf2cc00640 10 client.25760 handle_command_reply: tid=0 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 client.25760 shutdown 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 msgr2=0x55e1ed4847b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed4847b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf3402bfa0 tx=0x7fcf340043b0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 shutdown_connections 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ed48b870 0x55e1ed485230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1ed47edd0 0x55e1ed484cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e1ed4f7680 0x55e1ed4f9aa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 --2- 172.21.15.37:0/338427098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ed47e0f0 0x55e1ed4847b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf2cc00640 0 client.25760 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 >> 172.21.15.37:0/338427098 conn(0x55e1ed445200 msgr2=0x55e1ed486bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 shutdown_connections 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.537+0000 7fcf4c644740 1 -- 172.21.15.37:0/338427098 wait complete. 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf4c644740 20 client.25760 trim_cache size 0 max 16384 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stdout: "5336d1b3-64a2-4ebd-b37d-264806b2ce31": { 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:12:33.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "5336d1b3-64a2-4ebd-b37d-264806b2ce31", 2024-03-21T09:12:33.538 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:12:33.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:12:33.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:12:33.538 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:12:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf280a31a0 msgr2=0x7fcf280a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf280a31a0 0x7fcf280a5660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x55e1ed47f320 tx=0x7fcf34005f90 comp rx=0 tx=0).stop 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 msgr2=0x7fcf4419de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 0x7fcf4419de50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000d950 tx=0x7fcf3000de10 comp rx=0 tx=0).stop 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 shutdown_connections 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf280a31a0 0x7fcf280a5660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf4410ba50 0x7fcf44197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44102f50 0x7fcf4419de50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 --2- 172.21.15.37:0/3843860320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf44102630 0x7fcf4419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 >> 172.21.15.37:0/3843860320 conn(0x7fcf44077130 msgr2=0x7fcf440fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 shutdown_connections 2024-03-21T09:12:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:12:33.538+0000 7fcf43e00640 1 -- 172.21.15.37:0/3843860320 wait complete. 2024-03-21T09:12:33.550 INFO:tasks.cephfs.filesystem:scrub status for tag:5336d1b3-64a2-4ebd-b37d-264806b2ce31 - {'path': '/', 'tag': '5336d1b3-64a2-4ebd-b37d-264806b2ce31', 'options': 'recursive,force'} 2024-03-21T09:12:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:34 smithi071 ceph-mon[36597]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-21T09:12:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:34 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:12:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:34 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:34 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:12:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:34 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:34 smithi037 ceph-mon[31830]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-21T09:12:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:34 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:12:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:34 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:34 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:12:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:34 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:34 smithi053 ceph-mon[36169]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-21T09:12:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:34 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:12:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:34 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:34 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:12:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:34 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:12:36.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:36 smithi071 ceph-mon[36597]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-21T09:12:36.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:36 smithi037 ceph-mon[31830]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-21T09:12:36.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:36 smithi053 ceph-mon[36169]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-21T09:12:37.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:37 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:12:37.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:37 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:12:37.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:37 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:12:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:38 smithi071 ceph-mon[36597]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T09:12:38.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:38 smithi037 ceph-mon[31830]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T09:12:38.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:38 smithi053 ceph-mon[36169]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 -- 172.21.15.37:0/2179635018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78069d30 msgr2=0x7f5a7806a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 --2- 172.21.15.37:0/2179635018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78069d30 0x7f5a7806a1b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009cd0 tx=0x7f5a6002c170 comp rx=0 tx=0).stop 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 -- 172.21.15.37:0/2179635018 shutdown_connections 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 --2- 172.21.15.37:0/2179635018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7806a6f0 0x7f5a7810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 --2- 172.21.15.37:0/2179635018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78069d30 0x7f5a7806a1b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 --2- 172.21.15.37:0/2179635018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a78107710 0x7f5a78107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 -- 172.21.15.37:0/2179635018 >> 172.21.15.37:0/2179635018 conn(0x7f5a78077360 msgr2=0x7f5a78077770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 -- 172.21.15.37:0/2179635018 shutdown_connections 2024-03-21T09:12:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.274+0000 7f5a7d200640 1 -- 172.21.15.37:0/2179635018 wait complete. 2024-03-21T09:12:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.275+0000 7f5a7d200640 1 Processor -- start 2024-03-21T09:12:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 -- start start 2024-03-21T09:12:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a78069d30 0x7f5a781008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 0x7f5a78100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78107710 0x7f5a78101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78111a60 con 0x7f5a78069d30 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781118e0 con 0x7f5a78107710 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.276+0000 7f5a7d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78111be0 con 0x7f5a7806a6f0 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 0x7f5a78100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a78069d30 0x7f5a781008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a76800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78107710 0x7f5a78101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 0x7f5a78100e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52818/0 (socket says 172.21.15.37:52818) 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 -- 172.21.15.37:0/3815329155 learned_addr learned my addr 172.21.15.37:0/3815329155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 -- 172.21.15.37:0/3815329155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78107710 msgr2=0x7f5a78101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78107710 0x7f5a78101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 -- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a78069d30 msgr2=0x7f5a781008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a78069d30 0x7f5a781008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 -- 172.21.15.37:0/3815329155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a781a63c0 con 0x7f5a7806a6f0 2024-03-21T09:12:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.277+0000 7f5a75400640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 0x7f5a78100e00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a60004af0 tx=0x7f5a60035e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.278+0000 7f5a6ea00640 1 -- 172.21.15.37:0/3815329155 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60039040 con 0x7f5a7806a6f0 2024-03-21T09:12:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.278+0000 7f5a6ea00640 1 -- 172.21.15.37:0/3815329155 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a60044070 con 0x7f5a7806a6f0 2024-03-21T09:12:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.278+0000 7f5a6ea00640 1 -- 172.21.15.37:0/3815329155 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60032860 con 0x7f5a7806a6f0 2024-03-21T09:12:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.278+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a781a6650 con 0x7f5a7806a6f0 2024-03-21T09:12:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.278+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781a6af0 con 0x7f5a7806a6f0 2024-03-21T09:12:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.280+0000 7f5a6ea00640 1 -- 172.21.15.37:0/3815329155 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f5a60032a00 con 0x7f5a7806a6f0 2024-03-21T09:12:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.281+0000 7f5a7ff71740 1 Processor -- start 2024-03-21T09:12:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.281+0000 7f5a7ff71740 1 -- start start 2024-03-21T09:12:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf67180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf69ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ceaf203b0 con 0x560ceaf676c0 2024-03-21T09:12:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ceaec2830 con 0x560ceaf5e8f0 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ceae6b5d0 con 0x560ceaf5dfd0 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a76800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a76800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf5e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52830/0 (socket says 172.21.15.37:52830) 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a76800640 1 -- 172.21.15.37:0/2129956230 learned_addr learned my addr 172.21.15.37:0/2129956230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a75400640 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf69ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a6ea00640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5a5c0a31c0 0x7f5a5c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.282+0000 7f5a75e00640 1 -- 172.21.15.37:0/2129956230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 msgr2=0x560ceaf5e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf5e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a75e00640 1 -- 172.21.15.37:0/2129956230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 msgr2=0x560ceaf69ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf69ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a75e00640 1 -- 172.21.15.37:0/2129956230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ceae832c0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a76800640 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf5e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a6ea00640 1 -- 172.21.15.37:0/3815329155 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5a600f1fd0 con 0x7f5a7806a6f0 2024-03-21T09:12:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a76800640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5a5c0a31c0 0x7f5a5c0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf67180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009c20 tx=0x7f5a68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a0d189dfb8f970 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a67400640 1 -- 172.21.15.37:0/2129956230 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004050 con 0x560ceaf5e8f0 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a67400640 1 -- 172.21.15.37:0/2129956230 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a68005500 con 0x560ceaf5e8f0 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.283+0000 7f5a67400640 1 -- 172.21.15.37:0/2129956230 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004db0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2129956230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 msgr2=0x560ceaf67180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a76800640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5a5c0a31c0 0x7f5a5c0a5680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000ba00 tx=0x7f5a700046c0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf67180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009c20 tx=0x7f5a68011810 comp rx=0 tx=0).stop 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2129956230 shutdown_connections 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf69ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf67180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2129956230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf5e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2129956230 >> 172.21.15.37:0/2129956230 conn(0x560ceaf588f0 msgr2=0x560ceaf58cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2129956230 shutdown_connections 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.284+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2129956230 wait complete. 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 -- start start 2024-03-21T09:12:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 Processor -- start 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 0x560ceaf64750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5e8f0 0x560ceaf64c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ceae7ea20 con 0x560ceaf676c0 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ceaebe8d0 con 0x560ceaf5dfd0 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.285+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ceaf6a2f0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a7ff71740 10 client.0 _ll_register_callbacks cb 0x560ceaebb1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 0x560ceaf64750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 0x560ceaf64750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:43460/0 (socket says 172.21.15.37:43460) 2024-03-21T09:12:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5e8f0 0x560ceaf64c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 -- 172.21.15.37:0/3977126387 learned_addr learned my addr 172.21.15.37:0/3977126387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a76800640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 -- 172.21.15.37:0/3977126387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5e8f0 msgr2=0x560ceaf64c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5e8f0 0x560ceaf64c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 -- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 msgr2=0x560ceaf651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 -- 172.21.15.37:0/3977126387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ceabd7210 con 0x560ceaf5dfd0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75400640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5e8f0 0x560ceaf64c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a76800640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.286+0000 7f5a75e00640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 0x560ceaf64750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800a000 tx=0x7f5a680056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19fa627aa2f9504 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a5be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a5be00640 1 -- 172.21.15.37:0/3977126387 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68029070 con 0x560ceaf5dfd0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a5be00640 1 -- 172.21.15.37:0/3977126387 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5a68004d00 con 0x560ceaf5dfd0 2024-03-21T09:12:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a68005500 con 0x560ceaf5dfd0 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a5be00640 1 -- 172.21.15.37:0/3977126387 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68035050 con 0x560ceaf5dfd0 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a7ff71740 10 client.25772 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560ceaca3620 con 0x560ceaf5dfd0 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.287+0000 7f5a5be00640 1 -- 172.21.15.37:0/3977126387 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a68005500 con 0x560ceaf5dfd0 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a5be00640 1 -- 172.21.15.37:0/3977126387 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5a68003ca0 con 0x560ceaf5dfd0 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a7ff71740 10 client.25772 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a7ff71740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a7ff71740 20 client.25772 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceaf77610 0x560ceaf79a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a7ff71740 4 client.25772 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ceaecfd70 con 0x560ceaf77610 2024-03-21T09:12:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.288+0000 7f5a75400640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceaf77610 0x560ceaf79a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.289+0000 7f5a75400640 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceaf77610 0x560ceaf79a00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd37dbf1e7b5ccf2 server_cookie=4f5cfb670587b8c7 in_seq=0 out_seq=0 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.289+0000 7f5a5be00640 10 client.25772 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.289+0000 7f5a5be00640 1 -- 172.21.15.37:0/3977126387 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560ceaecfd70 con 0x560ceaf77610 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.289+0000 7f5a5be00640 10 client.25772 handle_command_reply: tid=0 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 client.25772 shutdown 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 msgr2=0x560ceaf64750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 0x560ceaf64750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800a000 tx=0x7f5a680056b0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 shutdown_connections 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf676c0 0x560ceaf651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5e8f0 0x560ceaf64c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceaf77610 0x560ceaf79a00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/3977126387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5dfd0 0x560ceaf64750 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a5be00640 0 client.25772 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 >> 172.21.15.37:0/3977126387 conn(0x560ceaf588f0 msgr2=0x560ceaf5a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 shutdown_connections 2024-03-21T09:12:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 1 -- 172.21.15.37:0/3977126387 wait complete. 2024-03-21T09:12:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.290+0000 7f5a7ff71740 20 client.25772 trim_cache size 0 max 16384 2024-03-21T09:12:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.305+0000 7f5a7ff71740 1 Processor -- start 2024-03-21T09:12:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.305+0000 7f5a7ff71740 1 -- start start 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5dfd0 0x560ceaf66e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf676c0 0x560ceaf63f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781b30d0 con 0x560ceaf5dfd0 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781b0bd0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781aec70 con 0x560ceaf676c0 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a76800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf676c0 0x560ceaf63f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a75400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a76800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf676c0 0x560ceaf63f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52852/0 (socket says 172.21.15.37:52852) 2024-03-21T09:12:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a76800640 1 -- 172.21.15.37:0/2082668849 learned_addr learned my addr 172.21.15.37:0/2082668849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.306+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5dfd0 0x560ceaf66e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75400640 1 -- 172.21.15.37:0/2082668849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf676c0 msgr2=0x560ceaf63f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75400640 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf676c0 0x560ceaf63f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75400640 1 -- 172.21.15.37:0/2082668849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5dfd0 msgr2=0x560ceaf66e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75400640 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5dfd0 0x560ceaf66e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75400640 1 -- 172.21.15.37:0/2082668849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ceaf64df0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5dfd0 0x560ceaf66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a75400640 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf533a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ef340 tx=0x7f5a600de040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dad64768e88b1f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a66a00640 1 -- 172.21.15.37:0/2082668849 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600bd8c0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a66a00640 1 -- 172.21.15.37:0/2082668849 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a600be660 con 0x560ceaf5e8f0 2024-03-21T09:12:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.307+0000 7f5a66a00640 1 -- 172.21.15.37:0/2082668849 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60003bf0 con 0x560ceaf5e8f0 2024-03-21T09:12:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2082668849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 msgr2=0x560ceaf533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf533a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ef340 tx=0x7f5a600de040 comp rx=0 tx=0).stop 2024-03-21T09:12:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2082668849 shutdown_connections 2024-03-21T09:12:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf676c0 0x560ceaf63f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf5e8f0 0x560ceaf533a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2082668849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5dfd0 0x560ceaf66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2082668849 >> 172.21.15.37:0/2082668849 conn(0x560ceaf588f0 msgr2=0x560ceaf570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2082668849 shutdown_connections 2024-03-21T09:12:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2082668849 wait complete. 2024-03-21T09:12:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.308+0000 7f5a7ff71740 1 -- start start 2024-03-21T09:12:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 Processor -- start 2024-03-21T09:12:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf6a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5e8f0 0x560ceaf6add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf676c0 0x560ceaf6b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781af270 con 0x560ceaf5e8f0 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781b0570 con 0x560ceaf676c0 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781af8b0 con 0x560ceaf5dfd0 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a7ff71740 10 client.0 _ll_register_callbacks cb 0x7f5a600ef200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf6a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf6a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52854/0 (socket says 172.21.15.37:52854) 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a75e00640 1 -- 172.21.15.37:0/2401210524 learned_addr learned my addr 172.21.15.37:0/2401210524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a75400640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5e8f0 0x560ceaf6add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.309+0000 7f5a76800640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf676c0 0x560ceaf6b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a75e00640 1 -- 172.21.15.37:0/2401210524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf676c0 msgr2=0x560ceaf6b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf676c0 0x560ceaf6b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a75e00640 1 -- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5e8f0 msgr2=0x560ceaf6add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5e8f0 0x560ceaf6add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a75e00640 1 -- 172.21.15.37:0/2401210524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ceafd1d30 con 0x560ceaf5dfd0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a75e00640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf6a890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009ae0 tx=0x7f5a68005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49a34e8dda9f0eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a5be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a5be00640 1 -- 172.21.15.37:0/2401210524 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6803e060 con 0x560ceaf5dfd0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a5be00640 1 -- 172.21.15.37:0/2401210524 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5a68004560 con 0x560ceaf5dfd0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.310+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a600be660 con 0x560ceaf5dfd0 2024-03-21T09:12:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a5be00640 1 -- 172.21.15.37:0/2401210524 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68045050 con 0x560ceaf5dfd0 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 10 client.25804 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560ceafd2f80 con 0x560ceaf5dfd0 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a5be00640 1 -- 172.21.15.37:0/2401210524 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a600be660 con 0x560ceaf5dfd0 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a5be00640 1 -- 172.21.15.37:0/2401210524 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5a680224b0 con 0x560ceaf5dfd0 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 10 client.25804 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 20 client.25804 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceafd3620 0x560ceafd5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 4 client.25804 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:12:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.311+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ceaf53a80 con 0x560ceafd3620 2024-03-21T09:12:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.312+0000 7f5a75400640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceafd3620 0x560ceafd5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.312+0000 7f5a75400640 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceafd3620 0x560ceafd5a40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f55e9458534a208 server_cookie=12c246aa565201ae in_seq=0 out_seq=0 2024-03-21T09:12:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.312+0000 7f5a5be00640 10 client.25804 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.315+0000 7f5a5be00640 1 -- 172.21.15.37:0/2401210524 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ceaf53a80 con 0x560ceafd3620 2024-03-21T09:12:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.315+0000 7f5a5be00640 10 client.25804 handle_command_reply: tid=0 2024-03-21T09:12:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.315+0000 7f5a7ff71740 1 client.25804 shutdown 2024-03-21T09:12:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 msgr2=0x560ceaf6a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf6a890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009ae0 tx=0x7f5a68005d30 comp rx=0 tx=0).stop 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 shutdown_connections 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560ceafd3620 0x560ceafd5a40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ceaf676c0 0x560ceaf6b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ceaf5e8f0 0x560ceaf6add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 --2- 172.21.15.37:0/2401210524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560ceaf5dfd0 0x560ceaf6a890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a5be00640 0 client.25804 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 >> 172.21.15.37:0/2401210524 conn(0x560ceaf588f0 msgr2=0x560ceaf57950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 shutdown_connections 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 1 -- 172.21.15.37:0/2401210524 wait complete. 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.316+0000 7f5a7ff71740 20 client.25804 trim_cache size 0 max 16384 2024-03-21T09:12:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:12:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:12:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:12:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5a5c0a31c0 msgr2=0x7f5a5c0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5a5c0a31c0 0x7f5a5c0a5680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000ba00 tx=0x7f5a700046c0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 msgr2=0x7f5a78100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 0x7f5a78100e00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a60004af0 tx=0x7f5a60035e30 comp rx=0 tx=0).stop 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 shutdown_connections 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5a5c0a31c0 0x7f5a5c0a5680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a78107710 0x7f5a78101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7806a6f0 0x7f5a78100e00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 --2- 172.21.15.37:0/3815329155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a78069d30 0x7f5a781008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 >> 172.21.15.37:0/3815329155 conn(0x7f5a78077360 msgr2=0x7f5a780ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 shutdown_connections 2024-03-21T09:12:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:40.317+0000 7f5a7d200640 1 -- 172.21.15.37:0/3815329155 wait complete. 2024-03-21T09:12:40.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:40 smithi071 ceph-mon[36597]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-21T09:12:40.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:40 smithi037 ceph-mon[31830]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-21T09:12:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:40 smithi053 ceph-mon[36169]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-21T09:12:42.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:42 smithi071 ceph-mon[36597]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-21T09:12:42.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:42 smithi037 ceph-mon[31830]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-21T09:12:42.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:42 smithi053 ceph-mon[36169]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-21T09:12:44.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:44 smithi071 ceph-mon[36597]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-21T09:12:44.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:44 smithi037 ceph-mon[31830]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-21T09:12:44.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:44 smithi053 ceph-mon[36169]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-21T09:12:45.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:45 smithi071 ceph-mon[36597]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T09:12:45.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:45 smithi037 ceph-mon[31830]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T09:12:45.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:45 smithi053 ceph-mon[36169]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-21T09:12:48.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:48 smithi071 ceph-mon[36597]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T09:12:48.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:48 smithi037 ceph-mon[31830]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T09:12:48.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:48 smithi053 ceph-mon[36169]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T09:12:50.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:50 smithi037 ceph-mon[31830]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:12:50.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:50 smithi053 ceph-mon[36169]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:12:50.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:50 smithi071 ceph-mon[36597]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:12:52.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:52 smithi037 ceph-mon[31830]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:12:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:52 smithi053 ceph-mon[36169]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:12:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:52 smithi071 ceph-mon[36597]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:12:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 -- 172.21.15.37:0/1578777607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d01014a0 msgr2=0x7f03d0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 --2- 172.21.15.37:0/1578777607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d01014a0 0x7f03d0101920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009930 tx=0x7f03bc02bc30 comp rx=0 tx=0).stop 2024-03-21T09:12:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 -- 172.21.15.37:0/1578777607 shutdown_connections 2024-03-21T09:12:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 --2- 172.21.15.37:0/1578777607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d0101e60 0x7f03d010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 --2- 172.21.15.37:0/1578777607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d01014a0 0x7f03d0101920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 --2- 172.21.15.37:0/1578777607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03d0107560 0x7f03d0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 -- 172.21.15.37:0/1578777607 >> 172.21.15.37:0/1578777607 conn(0x7f03d00fcf50 msgr2=0x7f03d00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 -- 172.21.15.37:0/1578777607 shutdown_connections 2024-03-21T09:12:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.488+0000 7f03d5400640 1 -- 172.21.15.37:0/1578777607 wait complete. 2024-03-21T09:12:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.489+0000 7f03d5400640 1 Processor -- start 2024-03-21T09:12:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 -- start start 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 0x7f03d0199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03d0101e60 0x7f03d019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d0107560 0x7f03d019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d0111870 con 0x7f03d0101e60 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01116f0 con 0x7f03d01014a0 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03d5400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01119f0 con 0x7f03d0107560 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.490+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 0x7f03d0199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 0x7f03d0199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50650/0 (socket says 172.21.15.37:50650) 2024-03-21T09:12:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 -- 172.21.15.37:0/2410123678 learned_addr learned my addr 172.21.15.37:0/2410123678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cf400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03d0101e60 0x7f03d019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03d4a00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d0107560 0x7f03d019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 -- 172.21.15.37:0/2410123678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d0107560 msgr2=0x7f03d019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d0107560 0x7f03d019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 -- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03d0101e60 msgr2=0x7f03d019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03d0101e60 0x7f03d019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 -- 172.21.15.37:0/2410123678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d019eef0 con 0x7f03d01014a0 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03d4a00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d0107560 0x7f03d019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.491+0000 7f03cfe00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 0x7f03d0199f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03c000e9f0 tx=0x7f03c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.492+0000 7f03ccc00640 1 -- 172.21.15.37:0/2410123678 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000cd80 con 0x7f03d01014a0 2024-03-21T09:12:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.492+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d019f1e0 con 0x7f03d01014a0 2024-03-21T09:12:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.492+0000 7f03ccc00640 1 -- 172.21.15.37:0/2410123678 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c0010910 con 0x7f03d01014a0 2024-03-21T09:12:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.492+0000 7f03ccc00640 1 -- 172.21.15.37:0/2410123678 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00188f0 con 0x7f03d01014a0 2024-03-21T09:12:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.492+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d01a6a70 con 0x7f03d01014a0 2024-03-21T09:12:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.496+0000 7f03d9826740 1 Processor -- start 2024-03-21T09:12:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.496+0000 7f03ccc00640 1 -- 172.21.15.37:0/2410123678 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f03c0010ac0 con 0x7f03d01014a0 2024-03-21T09:12:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.496+0000 7f03d9826740 1 -- start start 2024-03-21T09:12:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.496+0000 7f03d9826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db98335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.496+0000 7f03d9826740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db9833b00 0x560db983c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d9826740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 0x560db983ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d9826740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db97f53b0 con 0x560db9833b00 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d9826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db9797830 con 0x560db98331e0 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d9826740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db97405d0 con 0x560db983c8d0 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 0x560db983ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db98335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db98335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50654/0 (socket says 172.21.15.37:50654) 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 0x560db983ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42236/0 (socket says 172.21.15.37:42236) 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db9833b00 0x560db983c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03d4a00640 1 -- 172.21.15.37:0/3716251915 learned_addr learned my addr 172.21.15.37:0/3716251915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 -- 172.21.15.37:0/3716251915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 msgr2=0x560db98335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db98335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 -- 172.21.15.37:0/3716251915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db9833b00 msgr2=0x560db983c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db9833b00 0x560db983c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.497+0000 7f03cf400640 1 -- 172.21.15.37:0/3716251915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db97582c0 con 0x560db983c8d0 2024-03-21T09:12:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.498+0000 7f03cf400640 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 0x560db983ecc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036040 tx=0x7f03bc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be44c95c32ef190 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.498+0000 7f03c5200640 1 -- 172.21.15.37:0/3716251915 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc045070 con 0x560db983c8d0 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.498+0000 7f03c5200640 1 -- 172.21.15.37:0/3716251915 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03bc02c650 con 0x560db983c8d0 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.498+0000 7f03c5200640 1 -- 172.21.15.37:0/3716251915 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc004b20 con 0x560db983c8d0 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.498+0000 7f03d9826740 1 -- 172.21.15.37:0/3716251915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 msgr2=0x560db983ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.498+0000 7f03d9826740 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 0x560db983ecc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f03bc036040 tx=0x7f03bc009560 comp rx=0 tx=0).stop 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03ccc00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f03b40a3180 0x7f03b40a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 -- 172.21.15.37:0/3716251915 shutdown_connections 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db983c8d0 0x560db983ecc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db9833b00 0x560db983c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 --2- 172.21.15.37:0/3716251915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db98335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 -- 172.21.15.37:0/3716251915 >> 172.21.15.37:0/3716251915 conn(0x560db982db00 msgr2=0x560db982dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03ccc00640 1 -- 172.21.15.37:0/2410123678 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f03c00d0100 con 0x7f03d01014a0 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 -- 172.21.15.37:0/3716251915 shutdown_connections 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d9826740 1 -- 172.21.15.37:0/3716251915 wait complete. 2024-03-21T09:12:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.499+0000 7f03d4a00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f03b40a3180 0x7f03b40a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 -- start start 2024-03-21T09:12:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 Processor -- start 2024-03-21T09:12:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d4a00640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f03b40a3180 0x7f03b40a5640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005c10 tx=0x7f03c8009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db9839970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db9839eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db983a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db9753a20 con 0x560db983c8d0 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db97938d0 con 0x560db98331e0 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db983f4a0 con 0x560db9833b00 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.500+0000 7f03d9826740 10 client.0 _ll_register_callbacks cb 0x560db97901c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db9839eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db9839eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42244/0 (socket says 172.21.15.37:42244) 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 -- 172.21.15.37:0/2844703456 learned_addr learned my addr 172.21.15.37:0/2844703456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cfe00640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db9839970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03d4a00640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db983a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 -- 172.21.15.37:0/2844703456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 msgr2=0x560db9839970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db9839970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 -- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 msgr2=0x560db983a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db983a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 -- 172.21.15.37:0/2844703456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db94ac210 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03cf400640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db9839eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02cab0 tx=0x7f03bc02cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b224f6b66980d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.501+0000 7f03b9600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03b9600640 1 -- 172.21.15.37:0/2844703456 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc045040 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03bc02c650 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03b9600640 1 -- 172.21.15.37:0/2844703456 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f03bc0052f0 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03b9600640 1 -- 172.21.15.37:0/2844703456 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03bc05a050 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03d9826740 10 client.25816 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560db9578620 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03b9600640 1 -- 172.21.15.37:0/2844703456 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03bc02c650 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.502+0000 7f03b9600640 1 -- 172.21.15.37:0/2844703456 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f03bc04ac00 con 0x560db9833b00 2024-03-21T09:12:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03d9826740 10 client.25816 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03d9826740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03d9826740 20 client.25816 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03d9826740 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db984c760 0x560db984eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03d9826740 4 client.25816 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560db97a4d70 con 0x560db984c760 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.503+0000 7f03cfe00640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db984c760 0x560db984eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.504+0000 7f03cfe00640 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db984c760 0x560db984eb50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfaf45d2b9688d9 server_cookie=e0926b025718ad11 in_seq=0 out_seq=0 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.504+0000 7f03b9600640 10 client.25816 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.504+0000 7f03b9600640 1 -- 172.21.15.37:0/2844703456 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560db97a4d70 con 0x560db984c760 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.504+0000 7f03b9600640 10 client.25816 handle_command_reply: tid=0 2024-03-21T09:12:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.504+0000 7f03d9826740 1 client.25816 shutdown 2024-03-21T09:12:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 msgr2=0x560db9839eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db9839eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f03bc02cab0 tx=0x7f03bc02cae0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 shutdown_connections 2024-03-21T09:12:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db983a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db9839eb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db984c760 0x560db984eb50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 --2- 172.21.15.37:0/2844703456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db98331e0 0x560db9839970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 >> 172.21.15.37:0/2844703456 conn(0x560db982db00 msgr2=0x560db982f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03b9600640 0 client.25816 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 shutdown_connections 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 1 -- 172.21.15.37:0/2844703456 wait complete. 2024-03-21T09:12:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.505+0000 7f03d9826740 20 client.25816 trim_cache size 0 max 16384 2024-03-21T09:12:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.534+0000 7f03d9826740 1 Processor -- start 2024-03-21T09:12:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 -- start start 2024-03-21T09:12:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db98331e0 0x560db983c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db982f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 0x560db9847bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01b2ef0 con 0x560db98331e0 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01b09f0 con 0x560db983c8d0 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d9826740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01aea90 con 0x560db9833b00 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db982f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db98331e0 0x560db983c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 0x560db9847bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db982f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42254/0 (socket says 172.21.15.37:42254) 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db98331e0 0x560db983c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49286/0 (socket says 172.21.15.37:49286) 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03d4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 0x560db9847bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50672/0 (socket says 172.21.15.37:50672) 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.535+0000 7f03cf400640 1 -- 172.21.15.37:0/3955085100 learned_addr learned my addr 172.21.15.37:0/3955085100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03d4a00640 1 -- 172.21.15.37:0/3955085100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 msgr2=0x560db982f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03d4a00640 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db982f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03d4a00640 1 -- 172.21.15.37:0/3955085100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db98331e0 msgr2=0x560db983c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03d4a00640 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db98331e0 0x560db983c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03d4a00640 1 -- 172.21.15.37:0/3955085100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db983f030 con 0x560db983c8d0 2024-03-21T09:12:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03cf400640 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db982f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03d4a00640 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 0x560db9847bb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005c40 tx=0x7f03c8016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8042583d06934b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03bbe00640 1 -- 172.21.15.37:0/3955085100 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c802d070 con 0x560db983c8d0 2024-03-21T09:12:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03bbe00640 1 -- 172.21.15.37:0/3955085100 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c801fe10 con 0x560db983c8d0 2024-03-21T09:12:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.536+0000 7f03bbe00640 1 -- 172.21.15.37:0/3955085100 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8028c80 con 0x560db983c8d0 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 -- 172.21.15.37:0/3955085100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 msgr2=0x560db9847bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 0x560db9847bb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005c40 tx=0x7f03c8016000 comp rx=0 tx=0).stop 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 -- 172.21.15.37:0/3955085100 shutdown_connections 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db983c8d0 0x560db9847bb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db9833b00 0x560db982f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 --2- 172.21.15.37:0/3955085100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db98331e0 0x560db983c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.537+0000 7f03d9826740 1 -- 172.21.15.37:0/3955085100 >> 172.21.15.37:0/3955085100 conn(0x560db982db00 msgr2=0x560db982ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.538+0000 7f03d9826740 1 -- 172.21.15.37:0/3955085100 shutdown_connections 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.538+0000 7f03d9826740 1 -- 172.21.15.37:0/3955085100 wait complete. 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.538+0000 7f03d9826740 1 -- start start 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.538+0000 7f03d9826740 1 Processor -- start 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 0x560db989ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db9833b00 0x560db98a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db9840380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01af090 con 0x560db983c8d0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01b0390 con 0x560db9833b00 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01af6d0 con 0x560db98331e0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d9826740 10 client.0 _ll_register_callbacks cb 0x7f03c8009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 0x560db989ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db9833b00 0x560db98a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 0x560db989ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42262/0 (socket says 172.21.15.37:42262) 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 -- 172.21.15.37:0/10818933 learned_addr learned my addr 172.21.15.37:0/10818933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03d4a00640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db9840380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 -- 172.21.15.37:0/10818933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db9833b00 msgr2=0x560db98a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db9833b00 0x560db98a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 -- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 msgr2=0x560db9840380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db9840380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cfe00640 1 -- 172.21.15.37:0/10818933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db9840a70 con 0x560db98331e0 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.539+0000 7f03cf400640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db9833b00 0x560db98a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03cfe00640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 0x560db989ff90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f03c009bf70 tx=0x7f03c00d3960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09bf7a63cd239eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03b9600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03b9600640 1 -- 172.21.15.37:0/10818933 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00a0130 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03b9600640 1 -- 172.21.15.37:0/10818933 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f03c00bc0b0 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03c801fe10 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03b9600640 1 -- 172.21.15.37:0/10818933 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c00b50d0 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03d9826740 10 client.25822 fetch_fsmap learned FSMap version 24 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560db98a88d0 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03b9600640 1 -- 172.21.15.37:0/10818933 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c801fe10 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.540+0000 7f03b9600640 1 -- 172.21.15.37:0/10818933 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f03c0096b50 con 0x560db98331e0 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03d9826740 10 client.25822 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03d9826740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03d9826740 20 client.25822 populate_metadata read hostname 'smithi037' 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03d9826740 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db98a8fd0 0x560db98ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03d9826740 4 client.25822 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560db979f0c0 con 0x560db98a8fd0 2024-03-21T09:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.541+0000 7f03cf400640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db98a8fd0 0x560db98ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.542+0000 7f03cf400640 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db98a8fd0 0x560db98ab3f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f94f66bdb2aee74 server_cookie=b812543396657b63 in_seq=0 out_seq=0 2024-03-21T09:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.542+0000 7f03b9600640 10 client.25822 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.549+0000 7f03b9600640 1 -- 172.21.15.37:0/10818933 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560db979f0c0 con 0x560db98a8fd0 2024-03-21T09:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.549+0000 7f03b9600640 10 client.25822 handle_command_reply: tid=0 2024-03-21T09:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.549+0000 7f03d9826740 1 client.25822 shutdown 2024-03-21T09:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.549+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 msgr2=0x560db989ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.549+0000 7f03d9826740 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 0x560db989ff90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f03c009bf70 tx=0x7f03c00d3960 comp rx=0 tx=0).stop 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 shutdown_connections 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560db983c8d0 0x560db9840380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560db98a8fd0 0x560db98ab3f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560db9833b00 0x560db98a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 --2- 172.21.15.37:0/10818933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560db98331e0 0x560db989ff90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 >> 172.21.15.37:0/10818933 conn(0x560db982db00 msgr2=0x560db982cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03b9600640 0 client.25822 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 shutdown_connections 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 1 -- 172.21.15.37:0/10818933 wait complete. 2024-03-21T09:12:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.550+0000 7f03d9826740 20 client.25822 trim_cache size 0 max 16384 2024-03-21T09:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:12:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f03b40a3180 msgr2=0x7f03b40a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f03b40a3180 0x7f03b40a5640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03c8005c10 tx=0x7f03c8009500 comp rx=0 tx=0).stop 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 msgr2=0x7f03d0199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 0x7f03d0199f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03c000e9f0 tx=0x7f03c000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 shutdown_connections 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03d0107560 0x7f03d019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03d0101e60 0x7f03d019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f03b40a3180 0x7f03b40a5640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 --2- 172.21.15.37:0/2410123678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03d01014a0 0x7f03d0199f00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 >> 172.21.15.37:0/2410123678 conn(0x7f03d00fcf50 msgr2=0x7f03d00fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 shutdown_connections 2024-03-21T09:12:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:12:53.551+0000 7f03d5400640 1 -- 172.21.15.37:0/2410123678 wait complete. 2024-03-21T09:12:54.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:54 smithi037 ceph-mon[31830]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T09:12:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:54 smithi053 ceph-mon[36169]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T09:12:54.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:54 smithi071 ceph-mon[36597]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T09:12:56.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:56 smithi037 ceph-mon[31830]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-21T09:12:56.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:56 smithi053 ceph-mon[36169]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-21T09:12:56.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:56 smithi071 ceph-mon[36597]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-21T09:12:58.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:12:58 smithi037 ceph-mon[31830]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T09:12:58.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:12:58 smithi053 ceph-mon[36169]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T09:12:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:12:58 smithi071 ceph-mon[36597]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T09:13:00.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:00 smithi037 ceph-mon[31830]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T09:13:00.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:00 smithi053 ceph-mon[36169]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T09:13:00.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:00 smithi071 ceph-mon[36597]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T09:13:00.922 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:13:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:13:00.656+0000 7f0289400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:13:00.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:13:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:13:00.656+0000 7f0289400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:13:00.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:13:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:13:00.656+0000 7f0289400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:13:00.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:13:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:13:00.656+0000 7f0289400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:13:00.923 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:13:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:13:00.656+0000 7f0289400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T09:13:02.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:02 smithi071 ceph-mon[36597]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T09:13:02.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:02 smithi071 ceph-mon[36597]: mgrmap e27: x(active, since 50s), standbys: y, z 2024-03-21T09:13:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:02 smithi037 ceph-mon[31830]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T09:13:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:02 smithi037 ceph-mon[31830]: mgrmap e27: x(active, since 50s), standbys: y, z 2024-03-21T09:13:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:02 smithi053 ceph-mon[36169]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T09:13:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:02 smithi053 ceph-mon[36169]: mgrmap e27: x(active, since 50s), standbys: y, z 2024-03-21T09:13:03.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:13:03.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:13:03.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 -- 172.21.15.37:0/2402047006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac100b60 msgr2=0x7f6eac10d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 --2- 172.21.15.37:0/2402047006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac100b60 0x7f6eac10d820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009900 tx=0x7f6e9402bc30 comp rx=0 tx=0).stop 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 -- 172.21.15.37:0/2402047006 shutdown_connections 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 --2- 172.21.15.37:0/2402047006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac10dd60 0x7f6eac110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 --2- 172.21.15.37:0/2402047006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac100b60 0x7f6eac10d820 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 --2- 172.21.15.37:0/2402047006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6eac1001b0 0x7f6eac100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.768+0000 7f6eab200640 1 -- 172.21.15.37:0/2402047006 >> 172.21.15.37:0/2402047006 conn(0x7f6eac077130 msgr2=0x7f6eac0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.769+0000 7f6eab200640 1 -- 172.21.15.37:0/2402047006 shutdown_connections 2024-03-21T09:13:03.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.769+0000 7f6eab200640 1 -- 172.21.15.37:0/2402047006 wait complete. 2024-03-21T09:13:03.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 Processor -- start 2024-03-21T09:13:03.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 -- start start 2024-03-21T09:13:03.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 0x7f6eac199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6eac100b60 0x7f6eac19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac10dd60 0x7f6eac19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac113aa0 con 0x7f6eac100b60 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac113920 con 0x7f6eac1001b0 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6eab200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac113c20 con 0x7f6eac10dd60 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 0x7f6eac199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.770+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 0x7f6eac199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53458/0 (socket says 172.21.15.37:53458) 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 -- 172.21.15.37:0/1255389667 learned_addr learned my addr 172.21.15.37:0/1255389667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6eaa800640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac10dd60 0x7f6eac19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6eac100b60 0x7f6eac19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 -- 172.21.15.37:0/1255389667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac10dd60 msgr2=0x7f6eac19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac10dd60 0x7f6eac19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 -- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6eac100b60 msgr2=0x7f6eac19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6eac100b60 0x7f6eac19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 -- 172.21.15.37:0/1255389667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac19ef60 con 0x7f6eac1001b0 2024-03-21T09:13:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6eaa800640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac10dd60 0x7f6eac19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.771+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 0x7f6eac199fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00e9f0 tx=0x7f6e9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.772+0000 7f6ea3400640 1 -- 172.21.15.37:0/1255389667 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00cd80 con 0x7f6eac1001b0 2024-03-21T09:13:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.772+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac19f250 con 0x7f6eac1001b0 2024-03-21T09:13:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.772+0000 7f6ea3400640 1 -- 172.21.15.37:0/1255389667 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9c010910 con 0x7f6eac1001b0 2024-03-21T09:13:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.772+0000 7f6ea3400640 1 -- 172.21.15.37:0/1255389667 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0188f0 con 0x7f6eac1001b0 2024-03-21T09:13:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.772+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac1a6ae0 con 0x7f6eac1001b0 2024-03-21T09:13:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.776+0000 7f6ea3400640 1 -- 172.21.15.37:0/1255389667 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f6e9c010ac0 con 0x7f6eac1001b0 2024-03-21T09:13:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.776+0000 7f6eb3803740 1 Processor -- start 2024-03-21T09:13:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.776+0000 7f6eb3803740 1 -- start start 2024-03-21T09:13:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9add0 0x564dc3d99b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3da5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eb3803740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3cfe250 con 0x564dc3d9add0 2024-03-21T09:13:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eb3803740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3ca6a30 con 0x564dc3da3460 2024-03-21T09:13:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eb3803740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3cfa2f0 con 0x564dc3d9a0f0 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6eaa800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9add0 0x564dc3d99b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6ea1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3da5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6ea1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3da5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53462/0 (socket says 172.21.15.37:53462) 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6ea1400640 1 -- 172.21.15.37:0/1063095366 learned_addr learned my addr 172.21.15.37:0/1063095366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.777+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49398/0 (socket says 172.21.15.37:49398) 2024-03-21T09:13:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea1400640 1 -- 172.21.15.37:0/1063095366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 msgr2=0x564dc3d9a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea1400640 1 -- 172.21.15.37:0/1063095366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9add0 msgr2=0x564dc3d99b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9add0 0x564dc3d99b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea1400640 1 -- 172.21.15.37:0/1063095366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc3c91690 con 0x564dc3da3460 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3da5850 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009900 tx=0x7f6e94005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eaf7c7acb6ea656 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea3400640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e900a3230 0x7f6e900a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6ea3400640 1 -- 172.21.15.37:0/1255389667 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6e9c0d01d0 con 0x7f6eac1001b0 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6e9b000640 1 -- 172.21.15.37:0/1063095366 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94045070 con 0x564dc3da3460 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.778+0000 7f6e9b000640 1 -- 172.21.15.37:0/1063095366 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9402c880 con 0x564dc3da3460 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eaa800640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e900a3230 0x7f6e900a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6e9b000640 1 -- 172.21.15.37:0/1063095366 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94004c20 con 0x564dc3da3460 2024-03-21T09:13:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 -- 172.21.15.37:0/1063095366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 msgr2=0x564dc3da5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3da5850 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009900 tx=0x7f6e94005be0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 -- 172.21.15.37:0/1063095366 shutdown_connections 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3da5850 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1063095366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9add0 0x564dc3d99b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.779+0000 7f6eb3803740 1 -- 172.21.15.37:0/1063095366 >> 172.21.15.37:0/1063095366 conn(0x564dc3d61200 msgr2=0x564dc3d94b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.780+0000 7f6eb3803740 1 -- 172.21.15.37:0/1063095366 shutdown_connections 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.780+0000 7f6eaa800640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e900a3230 0x7f6e900a56f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009fd0 tx=0x7f6ea4009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.780+0000 7f6eb3803740 1 -- 172.21.15.37:0/1063095366 wait complete. 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.780+0000 7f6eb3803740 1 -- start start 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.780+0000 7f6eb3803740 1 Processor -- start 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3da2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3d9fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3da3460 0x564dc3da0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3da62c0 con 0x564dc3da3460 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3cb9e80 con 0x564dc3d9add0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc3da6440 con 0x564dc3d9a0f0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6eb3803740 10 client.0 _ll_register_callbacks cb 0x564dc3ccc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.781+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3da2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3d9fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3da2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49402/0 (socket says 172.21.15.37:49402) 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea9e00640 1 -- 172.21.15.37:0/1398435382 learned_addr learned my addr 172.21.15.37:0/1398435382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6eaa800640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3da3460 0x564dc3da0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 -- 172.21.15.37:0/1398435382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 msgr2=0x564dc3da2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3da2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 -- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3da3460 msgr2=0x564dc3da0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3da3460 0x564dc3da0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 -- 172.21.15.37:0/1398435382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc3adc7d0 con 0x564dc3d9add0 2024-03-21T09:13:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3da2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3d9fe40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6e94036040 tx=0x7f6e9402cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=264971b026689772 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.782+0000 7f6e8f400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6e8f400640 1 -- 172.21.15.37:0/1398435382 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94045040 con 0x564dc3d9add0 2024-03-21T09:13:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6e8f400640 1 -- 172.21.15.37:0/1398435382 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6e94005100 con 0x564dc3d9add0 2024-03-21T09:13:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e9402c880 con 0x564dc3d9add0 2024-03-21T09:13:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6e8f400640 1 -- 172.21.15.37:0/1398435382 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e94005300 con 0x564dc3d9add0 2024-03-21T09:13:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6eb3803740 10 client.25814 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564dc3a123f0 con 0x564dc3d9add0 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.783+0000 7f6e8f400640 1 -- 172.21.15.37:0/1398435382 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9402c880 con 0x564dc3d9add0 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6e8f400640 1 -- 172.21.15.37:0/1398435382 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6e94055c40 con 0x564dc3d9add0 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6eb3803740 10 client.25814 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6eb3803740 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6eb3803740 20 client.25814 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3db3150 0x564dc3db5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6eb3803740 4 client.25814 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dc3d0b790 con 0x564dc3db3150 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.784+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3db3150 0x564dc3db5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.785+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3db3150 0x564dc3db5540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b43358246090e2 server_cookie=e9f94149ae793de1 in_seq=0 out_seq=0 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.785+0000 7f6e8f400640 10 client.25814 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6e8f400640 1 -- 172.21.15.37:0/1398435382 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564dc3d0b790 con 0x564dc3db3150 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6e8f400640 10 client.25814 handle_command_reply: tid=0 2024-03-21T09:13:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 client.25814 shutdown 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 msgr2=0x564dc3d9fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3d9fe40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6e94036040 tx=0x7f6e9402cc60 comp rx=0 tx=0).stop 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 shutdown_connections 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3da3460 0x564dc3da0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3db3150 0x564dc3db5540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3d9fe40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1398435382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9a0f0 0x564dc3da2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6e8f400640 0 client.25814 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.786+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 >> 172.21.15.37:0/1398435382 conn(0x564dc3d61200 msgr2=0x564dc3d96890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.787+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 shutdown_connections 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.787+0000 7f6eb3803740 1 -- 172.21.15.37:0/1398435382 wait complete. 2024-03-21T09:13:03.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.787+0000 7f6eb3803740 20 client.25814 trim_cache size 0 max 16384 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 Processor -- start 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 -- start start 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3daf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3da2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3da3460 0x564dc3da2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1b2f00 con 0x564dc3d9a0f0 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1b0a00 con 0x564dc3d9add0 2024-03-21T09:13:03.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6eb3803740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1aeaa0 con 0x564dc3da3460 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6ea1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3da2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.808+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3daf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6eaa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3da3460 0x564dc3da2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3da2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53484/0 (socket says 172.21.15.37:53484) 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 -- 172.21.15.37:0/1303560724 learned_addr learned my addr 172.21.15.37:0/1303560724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3daf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56666/0 (socket says 172.21.15.37:56666) 2024-03-21T09:13:03.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6eaa800640 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3da3460 0x564dc3da2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49406/0 (socket says 172.21.15.37:49406) 2024-03-21T09:13:03.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 -- 172.21.15.37:0/1303560724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3da3460 msgr2=0x564dc3da2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3da3460 0x564dc3da2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 -- 172.21.15.37:0/1303560724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 msgr2=0x564dc3daf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3daf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 -- 172.21.15.37:0/1303560724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc39d20f0 con 0x564dc3d9add0 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.809+0000 7f6ea1400640 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3da2570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e94006010 tx=0x7f6e94005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b9aa06cbadc46b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.810+0000 7f6e9a600640 1 -- 172.21.15.37:0/1303560724 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94045070 con 0x564dc3d9add0 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.810+0000 7f6e9a600640 1 -- 172.21.15.37:0/1303560724 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9402c880 con 0x564dc3d9add0 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.810+0000 7f6e9a600640 1 -- 172.21.15.37:0/1303560724 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9405a030 con 0x564dc3d9add0 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.810+0000 7f6eb3803740 1 -- 172.21.15.37:0/1303560724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 msgr2=0x564dc3da2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3da2570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e94006010 tx=0x7f6e94005c90 comp rx=0 tx=0).stop 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 -- 172.21.15.37:0/1303560724 shutdown_connections 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3da3460 0x564dc3da2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3d9add0 0x564dc3da2570 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 --2- 172.21.15.37:0/1303560724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3daf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 -- 172.21.15.37:0/1303560724 >> 172.21.15.37:0/1303560724 conn(0x564dc3d61200 msgr2=0x564dc3d9e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:03.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 -- 172.21.15.37:0/1303560724 shutdown_connections 2024-03-21T09:13:03.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 -- 172.21.15.37:0/1303560724 wait complete. 2024-03-21T09:13:03.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.811+0000 7f6eb3803740 1 -- start start 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 Processor -- start 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9add0 0x564dc3d9cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3d9d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1af0a0 con 0x564dc3d9a0f0 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1b03a0 con 0x564dc3da3460 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1af6e0 con 0x564dc3d9add0 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eb3803740 10 client.0 _ll_register_callbacks cb 0x7f6ea4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.812+0000 7f6eaa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3d9d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3d9d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:53500/0 (socket says 172.21.15.37:53500) 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 -- 172.21.15.37:0/2749501222 learned_addr learned my addr 172.21.15.37:0/2749501222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6ea1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9add0 0x564dc3d9cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6ea9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 -- 172.21.15.37:0/2749501222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9add0 msgr2=0x564dc3d9cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9add0 0x564dc3d9cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 -- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 msgr2=0x564dc3d9c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 -- 172.21.15.37:0/2749501222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc3e0d670 con 0x564dc3da3460 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6ea1400640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9add0 0x564dc3d9cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6eaa800640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3d9d0b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005fd0 tx=0x7f6ea4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd58fcd5451e2c39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:03.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6e8f400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.813+0000 7f6e8f400640 1 -- 172.21.15.37:0/2749501222 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4004b20 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e9402c880 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6e8f400640 1 -- 172.21.15.37:0/2749501222 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6ea4004cc0 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6e8f400640 1 -- 172.21.15.37:0/2749501222 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea4012020 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 10 client.25820 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564dc3e0e9a0 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6e8f400640 1 -- 172.21.15.37:0/2749501222 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e9402c880 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6e8f400640 1 -- 172.21.15.37:0/2749501222 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6ea4029070 con 0x564dc3da3460 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 10 client.25820 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 20 client.25820 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.814+0000 7f6eb3803740 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3e0f0d0 0x564dc3e114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.815+0000 7f6eb3803740 4 client.25820 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.815+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564dc3d05ae0 con 0x564dc3e0f0d0 2024-03-21T09:13:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.815+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3e0f0d0 0x564dc3e114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:03.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.815+0000 7f6ea9e00640 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3e0f0d0 0x564dc3e114f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860760270458154c server_cookie=8c6d1f3dee8a1d82 in_seq=0 out_seq=0 2024-03-21T09:13:03.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.815+0000 7f6e8f400640 10 client.25820 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6e8f400640 1 -- 172.21.15.37:0/2749501222 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564dc3d05ae0 con 0x564dc3e0f0d0 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6e8f400640 10 client.25820 handle_command_reply: tid=0 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 client.25820 shutdown 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 msgr2=0x564dc3d9d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3d9d0b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005fd0 tx=0x7f6ea4004ab0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 shutdown_connections 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564dc3e0f0d0 0x564dc3e114f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564dc3da3460 0x564dc3d9d0b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564dc3d9add0 0x564dc3d9cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 --2- 172.21.15.37:0/2749501222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564dc3d9a0f0 0x564dc3d9c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6e8f400640 0 client.25820 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 >> 172.21.15.37:0/2749501222 conn(0x564dc3d61200 msgr2=0x564dc3d95f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:03.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 shutdown_connections 2024-03-21T09:13:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 1 -- 172.21.15.37:0/2749501222 wait complete. 2024-03-21T09:13:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.816+0000 7f6eb3803740 20 client.25820 trim_cache size 0 max 16384 2024-03-21T09:13:03.814 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:13:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:13:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:13:03.814 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e900a3230 msgr2=0x7f6e900a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e900a3230 0x7f6e900a56f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009fd0 tx=0x7f6ea4009500 comp rx=0 tx=0).stop 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 msgr2=0x7f6eac199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 0x7f6eac199fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00e9f0 tx=0x7f6e9c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 shutdown_connections 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eac10dd60 0x7f6eac19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6eac100b60 0x7f6eac19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6e900a3230 0x7f6e900a56f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 --2- 172.21.15.37:0/1255389667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac1001b0 0x7f6eac199fc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 >> 172.21.15.37:0/1255389667 conn(0x7f6eac077130 msgr2=0x7f6eac0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:03.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.817+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 shutdown_connections 2024-03-21T09:13:03.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:03.818+0000 7f6eab200640 1 -- 172.21.15.37:0/1255389667 wait complete. 2024-03-21T09:13:03.828 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:13:03.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:13:04.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.029+0000 7fd18ca00640 1 -- 172.21.15.37:0/1307483645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 msgr2=0x7fd1881019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1307483645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd1881019c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd174009930 tx=0x7fd17402bc20 comp rx=0 tx=0).stop 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 -- 172.21.15.37:0/1307483645 shutdown_connections 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1307483645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd18810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1307483645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd1881019c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1307483645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd188107600 0x7fd1881079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 -- 172.21.15.37:0/1307483645 >> 172.21.15.37:0/1307483645 conn(0x7fd1880fd050 msgr2=0x7fd1880ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 -- 172.21.15.37:0/1307483645 shutdown_connections 2024-03-21T09:13:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.030+0000 7fd18ca00640 1 -- 172.21.15.37:0/1307483645 wait complete. 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.031+0000 7fd18ca00640 1 Processor -- start 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.031+0000 7fd18ca00640 1 -- start start 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.031+0000 7fd18ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd188197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd18ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd1881982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd18ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd188107600 0x7fd18819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd18ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188111910 con 0x7fd188107600 2024-03-21T09:13:04.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd18ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188111790 con 0x7fd188101f00 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd18ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188111a90 con 0x7fd188101540 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd1881982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd187400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd188197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd1881982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54178/0 (socket says 172.21.15.37:54178) 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 -- 172.21.15.37:0/1391148013 learned_addr learned my addr 172.21.15.37:0/1391148013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd187400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd188197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58324/0 (socket says 172.21.15.37:58324) 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd187e00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd188107600 0x7fd18819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 -- 172.21.15.37:0/1391148013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 msgr2=0x7fd188197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd188197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 -- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd188107600 msgr2=0x7fd18819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd188107600 0x7fd18819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd186a00640 1 -- 172.21.15.37:0/1391148013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18819cd80 con 0x7fd188101f00 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.032+0000 7fd187400640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd188197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:04.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.033+0000 7fd186a00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd1881982d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd174005ab0 tx=0x7fd174005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.033+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174043020 con 0x7fd188101f00 2024-03-21T09:13:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.033+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd174005000 con 0x7fd188101f00 2024-03-21T09:13:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.033+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174003a80 con 0x7fd188101f00 2024-03-21T09:13:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.033+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18819d010 con 0x7fd188101f00 2024-03-21T09:13:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.033+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1881a4900 con 0x7fd188101f00 2024-03-21T09:13:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.034+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188102c60 con 0x7fd188101f00 2024-03-21T09:13:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.037+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fd174046020 con 0x7fd188101f00 2024-03-21T09:13:04.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.039+0000 7fd17fe00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd16c0a3220 0x7fd16c0a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.040+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd1740f1c70 con 0x7fd188101f00 2024-03-21T09:13:04.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.040+0000 7fd187400640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd16c0a3220 0x7fd16c0a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.041+0000 7fd187400640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd16c0a3220 0x7fd16c0a56e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd178009fc0 tx=0x7fd178005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.043+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd17404d020 con 0x7fd188101f00 2024-03-21T09:13:04.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:04 smithi037 ceph-mon[31830]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-21T09:13:04.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.364+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd188109950 con 0x7fd188101f00 2024-03-21T09:13:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.365+0000 7fd17fe00640 1 -- 172.21.15.37:0/1391148013 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fd1740be280 con 0x7fd188101f00 2024-03-21T09:13:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:13:04.359 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:11:48.827236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:13:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd16c0a3220 msgr2=0x7fd16c0a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd16c0a3220 0x7fd16c0a56e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd178009fc0 tx=0x7fd178005f90 comp rx=0 tx=0).stop 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 msgr2=0x7fd1881982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd1881982d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd174005ab0 tx=0x7fd174005e80 comp rx=0 tx=0).stop 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 shutdown_connections 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd188107600 0x7fd18819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd16c0a3220 0x7fd16c0a56e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd188101f00 0x7fd1881982d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 --2- 172.21.15.37:0/1391148013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd188101540 0x7fd188197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.368+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 >> 172.21.15.37:0/1391148013 conn(0x7fd1880fd050 msgr2=0x7fd1880fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.369+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 shutdown_connections 2024-03-21T09:13:04.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.369+0000 7fd18ca00640 1 -- 172.21.15.37:0/1391148013 wait complete. 2024-03-21T09:13:04.379 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:13:04.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:13:04.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:04 smithi053 ceph-mon[36169]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-21T09:13:04.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 -- 172.21.15.37:0/3943685338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 msgr2=0x7fa59410d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 --2- 172.21.15.37:0/3943685338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59410d760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa580009900 tx=0x7fa58002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 -- 172.21.15.37:0/3943685338 shutdown_connections 2024-03-21T09:13:04.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 --2- 172.21.15.37:0/3943685338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa59410dca0 0x7fa5941100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 --2- 172.21.15.37:0/3943685338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59410d760 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 --2- 172.21.15.37:0/3943685338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594073ad0 0x7fa594073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 -- 172.21.15.37:0/3943685338 >> 172.21.15.37:0/3943685338 conn(0x7fa5940fcff0 msgr2=0x7fa5940ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 -- 172.21.15.37:0/3943685338 shutdown_connections 2024-03-21T09:13:04.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.591+0000 7fa593200640 1 -- 172.21.15.37:0/3943685338 wait complete. 2024-03-21T09:13:04.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 Processor -- start 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 -- start start 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa594073ad0 0x7fa594199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59410dca0 0x7fa59419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594113a40 con 0x7fa59410dca0 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941138c0 con 0x7fa594073ad0 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa593200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594113bc0 con 0x7fa5940743f0 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa591400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa591e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa594073ad0 0x7fa594199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa591400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58338/0 (socket says 172.21.15.37:58338) 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.593+0000 7fa591400640 1 -- 172.21.15.37:0/2455991099 learned_addr learned my addr 172.21.15.37:0/2455991099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa592800640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59410dca0 0x7fa59419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa591400640 1 -- 172.21.15.37:0/2455991099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa594073ad0 msgr2=0x7fa594199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa591400640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa594073ad0 0x7fa594199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa591400640 1 -- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59410dca0 msgr2=0x7fa59419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa591400640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59410dca0 0x7fa59419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa591400640 1 -- 172.21.15.37:0/2455991099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59419ef90 con 0x7fa5940743f0 2024-03-21T09:13:04.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.594+0000 7fa591400640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59419a4e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa580036040 tx=0x7fa58002cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.595+0000 7fa58aa00640 1 -- 172.21.15.37:0/2455991099 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580043020 con 0x7fa5940743f0 2024-03-21T09:13:04.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.595+0000 7fa58aa00640 1 -- 172.21.15.37:0/2455991099 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5800054e0 con 0x7fa5940743f0 2024-03-21T09:13:04.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.595+0000 7fa58aa00640 1 -- 172.21.15.37:0/2455991099 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580003c20 con 0x7fa5940743f0 2024-03-21T09:13:04.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.595+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59419f220 con 0x7fa5940743f0 2024-03-21T09:13:04.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.595+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a6b10 con 0x7fa5940743f0 2024-03-21T09:13:04.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.598+0000 7fa58aa00640 1 -- 172.21.15.37:0/2455991099 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fa580046020 con 0x7fa5940743f0 2024-03-21T09:13:04.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.598+0000 7fa59bc06740 1 Processor -- start 2024-03-21T09:13:04.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 -- start start 2024-03-21T09:13:04.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628e34112c0 con 0x5628e34ae260 2024-03-21T09:13:04.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628e33b9c90 con 0x5628e34ad8b0 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa59bc06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628e340d360 con 0x5628e34b6de0 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa592800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa592800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54200/0 (socket says 172.21.15.37:54200) 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa592800640 1 -- 172.21.15.37:0/1848343666 learned_addr learned my addr 172.21.15.37:0/1848343666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa591400640 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.599+0000 7fa591e00640 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa592800640 1 -- 172.21.15.37:0/1848343666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 msgr2=0x5628e34b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa592800640 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa592800640 1 -- 172.21.15.37:0/1848343666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 msgr2=0x5628e34b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa592800640 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa592800640 1 -- 172.21.15.37:0/1848343666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628e33de080 con 0x5628e34ad8b0 2024-03-21T09:13:04.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa591e00640 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa592800640 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34adc90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009f70 tx=0x7fa58c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e2afb33b7169a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa58aa00640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5780a3230 0x7fa5780a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa57f400640 1 -- 172.21.15.37:0/1848343666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c016070 con 0x5628e34ad8b0 2024-03-21T09:13:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa57f400640 1 -- 172.21.15.37:0/1848343666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa58c004d20 con 0x5628e34ad8b0 2024-03-21T09:13:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.600+0000 7fa57f400640 1 -- 172.21.15.37:0/1848343666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c012950 con 0x5628e34ad8b0 2024-03-21T09:13:04.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.601+0000 7fa58aa00640 1 -- 172.21.15.37:0/2455991099 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa5800f1de0 con 0x7fa5940743f0 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.601+0000 7fa591e00640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5780a3230 0x7fa5780a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.601+0000 7fa59bc06740 1 -- 172.21.15.37:0/1848343666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 msgr2=0x5628e34adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.601+0000 7fa591e00640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5780a3230 0x7fa5780a56f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x5628e34a8ff0 tx=0x7fa584005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.601+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34adc90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009f70 tx=0x7fa58c010800 comp rx=0 tx=0).stop 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 -- 172.21.15.37:0/1848343666 shutdown_connections 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1848343666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34adc90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 -- 172.21.15.37:0/1848343666 >> 172.21.15.37:0/1848343666 conn(0x5628e34a7f80 msgr2=0x5628e34a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 -- 172.21.15.37:0/1848343666 shutdown_connections 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 -- 172.21.15.37:0/1848343666 wait complete. 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 -- start start 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.602+0000 7fa59bc06740 1 Processor -- start 2024-03-21T09:13:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628e34b9d50 con 0x5628e34ae260 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628e33cd0d0 con 0x5628e34ad8b0 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628e34b9ed0 con 0x5628e34b6de0 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa59bc06740 10 client.0 _ll_register_callbacks cb 0x5628e33d1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa591e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa591e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54210/0 (socket says 172.21.15.37:54210) 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa591e00640 1 -- 172.21.15.37:0/1019262283 learned_addr learned my addr 172.21.15.37:0/1019262283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa591400640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.603+0000 7fa592800640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa591e00640 1 -- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 msgr2=0x5628e34b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa591e00640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa591e00640 1 -- 172.21.15.37:0/1019262283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 msgr2=0x5628e34b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa591e00640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa591e00640 1 -- 172.21.15.37:0/1019262283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628e31253f0 con 0x5628e34ad8b0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa592800640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa591e00640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34b6780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa584013010 tx=0x7fa584004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928f05318d03c0a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa573e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa573e00640 1 -- 172.21.15.37:0/1019262283 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58402e060 con 0x5628e34ad8b0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa573e00640 1 -- 172.21.15.37:0/1019262283 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa584018030 con 0x5628e34ad8b0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.604+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa58c004d20 con 0x5628e34ad8b0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa573e00640 1 -- 172.21.15.37:0/1019262283 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa584035050 con 0x5628e34ad8b0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 10 client.25838 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5628e31ef7d0 con 0x5628e34ad8b0 2024-03-21T09:13:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa573e00640 1 -- 172.21.15.37:0/1019262283 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa58c004d20 con 0x5628e34ad8b0 2024-03-21T09:13:04.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:04 smithi071 ceph-mon[36597]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa573e00640 1 -- 172.21.15.37:0/1019262283 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa58401c3a0 con 0x5628e34ad8b0 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 10 client.25838 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 10 client.25838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 20 client.25838 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e34c6d30 0x5628e34c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 4 client.25838 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.605+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628e341e800 con 0x5628e34c6d30 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.606+0000 7fa591400640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e34c6d30 0x5628e34c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.606+0000 7fa591400640 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e34c6d30 0x5628e34c9140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83301412862d96f5 server_cookie=44efdb1aaa753278 in_seq=0 out_seq=0 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.606+0000 7fa573e00640 10 client.25838 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa573e00640 1 -- 172.21.15.37:0/1019262283 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5628e341e800 con 0x5628e34c6d30 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa573e00640 10 client.25838 handle_command_reply: tid=0 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 client.25838 shutdown 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 msgr2=0x5628e34b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34b6780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa584013010 tx=0x7fa584004730 comp rx=0 tx=0).stop 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 shutdown_connections 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e34c6d30 0x5628e34c9140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 --2- 172.21.15.37:0/1019262283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34b6780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 >> 172.21.15.37:0/1019262283 conn(0x5628e34a7f80 msgr2=0x5628e34a9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa573e00640 0 client.25838 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 shutdown_connections 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 1 -- 172.21.15.37:0/1019262283 wait complete. 2024-03-21T09:13:04.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.607+0000 7fa59bc06740 20 client.25838 trim_cache size 0 max 16384 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 Processor -- start 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 -- start start 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34a9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941b2f30 con 0x5628e34ae260 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941b0a30 con 0x5628e34ad8b0 2024-03-21T09:13:04.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa59bc06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941aead0 con 0x5628e34b6de0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa591e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.623+0000 7fa592800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa591e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34a9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54222/0 (socket says 172.21.15.37:54222) 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa591e00640 1 -- 172.21.15.37:0/2558526872 learned_addr learned my addr 172.21.15.37:0/2558526872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa591400640 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa592800640 1 -- 172.21.15.37:0/2558526872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 msgr2=0x5628e34a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa592800640 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa592800640 1 -- 172.21.15.37:0/2558526872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 msgr2=0x5628e34a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa592800640 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa592800640 1 -- 172.21.15.37:0/2558526872 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628e34b20c0 con 0x5628e34b6de0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa591400640 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa592800640 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b6240 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa58c010d00 tx=0x7fa58c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524460b0c54271b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa57ea00640 1 -- 172.21.15.37:0/2558526872 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c016070 con 0x5628e34b6de0 2024-03-21T09:13:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa57ea00640 1 -- 172.21.15.37:0/2558526872 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa58c003b70 con 0x5628e34b6de0 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.624+0000 7fa57ea00640 1 -- 172.21.15.37:0/2558526872 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c005330 con 0x5628e34b6de0 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 -- 172.21.15.37:0/2558526872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 msgr2=0x5628e34b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b6240 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa58c010d00 tx=0x7fa58c009b60 comp rx=0 tx=0).stop 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 -- 172.21.15.37:0/2558526872 shutdown_connections 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34b6de0 0x5628e34b6240 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34ae260 0x5628e34a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 --2- 172.21.15.37:0/2558526872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ad8b0 0x5628e34a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 -- 172.21.15.37:0/2558526872 >> 172.21.15.37:0/2558526872 conn(0x5628e34a7f80 msgr2=0x5628e34a9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 -- 172.21.15.37:0/2558526872 shutdown_connections 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.625+0000 7fa59bc06740 1 -- 172.21.15.37:0/2558526872 wait complete. 2024-03-21T09:13:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.626+0000 7fa59bc06740 1 -- start start 2024-03-21T09:13:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.626+0000 7fa59bc06740 1 Processor -- start 2024-03-21T09:13:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.626+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34ad8b0 0x5628e34bccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.626+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 0x5628e34bd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.626+0000 7fa59bc06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34b6de0 0x5628e35215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa59bc06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941af0d0 con 0x5628e34b6de0 2024-03-21T09:13:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa59bc06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941b03d0 con 0x5628e34ae260 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa59bc06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941af710 con 0x5628e34ad8b0 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa59bc06740 10 client.0 _ll_register_callbacks cb 0x7fa584009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 0x5628e34bd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34ad8b0 0x5628e34bccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa592800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34b6de0 0x5628e35215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 0x5628e34bd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54226/0 (socket says 172.21.15.37:54226) 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 -- 172.21.15.37:0/4189740376 learned_addr learned my addr 172.21.15.37:0/4189740376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 -- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34ad8b0 msgr2=0x5628e34bccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34ad8b0 0x5628e34bccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 -- 172.21.15.37:0/4189740376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34b6de0 msgr2=0x5628e35215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34b6de0 0x5628e35215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 -- 172.21.15.37:0/4189740376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628e3521bd0 con 0x5628e34ae260 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa591400640 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 0x5628e34bd230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa5800bdc20 tx=0x7fa5800fa040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe7f80964fe535d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.627+0000 7fa573e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa573e00640 1 -- 172.21.15.37:0/4189740376 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5800bd7d0 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa573e00640 1 -- 172.21.15.37:0/4189740376 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa5800bbf90 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa58c003b70 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa573e00640 1 -- 172.21.15.37:0/4189740376 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5800db050 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 10 client.25844 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5628e3522f10 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa573e00640 1 -- 172.21.15.37:0/4189740376 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa58c003b70 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa573e00640 1 -- 172.21.15.37:0/4189740376 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa5800ba300 con 0x5628e34ae260 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 10 client.25844 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 10 client.25844 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 20 client.25844 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e35235a0 0x5628e35259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 4 client.25844 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.628+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628e3418b50 con 0x5628e35235a0 2024-03-21T09:13:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.629+0000 7fa591e00640 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e35235a0 0x5628e35259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.629+0000 7fa591e00640 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e35235a0 0x5628e35259c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c51a2e9f5f83aef server_cookie=5e9d57045d03dc73 in_seq=0 out_seq=0 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.629+0000 7fa573e00640 10 client.25844 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.629+0000 7fa573e00640 1 -- 172.21.15.37:0/4189740376 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628e3418b50 con 0x5628e35235a0 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.629+0000 7fa573e00640 10 client.25844 handle_command_reply: tid=0 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.629+0000 7fa59bc06740 1 client.25844 shutdown 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 msgr2=0x5628e34bd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 0x5628e34bd230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa5800bdc20 tx=0x7fa5800fa040 comp rx=0 tx=0).stop 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 shutdown_connections 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5628e35235a0 0x5628e35259c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e34b6de0 0x5628e35215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628e34ae260 0x5628e34bd230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 --2- 172.21.15.37:0/4189740376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628e34ad8b0 0x5628e34bccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa573e00640 0 client.25844 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 >> 172.21.15.37:0/4189740376 conn(0x5628e34a7f80 msgr2=0x5628e34a99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 shutdown_connections 2024-03-21T09:13:04.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 1 -- 172.21.15.37:0/4189740376 wait complete. 2024-03-21T09:13:04.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.630+0000 7fa59bc06740 20 client.25844 trim_cache size 0 max 16384 2024-03-21T09:13:04.628 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5780a3230 msgr2=0x7fa5780a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5780a3230 0x7fa5780a56f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x5628e34a8ff0 tx=0x7fa584005f90 comp rx=0 tx=0).stop 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 msgr2=0x7fa59419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59419a4e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa580036040 tx=0x7fa58002cbc0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 shutdown_connections 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59410dca0 0x7fa59419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5940743f0 0x7fa59419a4e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5780a3230 0x7fa5780a56f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 --2- 172.21.15.37:0/2455991099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa594073ad0 0x7fa594199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 >> 172.21.15.37:0/2455991099 conn(0x7fa5940fcff0 msgr2=0x7fa5940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 shutdown_connections 2024-03-21T09:13:04.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.631+0000 7fa593200640 1 -- 172.21.15.37:0/2455991099 wait complete. 2024-03-21T09:13:04.648 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:13:04.648 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:13:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 -- 172.21.15.37:0/1121348391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 msgr2=0x7f304c103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 --2- 172.21.15.37:0/1121348391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c103b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3034009930 tx=0x7f303402bc20 comp rx=0 tx=0).stop 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 -- 172.21.15.37:0/1121348391 shutdown_connections 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 --2- 172.21.15.37:0/1121348391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 0x7f304c110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 --2- 172.21.15.37:0/1121348391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c103b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 --2- 172.21.15.37:0/1121348391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f304c109690 0x7f304c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 -- 172.21.15.37:0/1121348391 >> 172.21.15.37:0/1121348391 conn(0x7f304c0ff110 msgr2=0x7f304c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 -- 172.21.15.37:0/1121348391 shutdown_connections 2024-03-21T09:13:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.906+0000 7f3051400640 1 -- 172.21.15.37:0/1121348391 wait complete. 2024-03-21T09:13:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 Processor -- start 2024-03-21T09:13:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 -- start start 2024-03-21T09:13:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 0x7f304c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f304c109690 0x7f304c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c113af0 con 0x7f304c109690 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c113970 con 0x7f304c104040 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3051400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c113c70 con 0x7f304c103680 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f304be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.908+0000 7f3043e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 0x7f304c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f304be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58408/0 (socket says 172.21.15.37:58408) 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f304be00640 1 -- 172.21.15.37:0/16366259 learned_addr learned my addr 172.21.15.37:0/16366259 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3050a00640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f304c109690 0x7f304c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3043e00640 1 -- 172.21.15.37:0/16366259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 msgr2=0x7f304c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3043e00640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3043e00640 1 -- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f304c109690 msgr2=0x7f304c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3043e00640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f304c109690 0x7f304c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3043e00640 1 -- 172.21.15.37:0/16366259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c19efa0 con 0x7f304c104040 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.909+0000 7f3043e00640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 0x7f304c19a4f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3034036040 tx=0x7f3034004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.910+0000 7f3049600640 1 -- 172.21.15.37:0/16366259 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034043020 con 0x7f304c104040 2024-03-21T09:13:04.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.910+0000 7f3049600640 1 -- 172.21.15.37:0/16366259 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3034005000 con 0x7f304c104040 2024-03-21T09:13:04.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.910+0000 7f3049600640 1 -- 172.21.15.37:0/16366259 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034003a80 con 0x7f304c104040 2024-03-21T09:13:04.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.910+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c19f230 con 0x7f304c104040 2024-03-21T09:13:04.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.910+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c1a6b20 con 0x7f304c104040 2024-03-21T09:13:04.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.913+0000 7f3049600640 1 -- 172.21.15.37:0/16366259 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f3034046020 con 0x7f304c104040 2024-03-21T09:13:04.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 Processor -- start 2024-03-21T09:13:04.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 -- start start 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228f97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228fa0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fa3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228efb2c0 con 0x556228fa0de0 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228ea3c90 con 0x556228f978b0 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f30557d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228ef7360 con 0x556228f98260 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f3050a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228f97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f3050a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228f97c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54266/0 (socket says 172.21.15.37:54266) 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f3050a00640 1 -- 172.21.15.37:0/98311483 learned_addr learned my addr 172.21.15.37:0/98311483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.914+0000 7f304be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228fa0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3043e00640 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fa3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3050a00640 1 -- 172.21.15.37:0/98311483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 msgr2=0x556228fa0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3050a00640 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228fa0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3050a00640 1 -- 172.21.15.37:0/98311483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 msgr2=0x556228fa3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3050a00640 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fa3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3050a00640 1 -- 172.21.15.37:0/98311483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556228ec8080 con 0x556228f978b0 2024-03-21T09:13:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f304be00640 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228fa0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3050a00640 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228f97c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3044009fc0 tx=0x7f3044010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9addeb58d396fd20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.915+0000 7f3049600640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f30300a3230 0x7f30300a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.916+0000 7f3049600640 1 -- 172.21.15.37:0/16366259 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f30340f1bd0 con 0x7f304c104040 2024-03-21T09:13:04.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.916+0000 7f3041200640 1 -- 172.21.15.37:0/98311483 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044003b70 con 0x556228f978b0 2024-03-21T09:13:04.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.916+0000 7f304be00640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f30300a3230 0x7f30300a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.916+0000 7f3041200640 1 -- 172.21.15.37:0/98311483 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3044003d10 con 0x556228f978b0 2024-03-21T09:13:04.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.916+0000 7f3041200640 1 -- 172.21.15.37:0/98311483 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044011aa0 con 0x556228f978b0 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f304be00640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f30300a3230 0x7f30300a56f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x556228f92ff0 tx=0x7f303c005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 -- 172.21.15.37:0/98311483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 msgr2=0x556228f97c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228f97c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3044009fc0 tx=0x7f3044010860 comp rx=0 tx=0).stop 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 -- 172.21.15.37:0/98311483 shutdown_connections 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fa3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228fa0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 --2- 172.21.15.37:0/98311483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228f97c90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 -- 172.21.15.37:0/98311483 >> 172.21.15.37:0/98311483 conn(0x556228f91f80 msgr2=0x556228f92360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 -- 172.21.15.37:0/98311483 shutdown_connections 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.917+0000 7f30557d5740 1 -- 172.21.15.37:0/98311483 wait complete. 2024-03-21T09:13:04.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.918+0000 7f30557d5740 1 -- start start 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.918+0000 7f30557d5740 1 Processor -- start 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fa06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228f9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228f9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228fac410 con 0x556228fa0de0 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228eb70d0 con 0x556228f978b0 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556228fac590 con 0x556228f98260 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f30557d5740 10 client.0 _ll_register_callbacks cb 0x556228ebbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f304be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fa06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f304be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fa06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54268/0 (socket says 172.21.15.37:54268) 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f304be00640 1 -- 172.21.15.37:0/596633992 learned_addr learned my addr 172.21.15.37:0/596633992 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f3043e00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228f9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.919+0000 7f3050a00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228f9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f304be00640 1 -- 172.21.15.37:0/596633992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 msgr2=0x556228f9da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f304be00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228f9da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f304be00640 1 -- 172.21.15.37:0/596633992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 msgr2=0x556228f9e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f304be00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228f9e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f304be00640 1 -- 172.21.15.37:0/596633992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556228c0f3f0 con 0x556228f978b0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f304be00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fa06f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f303c0045f0 tx=0x7f303c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163985ecdc2955f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f3039600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f3039600640 1 -- 172.21.15.37:0/596633992 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c009c50 con 0x556228f978b0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3044003d10 con 0x556228f978b0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.920+0000 7f3039600640 1 -- 172.21.15.37:0/596633992 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f303c01f640 con 0x556228f978b0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.921+0000 7f3039600640 1 -- 172.21.15.37:0/596633992 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f303c01f840 con 0x556228f978b0 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.921+0000 7f30557d5740 10 client.25856 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:04.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.921+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556228cd97d0 con 0x556228f978b0 2024-03-21T09:13:04.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.921+0000 7f3039600640 1 -- 172.21.15.37:0/596633992 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3044003d10 con 0x556228f978b0 2024-03-21T09:13:04.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f3039600640 1 -- 172.21.15.37:0/596633992 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f303c027070 con 0x556228f978b0 2024-03-21T09:13:04.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f30557d5740 10 client.25856 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:04.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f30557d5740 10 client.25856 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f30557d5740 20 client.25856 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f30557d5740 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556228fb9510 0x556228fbb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f30557d5740 4 client.25856 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556228f08800 con 0x556228fb9510 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.922+0000 7f3043e00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556228fb9510 0x556228fbb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.923+0000 7f3043e00640 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556228fb9510 0x556228fbb920 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cf2d21cdfb65533 server_cookie=756fe102327d12ee in_seq=0 out_seq=0 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.923+0000 7f3039600640 10 client.25856 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.923+0000 7f3039600640 1 -- 172.21.15.37:0/596633992 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556228f08800 con 0x556228fb9510 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.923+0000 7f3039600640 10 client.25856 handle_command_reply: tid=0 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.923+0000 7f30557d5740 1 client.25856 shutdown 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 msgr2=0x556228fa06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fa06f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f303c0045f0 tx=0x7f303c004730 comp rx=0 tx=0).stop 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 shutdown_connections 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556228fb9510 0x556228fbb920 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228f9e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228f9da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f3039600640 0 client.25856 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 --2- 172.21.15.37:0/596633992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fa06f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 >> 172.21.15.37:0/596633992 conn(0x556228f91f80 msgr2=0x556228f93cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 shutdown_connections 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.924+0000 7f30557d5740 1 -- 172.21.15.37:0/596633992 wait complete. 2024-03-21T09:13:04.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.925+0000 7f30557d5740 20 client.25856 trim_cache size 0 max 16384 2024-03-21T09:13:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 Processor -- start 2024-03-21T09:13:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 -- start start 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 0x556228fa01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 0x556228fa06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228fa0de0 0x556228f9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1aeb90 con 0x556228f98260 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1ac690 con 0x556228fa0de0 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f30557d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1aa730 con 0x556228f978b0 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f3043e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 0x556228fa06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f304be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 0x556228fa01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f3050a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228fa0de0 0x556228f9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f3043e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 0x556228fa06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33506/0 (socket says 172.21.15.37:33506) 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f3043e00640 1 -- 172.21.15.37:0/2676748681 learned_addr learned my addr 172.21.15.37:0/2676748681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f304be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 0x556228fa01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58436/0 (socket says 172.21.15.37:58436) 2024-03-21T09:13:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.941+0000 7f3050a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228fa0de0 0x556228f9ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54274/0 (socket says 172.21.15.37:54274) 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f304be00640 1 -- 172.21.15.37:0/2676748681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228fa0de0 msgr2=0x556228f9ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f304be00640 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228fa0de0 0x556228f9ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f304be00640 1 -- 172.21.15.37:0/2676748681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 msgr2=0x556228fa06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f304be00640 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 0x556228fa06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f304be00640 1 -- 172.21.15.37:0/2676748681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556228f9e390 con 0x556228f978b0 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f3043e00640 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 0x556228fa06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f304be00640 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 0x556228fa01b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f303c019040 tx=0x7f303c0092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6ba67f43553363 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f303be00640 1 -- 172.21.15.37:0/2676748681 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c01fbe0 con 0x556228f978b0 2024-03-21T09:13:04.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f303be00640 1 -- 172.21.15.37:0/2676748681 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f303c01fd80 con 0x556228f978b0 2024-03-21T09:13:04.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.942+0000 7f303be00640 1 -- 172.21.15.37:0/2676748681 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c004d50 con 0x556228f978b0 2024-03-21T09:13:04.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 -- 172.21.15.37:0/2676748681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 msgr2=0x556228fa01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 0x556228fa01b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f303c019040 tx=0x7f303c0092a0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 -- 172.21.15.37:0/2676748681 shutdown_connections 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228fa0de0 0x556228f9ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228f98260 0x556228fa06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 --2- 172.21.15.37:0/2676748681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f978b0 0x556228fa01b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 -- 172.21.15.37:0/2676748681 >> 172.21.15.37:0/2676748681 conn(0x556228f91f80 msgr2=0x556228f91160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 -- 172.21.15.37:0/2676748681 shutdown_connections 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.943+0000 7f30557d5740 1 -- 172.21.15.37:0/2676748681 wait complete. 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.944+0000 7f30557d5740 1 -- start start 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.944+0000 7f30557d5740 1 Processor -- start 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fb1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228faf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fafdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1aad30 con 0x556228fa0de0 2024-03-21T09:13:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1ac030 con 0x556228f978b0 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1ab370 con 0x556228f98260 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f30557d5740 10 client.0 _ll_register_callbacks cb 0x7f303c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f304be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fb1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3043e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228faf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3050a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fafdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f304be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fb1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54290/0 (socket says 172.21.15.37:54290) 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f304be00640 1 -- 172.21.15.37:0/1493285926 learned_addr learned my addr 172.21.15.37:0/1493285926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3043e00640 1 -- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 msgr2=0x556228fb1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3043e00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fb1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3043e00640 1 -- 172.21.15.37:0/1493285926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 msgr2=0x556228fafdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3043e00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fafdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3043e00640 1 -- 172.21.15.37:0/1493285926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556228fb0360 con 0x556228f98260 2024-03-21T09:13:04.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f304be00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fb1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.945+0000 7f3050a00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fafdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f3043e00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228faf8b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30340bd780 tx=0x7f30340f5260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e2489e0c952e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f3039600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f3039600640 1 -- 172.21.15.37:0/1493285926 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30340be180 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f3039600640 1 -- 172.21.15.37:0/1493285926 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f30340ba0f0 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f303c01fd80 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f3039600640 1 -- 172.21.15.37:0/1493285926 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30340d4020 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f30557d5740 10 client.25888 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556229015bd0 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.946+0000 7f3039600640 1 -- 172.21.15.37:0/1493285926 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f303c01fd80 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f3039600640 1 -- 172.21.15.37:0/1493285926 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f30340c6000 con 0x556228f98260 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f30557d5740 10 client.25888 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f30557d5740 10 client.25888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:13:04.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f30557d5740 20 client.25888 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:04.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f30557d5740 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556229016350 0x556229018770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:04.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f30557d5740 4 client.25888 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:13:04.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556228f02b50 con 0x556229016350 2024-03-21T09:13:04.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.947+0000 7f304be00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556229016350 0x556229018770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:04.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.948+0000 7f304be00640 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556229016350 0x556229018770 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b511b10c64b9187 server_cookie=8a5486863d47ce2b in_seq=0 out_seq=0 2024-03-21T09:13:04.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.948+0000 7f3039600640 10 client.25888 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:04.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.948+0000 7f3039600640 1 -- 172.21.15.37:0/1493285926 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556228f02b50 con 0x556229016350 2024-03-21T09:13:04.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.948+0000 7f3039600640 10 client.25888 handle_command_reply: tid=0 2024-03-21T09:13:04.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.948+0000 7f30557d5740 1 client.25888 shutdown 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 msgr2=0x556228faf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228faf8b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30340bd780 tx=0x7f30340f5260 comp rx=0 tx=0).stop 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 shutdown_connections 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x556229016350 0x556229018770 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556228fa0de0 0x556228fafdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f3039600640 0 client.25888 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556228f98260 0x556228faf8b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 --2- 172.21.15.37:0/1493285926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556228f978b0 0x556228fb1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 >> 172.21.15.37:0/1493285926 conn(0x556228f91f80 msgr2=0x556228f933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 shutdown_connections 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 1 -- 172.21.15.37:0/1493285926 wait complete. 2024-03-21T09:13:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.949+0000 7f30557d5740 20 client.25888 trim_cache size 0 max 16384 2024-03-21T09:13:04.947 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:13:04.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f30300a3230 msgr2=0x7f30300a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f30300a3230 0x7f30300a56f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x556228f92ff0 tx=0x7f303c005f90 comp rx=0 tx=0).stop 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 msgr2=0x7f304c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 0x7f304c19a4f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3034036040 tx=0x7f3034004250 comp rx=0 tx=0).stop 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 shutdown_connections 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f304c109690 0x7f304c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f30300a3230 0x7f30300a56f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f304c104040 0x7f304c19a4f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 --2- 172.21.15.37:0/16366259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304c103680 0x7f304c199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.950+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 >> 172.21.15.37:0/16366259 conn(0x7f304c0ff110 msgr2=0x7f304c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.951+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 shutdown_connections 2024-03-21T09:13:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:04.951+0000 7f3051400640 1 -- 172.21.15.37:0/16366259 wait complete. 2024-03-21T09:13:04.970 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:13:04.971 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:13:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 -- 172.21.15.37:0/2758024200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720107560 msgr2=0x7fc720107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 --2- 172.21.15.37:0/2758024200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720107560 0x7fc720107940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc714009860 tx=0x7fc71402bb70 comp rx=0 tx=0).stop 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 -- 172.21.15.37:0/2758024200 shutdown_connections 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 --2- 172.21.15.37:0/2758024200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc720101e60 0x7fc72010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 --2- 172.21.15.37:0/2758024200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc720101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 --2- 172.21.15.37:0/2758024200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720107560 0x7fc720107940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.208+0000 7fc725e00640 1 -- 172.21.15.37:0/2758024200 >> 172.21.15.37:0/2758024200 conn(0x7fc7200fcfb0 msgr2=0x7fc7200ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.209+0000 7fc725e00640 1 -- 172.21.15.37:0/2758024200 shutdown_connections 2024-03-21T09:13:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.209+0000 7fc725e00640 1 -- 172.21.15.37:0/2758024200 wait complete. 2024-03-21T09:13:05.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.210+0000 7fc725e00640 1 Processor -- start 2024-03-21T09:13:05.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.210+0000 7fc725e00640 1 -- start start 2024-03-21T09:13:05.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc72019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720101e60 0x7fc72019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc720107560 0x7fc720197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720111820 con 0x7fc720101e60 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7201116a0 con 0x7fc720107560 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7201119a0 con 0x7fc7201014a0 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc72019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc72019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58464/0 (socket says 172.21.15.37:58464) 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 -- 172.21.15.37:0/4141677620 learned_addr learned my addr 172.21.15.37:0/4141677620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc71fe00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720101e60 0x7fc72019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc725400640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc720107560 0x7fc720197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 -- 172.21.15.37:0/4141677620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc720107560 msgr2=0x7fc720197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc720107560 0x7fc720197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 -- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720101e60 msgr2=0x7fc72019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720101e60 0x7fc72019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc724a00640 1 -- 172.21.15.37:0/4141677620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720198420 con 0x7fc7201014a0 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.211+0000 7fc71fe00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720101e60 0x7fc72019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:05.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.212+0000 7fc724a00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc72019da50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc714037040 tx=0x7fc714004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.212+0000 7fc71d600640 1 -- 172.21.15.37:0/4141677620 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714034850 con 0x7fc7201014a0 2024-03-21T09:13:05.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.212+0000 7fc71d600640 1 -- 172.21.15.37:0/4141677620 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc71402c9f0 con 0x7fc7201014a0 2024-03-21T09:13:05.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.212+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7201986b0 con 0x7fc7201014a0 2024-03-21T09:13:05.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.212+0000 7fc71d600640 1 -- 172.21.15.37:0/4141677620 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140035f0 con 0x7fc7201014a0 2024-03-21T09:13:05.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.212+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720073ab0 con 0x7fc7201014a0 2024-03-21T09:13:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.216+0000 7fc72a1fa740 1 Processor -- start 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 -- start start 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc71d600640 1 -- 172.21.15.37:0/4141677620 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fc7140326d0 con 0x7fc7201014a0 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a90dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90e260 0x56462a916810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a916de0 0x56462a919230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462a8712c0 con 0x56462a916de0 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462a819c90 con 0x56462a90d8b0 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462a86d360 con 0x56462a90e260 2024-03-21T09:13:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.217+0000 7fc725400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a90dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a90dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54328/0 (socket says 172.21.15.37:54328) 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 -- 172.21.15.37:0/3782682968 learned_addr learned my addr 172.21.15.37:0/3782682968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc71fe00640 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a916de0 0x56462a919230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 -- 172.21.15.37:0/3782682968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90e260 msgr2=0x56462a916810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90e260 0x56462a916810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 -- 172.21.15.37:0/3782682968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a916de0 msgr2=0x56462a919230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a916de0 0x56462a919230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc725400640 1 -- 172.21.15.37:0/3782682968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56462a83e080 con 0x56462a90d8b0 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.218+0000 7fc71fe00640 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a916de0 0x56462a919230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.219+0000 7fc725400640 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a90dc90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc718011440 tx=0x7fc718011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4ba922b96d96e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.219+0000 7fc711c00640 1 -- 172.21.15.37:0/3782682968 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718003c30 con 0x56462a90d8b0 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.219+0000 7fc711c00640 1 -- 172.21.15.37:0/3782682968 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc718003dd0 con 0x56462a90d8b0 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.219+0000 7fc711c00640 1 -- 172.21.15.37:0/3782682968 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71800fb50 con 0x56462a90d8b0 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.219+0000 7fc71d600640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7080a32b0 0x7fc7080a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3782682968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 msgr2=0x56462a90dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a90dc90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc718011440 tx=0x7fc718011900 comp rx=0 tx=0).stop 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc724a00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7080a32b0 0x7fc7080a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc71d600640 1 -- 172.21.15.37:0/4141677620 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc7140f58d0 con 0x7fc7201014a0 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3782682968 shutdown_connections 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a916de0 0x56462a919230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90e260 0x56462a916810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3782682968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a90dc90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.220+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3782682968 >> 172.21.15.37:0/3782682968 conn(0x56462a907f80 msgr2=0x56462a908360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.221+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3782682968 shutdown_connections 2024-03-21T09:13:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.221+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3782682968 wait complete. 2024-03-21T09:13:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.221+0000 7fc724a00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7080a32b0 0x7fc7080a5770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc7140f2c10 tx=0x7fc714005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.221+0000 7fc72a1fa740 1 -- start start 2024-03-21T09:13:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 Processor -- start 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90d8b0 0x56462a916720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 0x56462a913ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a9140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462a92ad60 con 0x56462a90e260 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462a82d0d0 con 0x56462a916de0 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462a92aee0 con 0x56462a90d8b0 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.222+0000 7fc72a1fa740 10 client.0 _ll_register_callbacks cb 0x56462a831e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc725400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90d8b0 0x56462a916720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 0x56462a913ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc725400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90d8b0 0x56462a916720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58484/0 (socket says 172.21.15.37:58484) 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc725400640 1 -- 172.21.15.37:0/2617417619 learned_addr learned my addr 172.21.15.37:0/2617417619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 0x56462a913ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33550/0 (socket says 172.21.15.37:33550) 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc725400640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a9140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 -- 172.21.15.37:0/2617417619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90d8b0 msgr2=0x56462a916720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90d8b0 0x56462a916720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 -- 172.21.15.37:0/2617417619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 msgr2=0x56462a9140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a9140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.223+0000 7fc71fe00640 1 -- 172.21.15.37:0/2617417619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56462a5853f0 con 0x56462a90e260 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.224+0000 7fc725400640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a9140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.224+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 0x56462a913ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc70c005fd0 tx=0x7fc70c010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebc31280f91121c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.224+0000 7fc706000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.224+0000 7fc706000640 1 -- 172.21.15.37:0/2617417619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004030 con 0x56462a90e260 2024-03-21T09:13:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.224+0000 7fc706000640 1 -- 172.21.15.37:0/2617417619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc70c0041d0 con 0x56462a90e260 2024-03-21T09:13:05.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.224+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc718003dd0 con 0x56462a90e260 2024-03-21T09:13:05.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.225+0000 7fc706000640 1 -- 172.21.15.37:0/2617417619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70c0043d0 con 0x56462a90e260 2024-03-21T09:13:05.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.225+0000 7fc72a1fa740 10 client.15741 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:05.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.225+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56462a64f7d0 con 0x56462a90e260 2024-03-21T09:13:05.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.225+0000 7fc706000640 1 -- 172.21.15.37:0/2617417619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc718003dd0 con 0x56462a90e260 2024-03-21T09:13:05.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.225+0000 7fc706000640 1 -- 172.21.15.37:0/2617417619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc70c003bf0 con 0x56462a90e260 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc72a1fa740 10 client.15741 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc72a1fa740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc72a1fa740 20 client.15741 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a937d40 0x56462a93a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc72a1fa740 4 client.15741 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56462a87e800 con 0x56462a937d40 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.226+0000 7fc725400640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a937d40 0x56462a93a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.227+0000 7fc725400640 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a937d40 0x56462a93a160 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ae64b461f2861a server_cookie=7d1bb57357de5676 in_seq=0 out_seq=0 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.227+0000 7fc706000640 10 client.15741 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc706000640 1 -- 172.21.15.37:0/2617417619 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56462a87e800 con 0x56462a937d40 2024-03-21T09:13:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc706000640 10 client.15741 handle_command_reply: tid=0 2024-03-21T09:13:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 client.15741 shutdown 2024-03-21T09:13:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 msgr2=0x56462a913ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 0x56462a913ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc70c005fd0 tx=0x7fc70c010b60 comp rx=0 tx=0).stop 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 shutdown_connections 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a937d40 0x56462a93a160 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a9140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90e260 0x56462a913ab0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2617417619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a90d8b0 0x56462a916720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc706000640 0 client.15741 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.228+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 >> 172.21.15.37:0/2617417619 conn(0x56462a907f80 msgr2=0x56462a909cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.229+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 shutdown_connections 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.229+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2617417619 wait complete. 2024-03-21T09:13:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.229+0000 7fc72a1fa740 20 client.15741 trim_cache size 0 max 16384 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.252+0000 7fc72a1fa740 1 Processor -- start 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.252+0000 7fc72a1fa740 1 -- start start 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.252+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90d8b0 0x56462a9161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.252+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a916720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.252+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a93f9f0 0x56462a910db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7201a6320 con 0x56462a90d8b0 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720072f00 con 0x56462a916de0 2024-03-21T09:13:05.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720070fa0 con 0x56462a93f9f0 2024-03-21T09:13:05.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc725400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90d8b0 0x56462a9161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a916720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a916720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54346/0 (socket says 172.21.15.37:54346) 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 -- 172.21.15.37:0/3747310034 learned_addr learned my addr 172.21.15.37:0/3747310034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc725400640 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a93f9f0 0x56462a910db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 -- 172.21.15.37:0/3747310034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a93f9f0 msgr2=0x56462a910db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a93f9f0 0x56462a910db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 -- 172.21.15.37:0/3747310034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90d8b0 msgr2=0x56462a9161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90d8b0 0x56462a9161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 -- 172.21.15.37:0/3747310034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56462a90cbd0 con 0x56462a916de0 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc725400640 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a93f9f0 0x56462a910db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.253+0000 7fc71fe00640 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a916720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc70c024490 tx=0x7fc70c00f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4538fa15aa729851 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.254+0000 7fc711200640 1 -- 172.21.15.37:0/3747310034 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c010d50 con 0x56462a916de0 2024-03-21T09:13:05.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.254+0000 7fc711200640 1 -- 172.21.15.37:0/3747310034 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc70c004000 con 0x56462a916de0 2024-03-21T09:13:05.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.254+0000 7fc711200640 1 -- 172.21.15.37:0/3747310034 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0055b0 con 0x56462a916de0 2024-03-21T09:13:05.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3747310034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 msgr2=0x56462a916720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a916720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc70c024490 tx=0x7fc70c00f810 comp rx=0 tx=0).stop 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3747310034 shutdown_connections 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a93f9f0 0x56462a910db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a916de0 0x56462a916720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/3747310034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a90d8b0 0x56462a9161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3747310034 >> 172.21.15.37:0/3747310034 conn(0x56462a907f80 msgr2=0x56462a907160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3747310034 shutdown_connections 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.255+0000 7fc72a1fa740 1 -- 172.21.15.37:0/3747310034 wait complete. 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.256+0000 7fc72a1fa740 1 -- start start 2024-03-21T09:13:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.256+0000 7fc72a1fa740 1 Processor -- start 2024-03-21T09:13:05.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.256+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a92fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a916de0 0x56462a92e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc72a1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a93f9f0 0x56462a92e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200715a0 con 0x56462a93f9f0 2024-03-21T09:13:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200728a0 con 0x56462a90d8b0 2024-03-21T09:13:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc72a1fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720071be0 con 0x56462a916de0 2024-03-21T09:13:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc72a1fa740 10 client.0 _ll_register_callbacks cb 0x7fc70c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a92fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a92fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54354/0 (socket says 172.21.15.37:54354) 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 -- 172.21.15.37:0/2301284259 learned_addr learned my addr 172.21.15.37:0/2301284259 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a93f9f0 0x56462a92e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a916de0 0x56462a92e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 -- 172.21.15.37:0/2301284259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a916de0 msgr2=0x56462a92e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a916de0 0x56462a92e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 -- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a93f9f0 msgr2=0x56462a92e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a93f9f0 0x56462a92e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.257+0000 7fc725400640 1 -- 172.21.15.37:0/2301284259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56462a92efd0 con 0x56462a90d8b0 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc725400640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a92fe50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc71800f7c0 tx=0x7fc718009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b4eba6a6e39dad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc706000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc706000640 1 -- 172.21.15.37:0/2301284259 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71802dcd0 con 0x56462a90d8b0 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc706000640 1 -- 172.21.15.37:0/2301284259 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc71800f820 con 0x56462a90d8b0 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc70c004000 con 0x56462a90d8b0 2024-03-21T09:13:05.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc706000640 1 -- 172.21.15.37:0/2301284259 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc71800fa20 con 0x56462a90d8b0 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc72a1fa740 10 client.25880 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.258+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56462a994510 con 0x56462a90d8b0 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc706000640 1 -- 172.21.15.37:0/2301284259 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc70c004000 con 0x56462a90d8b0 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc706000640 1 -- 172.21.15.37:0/2301284259 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc718025040 con 0x56462a90d8b0 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc72a1fa740 10 client.25880 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc72a1fa740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc72a1fa740 20 client.25880 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a994ab0 0x56462a996ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc72a1fa740 4 client.25880 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56462a878b50 con 0x56462a994ab0 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.259+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a994ab0 0x56462a996ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc71fe00640 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a994ab0 0x56462a996ed0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806e5868972556fd server_cookie=c04a67a7a89002a6 in_seq=0 out_seq=0 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc706000640 10 client.25880 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc706000640 1 -- 172.21.15.37:0/2301284259 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56462a878b50 con 0x56462a994ab0 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc706000640 10 client.25880 handle_command_reply: tid=0 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 client.25880 shutdown 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 msgr2=0x56462a92fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a92fe50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc71800f7c0 tx=0x7fc718009500 comp rx=0 tx=0).stop 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 shutdown_connections 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56462a93f9f0 0x56462a92e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56462a916de0 0x56462a92e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56462a994ab0 0x56462a996ed0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 --2- 172.21.15.37:0/2301284259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56462a90d8b0 0x56462a92fe50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 >> 172.21.15.37:0/2301284259 conn(0x56462a907f80 msgr2=0x56462a93e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.260+0000 7fc706000640 0 client.25880 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.261+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 shutdown_connections 2024-03-21T09:13:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.261+0000 7fc72a1fa740 1 -- 172.21.15.37:0/2301284259 wait complete. 2024-03-21T09:13:05.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.261+0000 7fc72a1fa740 20 client.25880 trim_cache size 0 max 16384 2024-03-21T09:13:05.265 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:13:05.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7080a32b0 msgr2=0x7fc7080a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7080a32b0 0x7fc7080a5770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc7140f2c10 tx=0x7fc714005f90 comp rx=0 tx=0).stop 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 msgr2=0x7fc72019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc72019da50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc714037040 tx=0x7fc714004590 comp rx=0 tx=0).stop 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 shutdown_connections 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7080a32b0 0x7fc7080a5770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc720107560 0x7fc720197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc720101e60 0x7fc72019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 --2- 172.21.15.37:0/4141677620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7201014a0 0x7fc72019da50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 >> 172.21.15.37:0/4141677620 conn(0x7fc7200fcfb0 msgr2=0x7fc7200fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 shutdown_connections 2024-03-21T09:13:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:05.262+0000 7fc725e00640 1 -- 172.21.15.37:0/4141677620 wait complete. 2024-03-21T09:13:05.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:05 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1391148013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:13:05.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:05 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1391148013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:13:05.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:05 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1391148013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:13:06.275 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:13:06.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:13:06.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:13:06.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:06 smithi037 ceph-mon[31830]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-21T09:13:06.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:06 smithi053 ceph-mon[36169]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-21T09:13:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 -- 172.21.15.37:0/2197549270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920101e60 msgr2=0x7fd92010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 --2- 172.21.15.37:0/2197549270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920101e60 0x7fd92010e320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd918009c10 tx=0x7fd91802bee0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 -- 172.21.15.37:0/2197549270 shutdown_connections 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 --2- 172.21.15.37:0/2197549270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920101e60 0x7fd92010e320 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 --2- 172.21.15.37:0/2197549270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9201014a0 0x7fd920101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 --2- 172.21.15.37:0/2197549270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd920107560 0x7fd920107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 -- 172.21.15.37:0/2197549270 >> 172.21.15.37:0/2197549270 conn(0x7fd9200fcfb0 msgr2=0x7fd9200ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 -- 172.21.15.37:0/2197549270 shutdown_connections 2024-03-21T09:13:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.532+0000 7fd925000640 1 -- 172.21.15.37:0/2197549270 wait complete. 2024-03-21T09:13:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.533+0000 7fd925000640 1 Processor -- start 2024-03-21T09:13:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.533+0000 7fd925000640 1 -- start start 2024-03-21T09:13:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd925000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9201014a0 0x7fd9200739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd925000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 0x7fd920073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd925000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920107560 0x7fd920074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd925000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920111890 con 0x7fd9201014a0 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd925000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920111710 con 0x7fd920107560 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd925000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920111a10 con 0x7fd920101e60 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 0x7fd920073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd91f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9201014a0 0x7fd9200739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 0x7fd920073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58538/0 (socket says 172.21.15.37:58538) 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd91ea00640 1 -- 172.21.15.37:0/3477496638 learned_addr learned my addr 172.21.15.37:0/3477496638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.534+0000 7fd91fe00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920107560 0x7fd920074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd91ea00640 1 -- 172.21.15.37:0/3477496638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920107560 msgr2=0x7fd920074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd91ea00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920107560 0x7fd920074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd91ea00640 1 -- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9201014a0 msgr2=0x7fd9200739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd91ea00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9201014a0 0x7fd9200739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd91ea00640 1 -- 172.21.15.37:0/3477496638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9201a61b0 con 0x7fd920101e60 2024-03-21T09:13:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd91ea00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 0x7fd920073ee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9f0 tx=0x7fd90c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd917e00640 1 -- 172.21.15.37:0/3477496638 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00ccc0 con 0x7fd920101e60 2024-03-21T09:13:06.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd917e00640 1 -- 172.21.15.37:0/3477496638 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c005500 con 0x7fd920101e60 2024-03-21T09:13:06.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd917e00640 1 -- 172.21.15.37:0/3477496638 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c010690 con 0x7fd920101e60 2024-03-21T09:13:06.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.535+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9201a64a0 con 0x7fd920101e60 2024-03-21T09:13:06.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.536+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9201a69a0 con 0x7fd920101e60 2024-03-21T09:13:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.540+0000 7fd917e00640 1 -- 172.21.15.37:0/3477496638 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fd90c0046c0 con 0x7fd920101e60 2024-03-21T09:13:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.540+0000 7fd9293af740 1 Processor -- start 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.541+0000 7fd9293af740 1 -- start start 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.541+0000 7fd917e00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd9040a31b0 0x7fd9040a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.541+0000 7fd9293af740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb737e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.541+0000 7fd9293af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb740a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.541+0000 7fd917e00640 1 -- 172.21.15.37:0/3477496638 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd90c0d0150 con 0x7fd920101e60 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.541+0000 7fd9293af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb743390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd9293af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eb693eb0 con 0x55f6eb740fa0 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd9293af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eb63ca60 con 0x55f6eb7383c0 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd9293af740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eb68ff50 con 0x55f6eb737aa0 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd91fe00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd9040a31b0 0x7fd9040a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd91f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb737e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb743390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb737e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58550/0 (socket says 172.21.15.37:58550) 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91f400640 1 -- 172.21.15.37:0/4237686844 learned_addr learned my addr 172.21.15.37:0/4237686844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.542+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb740a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91ea00640 1 -- 172.21.15.37:0/4237686844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 msgr2=0x55f6eb737e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91ea00640 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb737e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91ea00640 1 -- 172.21.15.37:0/4237686844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 msgr2=0x55f6eb743390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91ea00640 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb743390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91ea00640 1 -- 172.21.15.37:0/4237686844 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6eb4707e0 con 0x55f6eb7383c0 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.543+0000 7fd91fe00640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd9040a31b0 0x7fd9040a5670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd918033420 tx=0x7fd918033fb0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.544+0000 7fd91ea00640 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb740a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0cd320 tx=0x7fd90c09b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe55853347fbc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.544+0000 7fd914c00640 1 -- 172.21.15.37:0/4237686844 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c09bef0 con 0x55f6eb7383c0 2024-03-21T09:13:06.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.544+0000 7fd914c00640 1 -- 172.21.15.37:0/4237686844 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0a0040 con 0x55f6eb7383c0 2024-03-21T09:13:06.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.544+0000 7fd914c00640 1 -- 172.21.15.37:0/4237686844 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0cd600 con 0x55f6eb7383c0 2024-03-21T09:13:06.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 -- 172.21.15.37:0/4237686844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 msgr2=0x55f6eb740a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb740a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0cd320 tx=0x7fd90c09b850 comp rx=0 tx=0).stop 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 -- 172.21.15.37:0/4237686844 shutdown_connections 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb743390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb740a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 --2- 172.21.15.37:0/4237686844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb737e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 -- 172.21.15.37:0/4237686844 >> 172.21.15.37:0/4237686844 conn(0x55f6eb7321d0 msgr2=0x55f6eb7325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 -- 172.21.15.37:0/4237686844 shutdown_connections 2024-03-21T09:13:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 -- 172.21.15.37:0/4237686844 wait complete. 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.545+0000 7fd9293af740 1 -- start start 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 Processor -- start 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb740970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb73e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eb754da0 con 0x55f6eb740fa0 2024-03-21T09:13:06.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eb64feb0 con 0x55f6eb7383c0 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eb754f20 con 0x55f6eb737aa0 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.546+0000 7fd9293af740 10 client.0 _ll_register_callbacks cb 0x55f6eb644150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb740970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb73e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb740970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58566/0 (socket says 172.21.15.37:58566) 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 -- 172.21.15.37:0/2265615966 learned_addr learned my addr 172.21.15.37:0/2265615966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91ea00640 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 -- 172.21.15.37:0/2265615966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 msgr2=0x55f6eb73dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 -- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 msgr2=0x55f6eb73e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb73e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 -- 172.21.15.37:0/2265615966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6eb4d0f70 con 0x55f6eb737aa0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.547+0000 7fd91f400640 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb740970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd910012760 tx=0x7fd910012c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804e7caa407d564b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd909600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd909600640 1 -- 172.21.15.37:0/2265615966 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910004050 con 0x55f6eb737aa0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd909600640 1 -- 172.21.15.37:0/2265615966 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd9100041f0 con 0x55f6eb737aa0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd90c0a0040 con 0x55f6eb737aa0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd909600640 1 -- 172.21.15.37:0/2265615966 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9100043f0 con 0x55f6eb737aa0 2024-03-21T09:13:06.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd9293af740 10 client.25912 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.548+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6eb368170 con 0x55f6eb737aa0 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd909600640 1 -- 172.21.15.37:0/2265615966 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0a0040 con 0x55f6eb737aa0 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd909600640 1 -- 172.21.15.37:0/2265615966 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd910003b90 con 0x55f6eb737aa0 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd9293af740 10 client.25912 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd9293af740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd9293af740 20 client.25912 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd9293af740 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb762200 0x55f6eb764630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd9293af740 4 client.25912 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6eb6a13f0 con 0x55f6eb762200 2024-03-21T09:13:06.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.549+0000 7fd91ea00640 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb762200 0x55f6eb764630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.550+0000 7fd909600640 10 client.25912 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.550+0000 7fd91ea00640 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb762200 0x55f6eb764630 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce6fb1acd4635c3 server_cookie=da5502fe1d1e8983 in_seq=0 out_seq=0 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.551+0000 7fd909600640 1 -- 172.21.15.37:0/2265615966 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f6eb6a13f0 con 0x55f6eb762200 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.551+0000 7fd909600640 10 client.25912 handle_command_reply: tid=0 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.551+0000 7fd9293af740 1 client.25912 shutdown 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 msgr2=0x55f6eb740970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb740970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd910012760 tx=0x7fd910012c20 comp rx=0 tx=0).stop 2024-03-21T09:13:06.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 shutdown_connections 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb73e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb762200 0x55f6eb764630 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 --2- 172.21.15.37:0/2265615966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb740970 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 >> 172.21.15.37:0/2265615966 conn(0x55f6eb7321d0 msgr2=0x55f6eb733f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd909600640 0 client.25912 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 shutdown_connections 2024-03-21T09:13:06.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 1 -- 172.21.15.37:0/2265615966 wait complete. 2024-03-21T09:13:06.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.552+0000 7fd9293af740 20 client.25912 trim_cache size 0 max 16384 2024-03-21T09:13:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.569+0000 7fd9293af740 1 Processor -- start 2024-03-21T09:13:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 -- start start 2024-03-21T09:13:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb737aa0 0x55f6eb728900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 0x55f6eb740430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb740fa0 0x55f6eb740970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920072f40 con 0x55f6eb737aa0 2024-03-21T09:13:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920070a40 con 0x55f6eb740fa0 2024-03-21T09:13:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd9293af740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92006eae0 con 0x55f6eb7383c0 2024-03-21T09:13:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 0x55f6eb740430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 0x55f6eb740430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58574/0 (socket says 172.21.15.37:58574) 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd91ea00640 1 -- 172.21.15.37:0/3539016335 learned_addr learned my addr 172.21.15.37:0/3539016335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd91fe00640 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb740fa0 0x55f6eb740970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.570+0000 7fd91f400640 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb737aa0 0x55f6eb728900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd91ea00640 1 -- 172.21.15.37:0/3539016335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb740fa0 msgr2=0x55f6eb740970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd91ea00640 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb740fa0 0x55f6eb740970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd91ea00640 1 -- 172.21.15.37:0/3539016335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb737aa0 msgr2=0x55f6eb728900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd91ea00640 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb737aa0 0x55f6eb728900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd91ea00640 1 -- 172.21.15.37:0/3539016335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6eb73c270 con 0x55f6eb7383c0 2024-03-21T09:13:06.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd91ea00640 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 0x55f6eb740430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9c0 tx=0x7fd90c004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f974715eb52c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd90be00640 1 -- 172.21.15.37:0/3539016335 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0cd260 con 0x55f6eb7383c0 2024-03-21T09:13:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd90be00640 1 -- 172.21.15.37:0/3539016335 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0a0040 con 0x55f6eb7383c0 2024-03-21T09:13:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.571+0000 7fd90be00640 1 -- 172.21.15.37:0/3539016335 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0cd860 con 0x55f6eb7383c0 2024-03-21T09:13:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 -- 172.21.15.37:0/3539016335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 msgr2=0x55f6eb740430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 0x55f6eb740430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9c0 tx=0x7fd90c004ba0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 -- 172.21.15.37:0/3539016335 shutdown_connections 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb740fa0 0x55f6eb740970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb7383c0 0x55f6eb740430 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 --2- 172.21.15.37:0/3539016335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb737aa0 0x55f6eb728900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 -- 172.21.15.37:0/3539016335 >> 172.21.15.37:0/3539016335 conn(0x55f6eb7321d0 msgr2=0x55f6eb7281f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.572+0000 7fd9293af740 1 -- 172.21.15.37:0/3539016335 shutdown_connections 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.573+0000 7fd9293af740 1 -- 172.21.15.37:0/3539016335 wait complete. 2024-03-21T09:13:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.573+0000 7fd9293af740 1 -- start start 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.573+0000 7fd9293af740 1 Processor -- start 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb73a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb7bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92006f0e0 con 0x55f6eb740fa0 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200703e0 con 0x55f6eb7383c0 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92006f720 con 0x55f6eb737aa0 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd9293af740 10 client.0 _ll_register_callbacks cb 0x7fd910009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:06.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd91f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb73a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd91ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54392/0 (socket says 172.21.15.37:54392) 2024-03-21T09:13:06.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd91ea00640 1 -- 172.21.15.37:0/2230675795 learned_addr learned my addr 172.21.15.37:0/2230675795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.574+0000 7fd91f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb73a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58586/0 (socket says 172.21.15.37:58586) 2024-03-21T09:13:06.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91fe00640 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb7bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91ea00640 1 -- 172.21.15.37:0/2230675795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 msgr2=0x55f6eb73a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91ea00640 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb73a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91ea00640 1 -- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 msgr2=0x55f6eb7bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91ea00640 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb7bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91ea00640 1 -- 172.21.15.37:0/2230675795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6eb7bd000 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd91ea00640 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73abf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd90c09a1a0 tx=0x7fd90c09b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce6118d157cdab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd909600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.575+0000 7fd909600640 1 -- 172.21.15.37:0/2230675795 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0a4470 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd909600640 1 -- 172.21.15.37:0/2230675795 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd90c098560 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd90c0a0040 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd909600640 1 -- 172.21.15.37:0/2230675795 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c0b0030 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 10 client.25898 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6eb7be120 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd909600640 1 -- 172.21.15.37:0/2230675795 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c08c080 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd909600640 1 -- 172.21.15.37:0/2230675795 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd90c0cd600 con 0x55f6eb7383c0 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 10 client.25898 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:06.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 20 client.25898 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb7be340 0x55f6eb7c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 4 client.25898 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.576+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f6eb69b740 con 0x55f6eb7be340 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.577+0000 7fd91f400640 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb7be340 0x55f6eb7c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.577+0000 7fd91f400640 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb7be340 0x55f6eb7c0730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778f9fd60e5078be server_cookie=8c8f76958b292370 in_seq=0 out_seq=0 2024-03-21T09:13:06.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.577+0000 7fd909600640 10 client.25898 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.580+0000 7fd909600640 1 -- 172.21.15.37:0/2230675795 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f6eb69b740 con 0x55f6eb7be340 2024-03-21T09:13:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.580+0000 7fd909600640 10 client.25898 handle_command_reply: tid=0 2024-03-21T09:13:06.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.580+0000 7fd9293af740 1 client.25898 shutdown 2024-03-21T09:13:06.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.581+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 msgr2=0x55f6eb73abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.581+0000 7fd9293af740 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73abf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd90c09a1a0 tx=0x7fd90c09b850 comp rx=0 tx=0).stop 2024-03-21T09:13:06.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 shutdown_connections 2024-03-21T09:13:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd9293af740 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6eb740fa0 0x55f6eb7bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd9293af740 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6eb7be340 0x55f6eb7c0730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd9293af740 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6eb7383c0 0x55f6eb73abf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd9293af740 1 --2- 172.21.15.37:0/2230675795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6eb737aa0 0x55f6eb73a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd909600640 0 client.25898 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.582+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 >> 172.21.15.37:0/2230675795 conn(0x55f6eb7321d0 msgr2=0x55f6eb728a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.583+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 shutdown_connections 2024-03-21T09:13:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.583+0000 7fd9293af740 1 -- 172.21.15.37:0/2230675795 wait complete. 2024-03-21T09:13:06.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.584+0000 7fd9293af740 20 client.25898 trim_cache size 0 max 16384 2024-03-21T09:13:06.580 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:13:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:13:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "aa175720-f487-4416-a9b5-b44f38a8c226", 2024-03-21T09:13:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:13:06.580 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:13:06.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd9040a31b0 msgr2=0x7fd9040a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd9040a31b0 0x7fd9040a5670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd918033420 tx=0x7fd918033fb0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 msgr2=0x7fd920073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 0x7fd920073ee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9f0 tx=0x7fd90c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 shutdown_connections 2024-03-21T09:13:06.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd9040a31b0 0x7fd9040a5670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd920107560 0x7fd920074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd920101e60 0x7fd920073ee0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9f0 tx=0x7fd90c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 --2- 172.21.15.37:0/3477496638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9201014a0 0x7fd9200739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 >> 172.21.15.37:0/3477496638 conn(0x7fd9200fcfb0 msgr2=0x7fd9200fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 shutdown_connections 2024-03-21T09:13:06.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.586+0000 7fd925000640 1 -- 172.21.15.37:0/3477496638 wait complete. 2024-03-21T09:13:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:06 smithi071 ceph-mon[36597]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-21T09:13:06.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:13:06.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 -- 172.21.15.37:0/1728555950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c1096a0 msgr2=0x7f425c109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 --2- 172.21.15.37:0/1728555950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c1096a0 0x7f425c109a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f424c009900 tx=0x7f424c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 -- 172.21.15.37:0/1728555950 shutdown_connections 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 --2- 172.21.15.37:0/1728555950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c104050 0x7f425c110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 --2- 172.21.15.37:0/1728555950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c103690 0x7f425c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 --2- 172.21.15.37:0/1728555950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c1096a0 0x7f425c109a80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.730+0000 7f4262a00640 1 -- 172.21.15.37:0/1728555950 >> 172.21.15.37:0/1728555950 conn(0x7f425c0ff180 msgr2=0x7f425c1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.731+0000 7f4262a00640 1 -- 172.21.15.37:0/1728555950 shutdown_connections 2024-03-21T09:13:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.731+0000 7f4262a00640 1 -- 172.21.15.37:0/1728555950 wait complete. 2024-03-21T09:13:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.732+0000 7f4262a00640 1 Processor -- start 2024-03-21T09:13:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.732+0000 7f4262a00640 1 -- start start 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4262a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c103690 0x7f425c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4262a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c104050 0x7f425c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4262a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c1096a0 0x7f425c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4262a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c113ac0 con 0x7f425c103690 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4262a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c113940 con 0x7f425c104050 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4262a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c113c40 con 0x7f425c1096a0 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4261600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c103690 0x7f425c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4261600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c103690 0x7f425c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33600/0 (socket says 172.21.15.37:33600) 2024-03-21T09:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4261600640 1 -- 172.21.15.37:0/4243080921 learned_addr learned my addr 172.21.15.37:0/4243080921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.733+0000 7f4260c00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c104050 0x7f425c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4262000640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c1096a0 0x7f425c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4260c00640 1 -- 172.21.15.37:0/4243080921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c1096a0 msgr2=0x7f425c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4260c00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c1096a0 0x7f425c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4260c00640 1 -- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c103690 msgr2=0x7f425c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4260c00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c103690 0x7f425c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4260c00640 1 -- 172.21.15.37:0/4243080921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c19ef50 con 0x7f425c104050 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.734+0000 7f4262000640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c1096a0 0x7f425c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.735+0000 7f4260c00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c104050 0x7f425c19a4a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f425000d6d0 tx=0x7f425000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.735+0000 7f425a000640 1 -- 172.21.15.37:0/4243080921 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250017070 con 0x7f425c104050 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.735+0000 7f425a000640 1 -- 172.21.15.37:0/4243080921 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4250009820 con 0x7f425c104050 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.735+0000 7f425a000640 1 -- 172.21.15.37:0/4243080921 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250004d00 con 0x7f425c104050 2024-03-21T09:13:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.735+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c19f240 con 0x7f425c104050 2024-03-21T09:13:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.735+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c1a6ad0 con 0x7f425c104050 2024-03-21T09:13:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.740+0000 7f4266d77740 1 Processor -- start 2024-03-21T09:13:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.740+0000 7f4266d77740 1 -- start start 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f4266d77740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e89412dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f4266d77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8941bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f425a000640 1 -- 172.21.15.37:0/4243080921 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f42500099d0 con 0x7f425c104050 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f4266d77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e8941e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f4266d77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e893cc3b0 con 0x556e8941c0e0 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f4266d77740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8936e830 con 0x556e894129f0 2024-03-21T09:13:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.741+0000 7f4266d77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e893175d0 con 0x556e89413310 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4262000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e89412dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4262000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e89412dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54424/0 (socket says 172.21.15.37:54424) 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4262000640 1 -- 172.21.15.37:0/3847639550 learned_addr learned my addr 172.21.15.37:0/3847639550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4261600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8941bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4260c00640 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e8941e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4261600640 1 -- 172.21.15.37:0/3847639550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 msgr2=0x556e89412dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4261600640 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e89412dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4261600640 1 -- 172.21.15.37:0/3847639550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 msgr2=0x556e8941e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4261600640 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e8941e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.742+0000 7f4261600640 1 -- 172.21.15.37:0/3847639550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e8932f2c0 con 0x556e89413310 2024-03-21T09:13:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.743+0000 7f4261600640 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8941bba0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f424c000c00 tx=0x7f424c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21ca48e3c7c62238 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.743+0000 7f424a600640 1 -- 172.21.15.37:0/3847639550 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c042070 con 0x556e89413310 2024-03-21T09:13:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.743+0000 7f424a600640 1 -- 172.21.15.37:0/3847639550 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f424c004300 con 0x556e89413310 2024-03-21T09:13:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.743+0000 7f424a600640 1 -- 172.21.15.37:0/3847639550 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c005140 con 0x556e89413310 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.743+0000 7f425a000640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42440a3210 0x7f42440a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f425a000640 1 -- 172.21.15.37:0/4243080921 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f42500cf770 con 0x7f425c104050 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4262000640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42440a3210 0x7f42440a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 -- 172.21.15.37:0/3847639550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 msgr2=0x556e8941bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8941bba0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f424c000c00 tx=0x7f424c02c650 comp rx=0 tx=0).stop 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 -- 172.21.15.37:0/3847639550 shutdown_connections 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e8941e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8941bba0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 --2- 172.21.15.37:0/3847639550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e89412dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.744+0000 7f4266d77740 1 -- 172.21.15.37:0/3847639550 >> 172.21.15.37:0/3847639550 conn(0x556e8940d310 msgr2=0x556e8940d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.745+0000 7f4262000640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42440a3210 0x7f42440a56d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4254005cc0 tx=0x7f4254007780 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.745+0000 7f4266d77740 1 -- 172.21.15.37:0/3847639550 shutdown_connections 2024-03-21T09:13:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.745+0000 7f4266d77740 1 -- 172.21.15.37:0/3847639550 wait complete. 2024-03-21T09:13:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.746+0000 7f4266d77740 1 -- start start 2024-03-21T09:13:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.746+0000 7f4266d77740 1 Processor -- start 2024-03-21T09:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 0x556e894191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e89413310 0x556e89419700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e89419c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8932aa20 con 0x556e8941c0e0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8936a8d0 con 0x556e89413310 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8941ecb0 con 0x556e894129f0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4261600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 0x556e894191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4266d77740 10 client.0 _ll_register_callbacks cb 0x556e893671c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4261600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 0x556e894191c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58624/0 (socket says 172.21.15.37:58624) 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4261600640 1 -- 172.21.15.37:0/2769455360 learned_addr learned my addr 172.21.15.37:0/2769455360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.747+0000 7f4260c00640 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e89413310 0x556e89419700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f4261600640 1 -- 172.21.15.37:0/2769455360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e89413310 msgr2=0x556e89419700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f4261600640 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e89413310 0x556e89419700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f4261600640 1 -- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 msgr2=0x556e89419c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f4261600640 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e89419c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f4261600640 1 -- 172.21.15.37:0/2769455360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e89083210 con 0x556e894129f0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f4261600640 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 0x556e894191c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f424c036040 tx=0x7f424c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=319b3ab8c5ec5aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f423ea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f423ea00640 1 -- 172.21.15.37:0/2769455360 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c042070 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.748+0000 7f423ea00640 1 -- 172.21.15.37:0/2769455360 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f424c004ec0 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.749+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f424c004300 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.749+0000 7f423ea00640 1 -- 172.21.15.37:0/2769455360 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f424c058050 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.749+0000 7f4266d77740 10 client.25930 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.749+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556e8914f620 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.749+0000 7f423ea00640 1 -- 172.21.15.37:0/2769455360 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f424c004300 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f423ea00640 1 -- 172.21.15.37:0/2769455360 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f424c003c70 con 0x556e894129f0 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4266d77740 10 client.25930 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4266d77740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4266d77740 20 client.25930 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4266d77740 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e8942bfd0 0x556e8942e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4266d77740 4 client.25930 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e8937bd70 con 0x556e8942bfd0 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.750+0000 7f4260c00640 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e8942bfd0 0x556e8942e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.751+0000 7f4260c00640 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e8942bfd0 0x556e8942e3c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55da0451e13c945d server_cookie=9ade9f8a300067f0 in_seq=0 out_seq=0 2024-03-21T09:13:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.751+0000 7f423ea00640 10 client.25930 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.752+0000 7f423ea00640 1 -- 172.21.15.37:0/2769455360 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556e8937bd70 con 0x556e8942bfd0 2024-03-21T09:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.752+0000 7f423ea00640 10 client.25930 handle_command_reply: tid=0 2024-03-21T09:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.752+0000 7f4266d77740 1 client.25930 shutdown 2024-03-21T09:13:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 msgr2=0x556e894191c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 0x556e894191c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f424c036040 tx=0x7f424c009260 comp rx=0 tx=0).stop 2024-03-21T09:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 shutdown_connections 2024-03-21T09:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e8941c0e0 0x556e89419c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e8942bfd0 0x556e8942e3c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e89413310 0x556e89419700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 --2- 172.21.15.37:0/2769455360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e894129f0 0x556e894191c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 >> 172.21.15.37:0/2769455360 conn(0x556e8940d310 msgr2=0x556e8940f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.753+0000 7f423ea00640 0 client.25930 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.754+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 shutdown_connections 2024-03-21T09:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.754+0000 7f4266d77740 1 -- 172.21.15.37:0/2769455360 wait complete. 2024-03-21T09:13:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.754+0000 7f4266d77740 20 client.25930 trim_cache size 0 max 16384 2024-03-21T09:13:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 Processor -- start 2024-03-21T09:13:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 -- start start 2024-03-21T09:13:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e8941b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e89413310 0x556e894039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 0x556e89416440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1aeab0 con 0x556e89413310 2024-03-21T09:13:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1ac5b0 con 0x556e894129f0 2024-03-21T09:13:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.783+0000 7f4266d77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1aa670 con 0x556e8941c0e0 2024-03-21T09:13:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 0x556e89416440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 0x556e89416440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58626/0 (socket says 172.21.15.37:58626) 2024-03-21T09:13:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 -- 172.21.15.37:0/2615613834 learned_addr learned my addr 172.21.15.37:0/2615613834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4261600640 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e8941b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 -- 172.21.15.37:0/2615613834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 msgr2=0x556e8941b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e8941b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 -- 172.21.15.37:0/2615613834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e89413310 msgr2=0x556e894039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e89413310 0x556e894039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.784+0000 7f4262000640 1 -- 172.21.15.37:0/2615613834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e89433bd0 con 0x556e8941c0e0 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.785+0000 7f4262000640 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 0x556e89416440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f425400f6f0 tx=0x7f4254013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0966f9f869da4ee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.785+0000 7f4249c00640 1 -- 172.21.15.37:0/2615613834 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254029070 con 0x556e8941c0e0 2024-03-21T09:13:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.785+0000 7f4249c00640 1 -- 172.21.15.37:0/2615613834 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f425401cc30 con 0x556e8941c0e0 2024-03-21T09:13:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.785+0000 7f4249c00640 1 -- 172.21.15.37:0/2615613834 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254024410 con 0x556e8941c0e0 2024-03-21T09:13:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.786+0000 7f4266d77740 1 -- 172.21.15.37:0/2615613834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 msgr2=0x556e89416440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.786+0000 7f4266d77740 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 0x556e89416440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f425400f6f0 tx=0x7f4254013040 comp rx=0 tx=0).stop 2024-03-21T09:13:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 -- 172.21.15.37:0/2615613834 shutdown_connections 2024-03-21T09:13:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e8941c0e0 0x556e89416440 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e89413310 0x556e894039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 --2- 172.21.15.37:0/2615613834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e894129f0 0x556e8941b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 -- 172.21.15.37:0/2615613834 >> 172.21.15.37:0/2615613834 conn(0x556e8940d310 msgr2=0x556e894032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 -- 172.21.15.37:0/2615613834 shutdown_connections 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 -- 172.21.15.37:0/2615613834 wait complete. 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.787+0000 7f4266d77740 1 -- start start 2024-03-21T09:13:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 Processor -- start 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e894129f0 0x556e89430fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8942f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e8941c0e0 0x556e8942f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1aac70 con 0x556e894129f0 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1abf50 con 0x556e8941c0e0 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1ab290 con 0x556e89413310 2024-03-21T09:13:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.788+0000 7f4266d77740 10 client.0 _ll_register_callbacks cb 0x7f4254007620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.789+0000 7f4260c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8942f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.789+0000 7f4260c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8942f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58634/0 (socket says 172.21.15.37:58634) 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.789+0000 7f4260c00640 1 -- 172.21.15.37:0/1048227357 learned_addr learned my addr 172.21.15.37:0/1048227357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.789+0000 7f4262000640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e8941c0e0 0x556e8942f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.789+0000 7f4261600640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e894129f0 0x556e89430fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4262000640 1 -- 172.21.15.37:0/1048227357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 msgr2=0x556e8942f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4262000640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8942f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4262000640 1 -- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e894129f0 msgr2=0x556e89430fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4262000640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e894129f0 0x556e89430fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4262000640 1 -- 172.21.15.37:0/1048227357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e8942ff00 con 0x556e8941c0e0 2024-03-21T09:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4261600640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e894129f0 0x556e89430fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4260c00640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8942f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f4262000640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e8941c0e0 0x556e8942f890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4254005c50 tx=0x7f425401ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e14a449fa0992dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.790+0000 7f423ea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.791+0000 7f423ea00640 1 -- 172.21.15.37:0/1048227357 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254031020 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.791+0000 7f423ea00640 1 -- 172.21.15.37:0/1048227357 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f425401c750 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.791+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f425401cc30 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.791+0000 7f423ea00640 1 -- 172.21.15.37:0/1048227357 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f425403c050 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.791+0000 7f4266d77740 10 client.25916 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.791+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556e894883d0 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f423ea00640 1 -- 172.21.15.37:0/1048227357 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f425401cc30 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f423ea00640 1 -- 172.21.15.37:0/1048227357 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f425402c070 con 0x556e8941c0e0 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f4266d77740 10 client.25916 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f4266d77740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f4266d77740 20 client.25916 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f4266d77740 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e89488a70 0x556e8948ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f4266d77740 4 client.25916 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.792+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e893760c0 con 0x556e89488a70 2024-03-21T09:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.793+0000 7f4261600640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e89488a70 0x556e8948ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.793+0000 7f423ea00640 10 client.25916 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.793+0000 7f4261600640 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e89488a70 0x556e8948ae90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f480e2322104c563 server_cookie=5d6fda703f1e1c76 in_seq=0 out_seq=0 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.801+0000 7f423ea00640 1 -- 172.21.15.37:0/1048227357 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e893760c0 con 0x556e89488a70 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.801+0000 7f423ea00640 10 client.25916 handle_command_reply: tid=0 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.801+0000 7f4266d77740 1 client.25916 shutdown 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.801+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e8941c0e0 msgr2=0x556e8942f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.801+0000 7f4266d77740 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e8941c0e0 0x556e8942f890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4254005c50 tx=0x7f425401ae50 comp rx=0 tx=0).stop 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 shutdown_connections 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556e89488a70 0x556e8948ae90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e8941c0e0 0x556e8942f890 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e89413310 0x556e8942f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 --2- 172.21.15.37:0/1048227357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e894129f0 0x556e89430fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 >> 172.21.15.37:0/1048227357 conn(0x556e8940d310 msgr2=0x556e89403b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f423ea00640 0 client.25916 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 shutdown_connections 2024-03-21T09:13:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 1 -- 172.21.15.37:0/1048227357 wait complete. 2024-03-21T09:13:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.802+0000 7f4266d77740 20 client.25916 trim_cache size 0 max 16384 2024-03-21T09:13:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:13:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:13:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:13:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:13:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.805+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42440a3210 msgr2=0x7f42440a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.805+0000 7f4262a00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42440a3210 0x7f42440a56d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4254005cc0 tx=0x7f4254007780 comp rx=0 tx=0).stop 2024-03-21T09:13:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.805+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c104050 msgr2=0x7f425c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.805+0000 7f4262a00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c104050 0x7f425c19a4a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f425000d6d0 tx=0x7f425000db90 comp rx=0 tx=0).stop 2024-03-21T09:13:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 shutdown_connections 2024-03-21T09:13:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425c1096a0 0x7f425c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42440a3210 0x7f42440a56d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c104050 0x7f425c19a4a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 --2- 172.21.15.37:0/4243080921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c103690 0x7f425c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 >> 172.21.15.37:0/4243080921 conn(0x7f425c0ff180 msgr2=0x7f425c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 shutdown_connections 2024-03-21T09:13:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:06.806+0000 7f4262a00640 1 -- 172.21.15.37:0/4243080921 wait complete. 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 -- 172.21.15.37:0/1520240033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102f50 msgr2=0x7f59bc10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 --2- 172.21.15.37:0/1520240033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102f50 0x7f59bc10b430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59a800fb10 tx=0x7f59a802fe50 comp rx=0 tx=0).stop 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 -- 172.21.15.37:0/1520240033 shutdown_connections 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 --2- 172.21.15.37:0/1520240033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 --2- 172.21.15.37:0/1520240033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102f50 0x7f59bc10b430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 --2- 172.21.15.37:0/1520240033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102630 0x7f59bc102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 -- 172.21.15.37:0/1520240033 >> 172.21.15.37:0/1520240033 conn(0x7f59bc077130 msgr2=0x7f59bc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 -- 172.21.15.37:0/1520240033 shutdown_connections 2024-03-21T09:13:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.853+0000 7f59c2c00640 1 -- 172.21.15.37:0/1520240033 wait complete. 2024-03-21T09:13:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 Processor -- start 2024-03-21T09:13:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 -- start start 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102630 0x7f59bc19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102f50 0x7f59bc19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc10f690 con 0x7f59bc102f50 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc10f510 con 0x7f59bc10ba50 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.855+0000 7f59c2c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc10f810 con 0x7f59bc102630 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc1959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54474/0 (socket says 172.21.15.37:54474) 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 -- 172.21.15.37:0/2990799885 learned_addr learned my addr 172.21.15.37:0/2990799885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c1800640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102630 0x7f59bc19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c0e00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102f50 0x7f59bc19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 -- 172.21.15.37:0/2990799885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102630 msgr2=0x7f59bc19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102630 0x7f59bc19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 -- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102f50 msgr2=0x7f59bc19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102f50 0x7f59bc19bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 -- 172.21.15.37:0/2990799885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc196150 con 0x7f59bc10ba50 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c1800640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102630 0x7f59bc19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c0e00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102f50 0x7f59bc19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:06.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.856+0000 7f59c2200640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc1959d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59b400eb10 tx=0x7f59b400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.857+0000 7f59ba000640 1 -- 172.21.15.37:0/2990799885 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4004b20 con 0x7f59bc10ba50 2024-03-21T09:13:06.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.857+0000 7f59ba000640 1 -- 172.21.15.37:0/2990799885 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59b4004cc0 con 0x7f59bc10ba50 2024-03-21T09:13:06.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.857+0000 7f59ba000640 1 -- 172.21.15.37:0/2990799885 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4010760 con 0x7f59bc10ba50 2024-03-21T09:13:06.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.857+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc196440 con 0x7f59bc10ba50 2024-03-21T09:13:06.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.857+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc1a26e0 con 0x7f59bc10ba50 2024-03-21T09:13:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.861+0000 7f59ba000640 1 -- 172.21.15.37:0/2990799885 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f59b4005020 con 0x7f59bc10ba50 2024-03-21T09:13:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.861+0000 7f59c6fc6740 1 Processor -- start 2024-03-21T09:13:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.863+0000 7f59ba000640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f59a40a32b0 0x7f59a40a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59c6fc6740 1 -- start start 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59ba000640 1 -- 172.21.15.37:0/2990799885 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f59b40d0480 con 0x7f59bc10ba50 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 0x55af396afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59c0e00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f59a40a32b0 0x7f59a40a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59c1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 0x55af396afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.864+0000 7f59c1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 0x55af396afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58668/0 (socket says 172.21.15.37:58668) 2024-03-21T09:13:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.865+0000 7f59c1800640 1 -- 172.21.15.37:0/1959989893 learned_addr learned my addr 172.21.15.37:0/1959989893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.865+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.865+0000 7f59c0e00640 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.865+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af39614250 con 0x55af396c1c70 2024-03-21T09:13:06.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.865+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af395bca30 con 0x55af396b00f0 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.865+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af396102f0 con 0x55af396b0dd0 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c1800640 1 -- 172.21.15.37:0/1959989893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 msgr2=0x55af396b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c1800640 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c1800640 1 -- 172.21.15.37:0/1959989893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 msgr2=0x55af396c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c1800640 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c1800640 1 -- 172.21.15.37:0/1959989893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af395a7690 con 0x55af396b0dd0 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c0e00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f59a40a32b0 0x7f59a40a5770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59bc1971c0 tx=0x7f59a8005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.866+0000 7f59c1800640 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 0x55af396afb20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009cf0 tx=0x7f59b00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8f6ebd9b214bdd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.867+0000 7f59aea00640 1 -- 172.21.15.37:0/1959989893 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0012450 con 0x55af396b0dd0 2024-03-21T09:13:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.867+0000 7f59aea00640 1 -- 172.21.15.37:0/1959989893 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59b0012ad0 con 0x55af396b0dd0 2024-03-21T09:13:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.867+0000 7f59aea00640 1 -- 172.21.15.37:0/1959989893 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b000fcc0 con 0x55af396b0dd0 2024-03-21T09:13:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.867+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 msgr2=0x55af396afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.867+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 0x55af396afb20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009cf0 tx=0x7f59b00117b0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 shutdown_connections 2024-03-21T09:13:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/1959989893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b0dd0 0x55af396afb20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 >> 172.21.15.37:0/1959989893 conn(0x55af39677200 msgr2=0x55af396aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 shutdown_connections 2024-03-21T09:13:06.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 -- 172.21.15.37:0/1959989893 wait complete. 2024-03-21T09:13:06.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.868+0000 7f59c6fc6740 1 -- start start 2024-03-21T09:13:06.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 Processor -- start 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b0dd0 0x55af396be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af396c4ae0 con 0x55af396b0dd0 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af395cfe80 con 0x55af396b00f0 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af396c4c60 con 0x55af396c1c70 2024-03-21T09:13:06.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c6fc6740 10 client.0 _ll_register_callbacks cb 0x55af395e2130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.869+0000 7f59c2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58672/0 (socket says 172.21.15.37:58672) 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 -- 172.21.15.37:0/44665921 learned_addr learned my addr 172.21.15.37:0/44665921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c1800640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c0e00640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b0dd0 0x55af396be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 -- 172.21.15.37:0/44665921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 msgr2=0x55af396c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 -- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b0dd0 msgr2=0x55af396be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b0dd0 0x55af396be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 -- 172.21.15.37:0/44665921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af393f27d0 con 0x55af396c1c70 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c0e00640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b0dd0 0x55af396be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c2200640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bec90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f59b409bf20 tx=0x7f59b409be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269c03c7ff091ab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59a3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59a3400640 1 -- 172.21.15.37:0/44665921 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40a0370 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59a3400640 1 -- 172.21.15.37:0/44665921 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f59b40a47a0 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.870+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b0012ad0 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59a3400640 1 -- 172.21.15.37:0/44665921 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b40a4c30 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 10 client.25948 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af393283f0 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59a3400640 1 -- 172.21.15.37:0/44665921 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59b0012ad0 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59a3400640 1 -- 172.21.15.37:0/44665921 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f59b4005310 con 0x55af396c1c70 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 10 client.25948 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 20 client.25948 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af396d1b80 0x55af396d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 4 client.25948 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:06.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.871+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af39621790 con 0x55af396d1b80 2024-03-21T09:13:06.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.872+0000 7f59c1800640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af396d1b80 0x55af396d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.873+0000 7f59c1800640 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af396d1b80 0x55af396d3f70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacefecf4c3d57d3 server_cookie=b90890f65eaad5be in_seq=0 out_seq=0 2024-03-21T09:13:06.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.873+0000 7f59a3400640 10 client.25948 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.874+0000 7f59a3400640 1 -- 172.21.15.37:0/44665921 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55af39621790 con 0x55af396d1b80 2024-03-21T09:13:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.874+0000 7f59a3400640 10 client.25948 handle_command_reply: tid=0 2024-03-21T09:13:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.874+0000 7f59c6fc6740 1 client.25948 shutdown 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 msgr2=0x55af396bec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bec90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f59b409bf20 tx=0x7f59b409be20 comp rx=0 tx=0).stop 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 shutdown_connections 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bec90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b0dd0 0x55af396be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af396d1b80 0x55af396d3f70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/44665921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b00f0 0x55af396c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 >> 172.21.15.37:0/44665921 conn(0x55af39677200 msgr2=0x55af396ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59a3400640 0 client.25948 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 shutdown_connections 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 1 -- 172.21.15.37:0/44665921 wait complete. 2024-03-21T09:13:06.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.875+0000 7f59c6fc6740 20 client.25948 trim_cache size 0 max 16384 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.909+0000 7f59c6fc6740 1 Processor -- start 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.909+0000 7f59c6fc6740 1 -- start start 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 0x55af396c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1aeb00 con 0x55af396c1c70 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1ac600 con 0x55af396b0dd0 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1aa6a0 con 0x55af396b00f0 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 0x55af396c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 0x55af396c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58686/0 (socket says 172.21.15.37:58686) 2024-03-21T09:13:06.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c1800640 1 -- 172.21.15.37:0/251539619 learned_addr learned my addr 172.21.15.37:0/251539619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.910+0000 7f59c0e00640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c1800640 1 -- 172.21.15.37:0/251539619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 msgr2=0x55af396c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c1800640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c1800640 1 -- 172.21.15.37:0/251539619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 msgr2=0x55af396cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c2200640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c1800640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c1800640 1 -- 172.21.15.37:0/251539619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af396ba350 con 0x55af396b00f0 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c0e00640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c2200640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.911+0000 7f59c1800640 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 0x55af396c0e80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59b0011cb0 tx=0x7f59b0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11ce5c75bf2f991 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.912+0000 7f59ae000640 1 -- 172.21.15.37:0/251539619 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0009550 con 0x55af396b00f0 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.912+0000 7f59ae000640 1 -- 172.21.15.37:0/251539619 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59b002d070 con 0x55af396b00f0 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.912+0000 7f59ae000640 1 -- 172.21.15.37:0/251539619 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b000fb20 con 0x55af396b00f0 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.912+0000 7f59c6fc6740 1 -- 172.21.15.37:0/251539619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 msgr2=0x55af396c0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 0x55af396c0e80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59b0011cb0 tx=0x7f59b0009bf0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 -- 172.21.15.37:0/251539619 shutdown_connections 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396c1c70 0x55af396cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/251539619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396b00f0 0x55af396c0e80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 -- 172.21.15.37:0/251539619 >> 172.21.15.37:0/251539619 conn(0x55af39677200 msgr2=0x55af396aa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 -- 172.21.15.37:0/251539619 shutdown_connections 2024-03-21T09:13:06.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 -- 172.21.15.37:0/251539619 wait complete. 2024-03-21T09:13:06.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 -- start start 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.913+0000 7f59c6fc6740 1 Processor -- start 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b00f0 0x55af396bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1aaca0 con 0x55af396b00f0 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1abfa0 con 0x55af396b0dd0 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1ab2e0 con 0x55af396c1c70 2024-03-21T09:13:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c6fc6740 10 client.0 _ll_register_callbacks cb 0x7f59b40984c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b00f0 0x55af396bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b00f0 0x55af396bb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33698/0 (socket says 172.21.15.37:33698) 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c0e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.914+0000 7f59c1800640 1 -- 172.21.15.37:0/2750575215 learned_addr learned my addr 172.21.15.37:0/2750575215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c2200640 1 -- 172.21.15.37:0/2750575215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 msgr2=0x55af396bbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c2200640 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396bbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c2200640 1 -- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b00f0 msgr2=0x55af396bb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c2200640 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b00f0 0x55af396bb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c2200640 1 -- 172.21.15.37:0/2750575215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af392e80f0 con 0x55af396c1c70 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c2200640 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bc450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55af396afca0 tx=0x7f59b400cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604651274fdc1545 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59a3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59a3400640 1 -- 172.21.15.37:0/2750575215 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40a0370 con 0x55af396c1c70 2024-03-21T09:13:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59a3400640 1 -- 172.21.15.37:0/2750575215 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f59b40b5200 con 0x55af396c1c70 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.915+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b002d070 con 0x55af396c1c70 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59a3400640 1 -- 172.21.15.37:0/2750575215 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b40cdc10 con 0x55af396c1c70 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59c6fc6740 10 client.25954 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af3972d620 con 0x55af396c1c70 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59a3400640 1 -- 172.21.15.37:0/2750575215 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59b002d070 con 0x55af396c1c70 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59a3400640 1 -- 172.21.15.37:0/2750575215 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f59b409b670 con 0x55af396c1c70 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59c6fc6740 10 client.25954 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59c6fc6740 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59c6fc6740 20 client.25954 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.916+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af3972dcf0 0x55af39730110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.917+0000 7f59c6fc6740 4 client.25954 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.917+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af3961bae0 con 0x55af3972dcf0 2024-03-21T09:13:06.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.917+0000 7f59c1800640 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af3972dcf0 0x55af39730110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:06.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.917+0000 7f59c1800640 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af3972dcf0 0x55af39730110 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8233a2ba8f34ca82 server_cookie=788ca32c5028b443 in_seq=0 out_seq=0 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.917+0000 7f59a3400640 10 client.25954 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59a3400640 1 -- 172.21.15.37:0/2750575215 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55af3961bae0 con 0x55af3972dcf0 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59a3400640 10 client.25954 handle_command_reply: tid=0 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 client.25954 shutdown 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 msgr2=0x55af396bc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bc450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55af396afca0 tx=0x7f59b400cee0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 shutdown_connections 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af396c1c70 0x55af396bc450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55af3972dcf0 0x55af39730110 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af396b0dd0 0x55af396bbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 --2- 172.21.15.37:0/2750575215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af396b00f0 0x55af396bb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59a3400640 0 client.25954 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.918+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 >> 172.21.15.37:0/2750575215 conn(0x55af39677200 msgr2=0x55af396abe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 shutdown_connections 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c6fc6740 1 -- 172.21.15.37:0/2750575215 wait complete. 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c6fc6740 20 client.25954 trim_cache size 0 max 16384 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: "aa175720-f487-4416-a9b5-b44f38a8c226": { 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "aa175720-f487-4416-a9b5-b44f38a8c226", 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:13:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:13:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:13:06.916 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:13:06.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f59a40a32b0 msgr2=0x7f59a40a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c2c00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f59a40a32b0 0x7f59a40a5770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59bc1971c0 tx=0x7f59a8005a20 comp rx=0 tx=0).stop 2024-03-21T09:13:06.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 msgr2=0x7f59bc1959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:06.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.919+0000 7f59c2c00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc1959d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59b400eb10 tx=0x7f59b400c2d0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 shutdown_connections 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f59a40a32b0 0x7f59a40a5770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59bc10ba50 0x7f59bc1959d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59bc102f50 0x7f59bc19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 --2- 172.21.15.37:0/2990799885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59bc102630 0x7f59bc19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 >> 172.21.15.37:0/2990799885 conn(0x7f59bc077130 msgr2=0x7f59bc0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 shutdown_connections 2024-03-21T09:13:06.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:06.920+0000 7f59c2c00640 1 -- 172.21.15.37:0/2990799885 wait complete. 2024-03-21T09:13:06.930 INFO:tasks.cephfs.filesystem:scrub status for tag:aa175720-f487-4416-a9b5-b44f38a8c226 - {'path': '/', 'tag': 'aa175720-f487-4416-a9b5-b44f38a8c226', 'options': 'recursive,force'} 2024-03-21T09:13:08.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:08 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:13:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:08 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:08 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:13:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:08 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:08.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:08 smithi037 ceph-mon[31830]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-21T09:13:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:08 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:13:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:08 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:08 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:13:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:08 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:08 smithi053 ceph-mon[36169]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-21T09:13:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:08 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:13:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:08 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:08 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:13:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:08 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:08 smithi071 ceph-mon[36597]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-21T09:13:10.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:10 smithi037 ceph-mon[31830]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T09:13:10.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:10 smithi053 ceph-mon[36169]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T09:13:10.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:10 smithi071 ceph-mon[36597]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T09:13:12.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:12 smithi037 ceph-mon[31830]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T09:13:12.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:12 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:13:12.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:12 smithi053 ceph-mon[36169]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T09:13:12.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:12 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:13:12.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:12 smithi071 ceph-mon[36597]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-21T09:13:12.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:12 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:13:14.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:14 smithi037 ceph-mon[31830]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-21T09:13:14.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:14 smithi053 ceph-mon[36169]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-21T09:13:14.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:14 smithi071 ceph-mon[36597]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-21T09:13:16.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:16 smithi037 ceph-mon[31830]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-21T09:13:16.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:16 smithi053 ceph-mon[36169]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-21T09:13:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:16 smithi071 ceph-mon[36597]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-21T09:13:18.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:18 smithi037 ceph-mon[31830]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-21T09:13:18.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:18 smithi053 ceph-mon[36169]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-21T09:13:18.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:18 smithi071 ceph-mon[36597]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-21T09:13:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.953+0000 7fd6cf200640 1 -- 172.21.15.37:0/2234990247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 msgr2=0x7fd6d010d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.953+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2234990247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 0x7fd6d010d800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009900 tx=0x7fd6b802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 -- 172.21.15.37:0/2234990247 shutdown_connections 2024-03-21T09:13:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2234990247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d010dd40 0x7fd6d0110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2234990247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 0x7fd6d010d800 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2234990247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d0073a50 0x7fd6d0073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 -- 172.21.15.37:0/2234990247 >> 172.21.15.37:0/2234990247 conn(0x7fd6d00fcfb0 msgr2=0x7fd6d00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 -- 172.21.15.37:0/2234990247 shutdown_connections 2024-03-21T09:13:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.954+0000 7fd6cf200640 1 -- 172.21.15.37:0/2234990247 wait complete. 2024-03-21T09:13:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.956+0000 7fd6cf200640 1 Processor -- start 2024-03-21T09:13:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.956+0000 7fd6cf200640 1 -- start start 2024-03-21T09:13:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.956+0000 7fd6cf200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 0x7fd6d0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.956+0000 7fd6cf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 0x7fd6d019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cf200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d010dd40 0x7fd6d019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cf200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0113a70 con 0x7fd6d0073a50 2024-03-21T09:13:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cf200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01138f0 con 0x7fd6d0074370 2024-03-21T09:13:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cf200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0113bf0 con 0x7fd6d010dd40 2024-03-21T09:13:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 0x7fd6d0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d010dd40 0x7fd6d019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 0x7fd6d0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50874/0 (socket says 172.21.15.37:50874) 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6cde00640 1 -- 172.21.15.37:0/2905786031 learned_addr learned my addr 172.21.15.37:0/2905786031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.957+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 0x7fd6d019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6ce800640 1 -- 172.21.15.37:0/2905786031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 msgr2=0x7fd6d019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6ce800640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 0x7fd6d019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6ce800640 1 -- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 msgr2=0x7fd6d0199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6ce800640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 0x7fd6d0199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6ce800640 1 -- 172.21.15.37:0/2905786031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d019efa0 con 0x7fd6d010dd40 2024-03-21T09:13:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6ce800640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d010dd40 0x7fd6d019e870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800d680 tx=0x7fd6c800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6cde00640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 0x7fd6d0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6c7400640 1 -- 172.21.15.37:0/2905786031 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8017070 con 0x7fd6d010dd40 2024-03-21T09:13:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6c7400640 1 -- 172.21.15.37:0/2905786031 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6c8009820 con 0x7fd6d010dd40 2024-03-21T09:13:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6c7400640 1 -- 172.21.15.37:0/2905786031 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004e00 con 0x7fd6d010dd40 2024-03-21T09:13:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d019f290 con 0x7fd6d010dd40 2024-03-21T09:13:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.958+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d0101390 con 0x7fd6d010dd40 2024-03-21T09:13:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.963+0000 7fd6d7dae740 1 Processor -- start 2024-03-21T09:13:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.963+0000 7fd6c7400640 1 -- 172.21.15.37:0/2905786031 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fd6c80099d0 con 0x7fd6d010dd40 2024-03-21T09:13:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.963+0000 7fd6d7dae740 1 -- start start 2024-03-21T09:13:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862091e0 0x5617862095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 0x561786212390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617862128d0 0x561786214cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617861cb3b0 con 0x5617862091e0 2024-03-21T09:13:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178616d830 con 0x5617862128d0 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617861165d0 con 0x561786209b00 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 0x561786212390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6c5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862091e0 0x5617862095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 0x561786212390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38538/0 (socket says 172.21.15.37:38538) 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6cde00640 1 -- 172.21.15.37:0/1709899887 learned_addr learned my addr 172.21.15.37:0/1709899887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.964+0000 7fd6c5400640 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617862128d0 0x561786214cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6cde00640 1 -- 172.21.15.37:0/1709899887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617862128d0 msgr2=0x561786214cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6cde00640 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617862128d0 0x561786214cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6cde00640 1 -- 172.21.15.37:0/1709899887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862091e0 msgr2=0x5617862095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6cde00640 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862091e0 0x5617862095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6cde00640 1 -- 172.21.15.37:0/1709899887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56178612e2c0 con 0x561786209b00 2024-03-21T09:13:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6c5400640 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617862128d0 0x561786214cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6cde00640 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 0x561786212390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000f8d0 tx=0x7fd6c000fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd754f246b7674e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6bf400640 1 -- 172.21.15.37:0/1709899887 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0011c20 con 0x561786209b00 2024-03-21T09:13:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6bf400640 1 -- 172.21.15.37:0/1709899887 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6c0011dc0 con 0x561786209b00 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.965+0000 7fd6bf400640 1 -- 172.21.15.37:0/1709899887 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0018830 con 0x561786209b00 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6c7400640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd6b40aba20 0x7fd6b40adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6c7400640 1 -- 172.21.15.37:0/2905786031 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd6c80cf5e0 con 0x7fd6d010dd40 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 -- 172.21.15.37:0/1709899887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 msgr2=0x561786212390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 0x561786212390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000f8d0 tx=0x7fd6c000fd90 comp rx=0 tx=0).stop 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 -- 172.21.15.37:0/1709899887 shutdown_connections 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617862128d0 0x561786214cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561786209b00 0x561786212390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/1709899887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862091e0 0x5617862095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.966+0000 7fd6d7dae740 1 -- 172.21.15.37:0/1709899887 >> 172.21.15.37:0/1709899887 conn(0x561786203b00 msgr2=0x561786203ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.967+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd6b40aba20 0x7fd6b40adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.967+0000 7fd6d7dae740 1 -- 172.21.15.37:0/1709899887 shutdown_connections 2024-03-21T09:13:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.967+0000 7fd6d7dae740 1 -- 172.21.15.37:0/1709899887 wait complete. 2024-03-21T09:13:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 -- start start 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 Processor -- start 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd6b40aba20 0x7fd6b40adee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8005ab0 tx=0x7fd6b8005b50 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x561786210410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561786129a20 con 0x5617862128d0 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617861698d0 con 0x561786209b00 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.968+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561786215500 con 0x5617862091e0 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6d7dae740 10 client.0 _ll_register_callbacks cb 0x5617861661c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38554/0 (socket says 172.21.15.37:38554) 2024-03-21T09:13:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6cde00640 1 -- 172.21.15.37:0/986578463 learned_addr learned my addr 172.21.15.37:0/986578463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6ce800640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x561786210410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6c5400640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6ce800640 1 -- 172.21.15.37:0/986578463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 msgr2=0x56178620f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6ce800640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6ce800640 1 -- 172.21.15.37:0/986578463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 msgr2=0x56178620fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6ce800640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.969+0000 7fd6ce800640 1 -- 172.21.15.37:0/986578463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561785e82210 con 0x5617862128d0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6c5400640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6ce800640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x561786210410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80cc7b0 tx=0x7fd6c8004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69c5073465e2afc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6b3e00640 1 -- 172.21.15.37:0/986578463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8020070 con 0x5617862128d0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6b3e00640 1 -- 172.21.15.37:0/986578463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd6c8097aa0 con 0x5617862128d0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6c0011dc0 con 0x5617862128d0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6b3e00640 1 -- 172.21.15.37:0/986578463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6c80a3d90 con 0x5617862128d0 2024-03-21T09:13:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6d7dae740 10 client.15795 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.970+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561785f4e620 con 0x5617862128d0 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6b3e00640 1 -- 172.21.15.37:0/986578463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6c0011dc0 con 0x5617862128d0 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6b3e00640 1 -- 172.21.15.37:0/986578463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd6c80cca90 con 0x5617862128d0 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6d7dae740 10 client.15795 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6d7dae740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6d7dae740 20 client.15795 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561786222960 0x561786224d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6d7dae740 4 client.15795 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56178617ad70 con 0x561786222960 2024-03-21T09:13:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.971+0000 7fd6cde00640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561786222960 0x561786224d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.972+0000 7fd6cde00640 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561786222960 0x561786224d50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a3ea1ef6314ced server_cookie=e895d14fe56f7a2 in_seq=0 out_seq=0 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.972+0000 7fd6b3e00640 10 client.15795 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.973+0000 7fd6b3e00640 1 -- 172.21.15.37:0/986578463 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56178617ad70 con 0x561786222960 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.973+0000 7fd6b3e00640 10 client.15795 handle_command_reply: tid=0 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.973+0000 7fd6d7dae740 1 client.15795 shutdown 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 msgr2=0x561786210410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x561786210410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80cc7b0 tx=0x7fd6c8004780 comp rx=0 tx=0).stop 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 shutdown_connections 2024-03-21T09:13:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x561786210410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561786222960 0x561786224d50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/986578463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 >> 172.21.15.37:0/986578463 conn(0x561786203b00 msgr2=0x561786205850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6b3e00640 0 client.15795 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 shutdown_connections 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.974+0000 7fd6d7dae740 1 -- 172.21.15.37:0/986578463 wait complete. 2024-03-21T09:13:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:19.975+0000 7fd6d7dae740 20 client.15795 trim_cache size 0 max 16384 2024-03-21T09:13:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.004+0000 7fd6d7dae740 1 Processor -- start 2024-03-21T09:13:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.004+0000 7fd6d7dae740 1 -- start start 2024-03-21T09:13:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.004+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178621dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.004+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178621e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.004+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.004+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01aeb80 con 0x5617862128d0 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178621dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01ac680 con 0x561786209b00 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01aa740 con 0x5617862091e0 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178621dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38570/0 (socket says 172.21.15.37:38570) 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6cde00640 1 -- 172.21.15.37:0/4068590370 learned_addr learned my addr 172.21.15.37:0/4068590370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6c5400640 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178621e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6ce800640 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6cde00640 1 -- 172.21.15.37:0/4068590370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 msgr2=0x56178621e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.005+0000 7fd6cde00640 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178621e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6cde00640 1 -- 172.21.15.37:0/4068590370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 msgr2=0x56178620e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6cde00640 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6cde00640 1 -- 172.21.15.37:0/4068590370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617862250c0 con 0x5617862091e0 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6ce800640 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6cde00640 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178621dc10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0011fd0 tx=0x7fd6c00183d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e6ab48c453e3218 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6bea00640 1 -- 172.21.15.37:0/4068590370 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0030070 con 0x5617862091e0 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6bea00640 1 -- 172.21.15.37:0/4068590370 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6c0011da0 con 0x5617862091e0 2024-03-21T09:13:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.006+0000 7fd6bea00640 1 -- 172.21.15.37:0/4068590370 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0005830 con 0x5617862091e0 2024-03-21T09:13:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.007+0000 7fd6d7dae740 1 -- 172.21.15.37:0/4068590370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 msgr2=0x56178621dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.007+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178621dc10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0011fd0 tx=0x7fd6c00183d0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 -- 172.21.15.37:0/4068590370 shutdown_connections 2024-03-21T09:13:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178621e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/4068590370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178621dc10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 -- 172.21.15.37:0/4068590370 >> 172.21.15.37:0/4068590370 conn(0x561786203b00 msgr2=0x5617861fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 -- 172.21.15.37:0/4068590370 shutdown_connections 2024-03-21T09:13:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 -- 172.21.15.37:0/4068590370 wait complete. 2024-03-21T09:13:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.008+0000 7fd6d7dae740 1 -- start start 2024-03-21T09:13:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 Processor -- start 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01aad40 con 0x5617862128d0 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01ac020 con 0x561786209b00 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01ab360 con 0x5617862091e0 2024-03-21T09:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.009+0000 7fd6d7dae740 10 client.0 _ll_register_callbacks cb 0x7fd6c80057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38586/0 (socket says 172.21.15.37:38586) 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 -- 172.21.15.37:0/2296475247 learned_addr learned my addr 172.21.15.37:0/2296475247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6ce800640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 -- 172.21.15.37:0/2296475247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 msgr2=0x56178620c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 -- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 msgr2=0x56178620cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 -- 172.21.15.37:0/2296475247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56178627d200 con 0x5617862091e0 2024-03-21T09:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6ce800640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.010+0000 7fd6cde00640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620c180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0005df0 tx=0x7fd6c0005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=284439360cffd836 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6b3e00640 1 -- 172.21.15.37:0/2296475247 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c003b060 con 0x5617862091e0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6b3e00640 1 -- 172.21.15.37:0/2296475247 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd6c00055b0 con 0x5617862091e0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6c0011da0 con 0x5617862091e0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6b3e00640 1 -- 172.21.15.37:0/2296475247 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6c0030070 con 0x5617862091e0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6d7dae740 10 client.25972 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.011+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56178627e440 con 0x5617862091e0 2024-03-21T09:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.012+0000 7fd6b3e00640 1 -- 172.21.15.37:0/2296475247 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6c0011da0 con 0x5617862091e0 2024-03-21T09:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6b3e00640 1 -- 172.21.15.37:0/2296475247 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd6c00049b0 con 0x5617862091e0 2024-03-21T09:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6d7dae740 10 client.25972 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6d7dae740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6d7dae740 20 client.25972 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178627e7c0 0x561786280bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6d7dae740 4 client.25972 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.013+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617861750c0 con 0x56178627e7c0 2024-03-21T09:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.014+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178627e7c0 0x561786280bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.015+0000 7fd6c5400640 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178627e7c0 0x561786280bb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f54cc5533149c12 server_cookie=2227c24a37eaadb9 in_seq=0 out_seq=0 2024-03-21T09:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.015+0000 7fd6b3e00640 10 client.25972 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.019+0000 7fd6b3e00640 1 -- 172.21.15.37:0/2296475247 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617861750c0 con 0x56178627e7c0 2024-03-21T09:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.019+0000 7fd6b3e00640 10 client.25972 handle_command_reply: tid=0 2024-03-21T09:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.019+0000 7fd6d7dae740 1 client.25972 shutdown 2024-03-21T09:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 msgr2=0x56178620c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620c180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0005df0 tx=0x7fd6c0005bd0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 shutdown_connections 2024-03-21T09:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5617862128d0 0x56178620cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178627e7c0 0x561786280bb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561786209b00 0x56178620c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 --2- 172.21.15.37:0/2296475247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617862091e0 0x56178620c180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 >> 172.21.15.37:0/2296475247 conn(0x561786203b00 msgr2=0x56178620e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6b3e00640 0 client.25972 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 shutdown_connections 2024-03-21T09:13:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.020+0000 7fd6d7dae740 1 -- 172.21.15.37:0/2296475247 wait complete. 2024-03-21T09:13:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.022+0000 7fd6d7dae740 20 client.25972 trim_cache size 0 max 16384 2024-03-21T09:13:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:13:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:13:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:13:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:13:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.022+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd6b40aba20 msgr2=0x7fd6b40adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.022+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd6b40aba20 0x7fd6b40adee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8005ab0 tx=0x7fd6b8005b50 comp rx=0 tx=0).stop 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.022+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d010dd40 msgr2=0x7fd6d019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.022+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d010dd40 0x7fd6d019e870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800d680 tx=0x7fd6c800db40 comp rx=0 tx=0).stop 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 shutdown_connections 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d010dd40 0x7fd6d019e870 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd6b40aba20 0x7fd6b40adee0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d0074370 0x7fd6d019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 --2- 172.21.15.37:0/2905786031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0073a50 0x7fd6d0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 >> 172.21.15.37:0/2905786031 conn(0x7fd6d00fcfb0 msgr2=0x7fd6d00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 shutdown_connections 2024-03-21T09:13:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:20.023+0000 7fd6cf200640 1 -- 172.21.15.37:0/2905786031 wait complete. 2024-03-21T09:13:20.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:20 smithi037 ceph-mon[31830]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:13:20.440 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:20 smithi053 ceph-mon[36169]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:13:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:20 smithi071 ceph-mon[36597]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:13:21.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:21 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:13:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:21 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:13:21.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:21 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:13:22.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:22 smithi053 ceph-mon[36169]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:13:22.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:22 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:13:22.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:22 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:13:22.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:22 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:13:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:22 smithi071 ceph-mon[36597]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:13:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:22 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:13:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:22 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:13:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:22 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:13:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:22 smithi037 ceph-mon[31830]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:13:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:22 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:13:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:22 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:13:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:22 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:13:24.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:24 smithi053 ceph-mon[36169]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:24 smithi071 ceph-mon[36597]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:24 smithi037 ceph-mon[31830]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:26.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:26 smithi053 ceph-mon[36169]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:26 smithi071 ceph-mon[36597]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:26 smithi037 ceph-mon[31830]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:28.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:28 smithi053 ceph-mon[36169]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:28 smithi071 ceph-mon[36597]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:28.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:28 smithi037 ceph-mon[31830]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:30.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:30 smithi053 ceph-mon[36169]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:30 smithi071 ceph-mon[36597]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:30.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:30 smithi037 ceph-mon[31830]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:32.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:32 smithi071 ceph-mon[36597]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:32.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:32 smithi037 ceph-mon[31830]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:32.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:32 smithi053 ceph-mon[36169]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.172+0000 7f0aa2a00640 1 -- 172.21.15.37:0/1711467491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100430 msgr2=0x7f0a9c100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.172+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/1711467491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100430 0x7f0a9c100810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009900 tx=0x7f0a8c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 -- 172.21.15.37:0/1711467491 shutdown_connections 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/1711467491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c10dfe0 0x7f0a9c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/1711467491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c100de0 0x7f0a9c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/1711467491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100430 0x7f0a9c100810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 -- 172.21.15.37:0/1711467491 >> 172.21.15.37:0/1711467491 conn(0x7f0a9c0fc040 msgr2=0x7f0a9c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 -- 172.21.15.37:0/1711467491 shutdown_connections 2024-03-21T09:13:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.173+0000 7f0aa2a00640 1 -- 172.21.15.37:0/1711467491 wait complete. 2024-03-21T09:13:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.174+0000 7f0aa2a00640 1 Processor -- start 2024-03-21T09:13:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.174+0000 7f0aa2a00640 1 -- start start 2024-03-21T09:13:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.174+0000 7f0aa2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 0x7f0a9c1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0aa2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100de0 0x7f0a9c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0aa2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c10dfe0 0x7f0a9c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 0x7f0a9c1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 0x7f0a9c1961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57596/0 (socket says 172.21.15.37:57596) 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100de0 0x7f0a9c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100de0 0x7f0a9c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50864/0 (socket says 172.21.15.37:50864) 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0a9b400640 1 -- 172.21.15.37:0/971026577 learned_addr learned my addr 172.21.15.37:0/971026577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c104fb0 con 0x7f0a9c10dfe0 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c104e30 con 0x7f0a9c100de0 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c105130 con 0x7f0a9c100430 2024-03-21T09:13:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.175+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c10dfe0 0x7f0a9c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0a9be00640 1 -- 172.21.15.37:0/971026577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100de0 msgr2=0x7f0a9c1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0a9be00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100de0 0x7f0a9c1966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0a9be00640 1 -- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c10dfe0 msgr2=0x7f0a9c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0a9be00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c10dfe0 0x7f0a9c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0a9be00640 1 -- 172.21.15.37:0/971026577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c19ade0 con 0x7f0a9c100430 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c10dfe0 0x7f0a9c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.176+0000 7f0a9be00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 0x7f0a9c1961b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c170 tx=0x7f0a8c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.177+0000 7f0a98c00640 1 -- 172.21.15.37:0/971026577 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c043020 con 0x7f0a9c100430 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.177+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c19b070 con 0x7f0a9c100430 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.177+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1a2960 con 0x7f0a9c100430 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.177+0000 7f0a98c00640 1 -- 172.21.15.37:0/971026577 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c0054e0 con 0x7f0a9c100430 2024-03-21T09:13:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.177+0000 7f0a98c00640 1 -- 172.21.15.37:0/971026577 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003b50 con 0x7f0a9c100430 2024-03-21T09:13:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 Processor -- start 2024-03-21T09:13:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 -- start start 2024-03-21T09:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ce73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cf0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cf2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709ca93b0 con 0x562709ce6fd0 2024-03-21T09:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709c4b830 con 0x562709ce78f0 2024-03-21T09:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.181+0000 7f0aa5641740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709bf45d0 con 0x562709cf06c0 2024-03-21T09:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.182+0000 7f0aa0a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ce73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.182+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cf2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.182+0000 7f0aa0a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ce73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38078/0 (socket says 172.21.15.37:38078) 2024-03-21T09:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.182+0000 7f0aa0a00640 1 -- 172.21.15.37:0/2800373216 learned_addr learned my addr 172.21.15.37:0/2800373216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.182+0000 7f0a98c00640 1 -- 172.21.15.37:0/971026577 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f0a8c046020 con 0x7f0a9c100430 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a9b400640 1 -- 172.21.15.37:0/2800373216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 msgr2=0x562709cf0180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a9b400640 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cf0180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a9b400640 1 -- 172.21.15.37:0/2800373216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 msgr2=0x562709ce73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a9b400640 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ce73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a9b400640 1 -- 172.21.15.37:0/2800373216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562709c0c2c0 con 0x562709cf06c0 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ce73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a9b400640 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cf2ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0ff640 tx=0x7f0a88016a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f20f3395c21453 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.183+0000 7f0a91200640 1 -- 172.21.15.37:0/2800373216 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88022070 con 0x562709cf06c0 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.184+0000 7f0a91200640 1 -- 172.21.15.37:0/2800373216 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8801dbf0 con 0x562709cf06c0 2024-03-21T09:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.184+0000 7f0a91200640 1 -- 172.21.15.37:0/2800373216 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88003b20 con 0x562709cf06c0 2024-03-21T09:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.184+0000 7f0aa5641740 1 -- 172.21.15.37:0/2800373216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 msgr2=0x562709cf2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.184+0000 7f0aa5641740 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cf2ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0ff640 tx=0x7f0a88016a20 comp rx=0 tx=0).stop 2024-03-21T09:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.184+0000 7f0aa5641740 1 -- 172.21.15.37:0/2800373216 shutdown_connections 2024-03-21T09:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.184+0000 7f0aa5641740 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cf2ab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0aa5641740 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cf0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0aa5641740 1 --2- 172.21.15.37:0/2800373216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ce73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0aa5641740 1 -- 172.21.15.37:0/2800373216 >> 172.21.15.37:0/2800373216 conn(0x562709ce18f0 msgr2=0x562709ce1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0a98c00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0a800a3220 0x7f0a800a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0aa5641740 1 -- 172.21.15.37:0/2800373216 shutdown_connections 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0aa5641740 1 -- 172.21.15.37:0/2800373216 wait complete. 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0a9b400640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0a800a3220 0x7f0a800a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0a98c00640 1 -- 172.21.15.37:0/971026577 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0a8c0f1cf0 con 0x7f0a9c100430 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.185+0000 7f0aa5641740 1 -- start start 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.186+0000 7f0aa5641740 1 Processor -- start 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.186+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ced6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.186+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cedc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.186+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0aa5641740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709c07a20 con 0x562709ce6fd0 2024-03-21T09:13:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0aa5641740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709c478d0 con 0x562709ce78f0 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cedc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ced6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cedc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50876/0 (socket says 172.21.15.37:50876) 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0a9b400640 1 -- 172.21.15.37:0/1306506392 learned_addr learned my addr 172.21.15.37:0/1306506392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709cf32f0 con 0x562709cf06c0 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0aa5641740 10 client.0 _ll_register_callbacks cb 0x562709c441c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.187+0000 7f0a9b400640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0a800a3220 0x7f0a800a56e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a88016350 tx=0x7f0a88003700 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a9be00640 1 -- 172.21.15.37:0/1306506392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 msgr2=0x562709cee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a9be00640 1 -- 172.21.15.37:0/1306506392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 msgr2=0x562709cedc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cedc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a9be00640 1 -- 172.21.15.37:0/1306506392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562709960210 con 0x562709ce6fd0 2024-03-21T09:13:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ced6c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0ef030 tx=0x7f0a8c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72741e6d01245027 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a85600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:13:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.188+0000 7f0a85600640 1 -- 172.21.15.37:0/1306506392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0be300 con 0x562709ce6fd0 2024-03-21T09:13:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.189+0000 7f0a85600640 1 -- 172.21.15.37:0/1306506392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0a8c0c6180 con 0x562709ce6fd0 2024-03-21T09:13:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.189+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a8801dbf0 con 0x562709ce6fd0 2024-03-21T09:13:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.189+0000 7f0a85600640 1 -- 172.21.15.37:0/1306506392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a8c0bdc00 con 0x562709ce6fd0 2024-03-21T09:13:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.189+0000 7f0aa5641740 10 client.15810 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.189+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562709a2c620 con 0x562709ce6fd0 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.189+0000 7f0a85600640 1 -- 172.21.15.37:0/1306506392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8801dbf0 con 0x562709ce6fd0 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0a85600640 1 -- 172.21.15.37:0/1306506392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0a8c0d4060 con 0x562709ce6fd0 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0aa5641740 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0aa5641740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0aa5641740 20 client.15810 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d116d0 0x562709d13b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0aa5641740 4 client.15810 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562709c58d70 con 0x562709d116d0 2024-03-21T09:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.190+0000 7f0a9b400640 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d116d0 0x562709d13b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.191+0000 7f0a9b400640 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d116d0 0x562709d13b40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82aed8923f5fa1cb server_cookie=7a9ea4229fdc697 in_seq=0 out_seq=0 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.191+0000 7f0a85600640 10 client.15810 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.192+0000 7f0a85600640 1 -- 172.21.15.37:0/1306506392 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562709c58d70 con 0x562709d116d0 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.192+0000 7f0a85600640 10 client.15810 handle_command_reply: tid=0 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 client.15810 shutdown 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 msgr2=0x562709ced6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ced6c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0ef030 tx=0x7f0a8c004ee0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 shutdown_connections 2024-03-21T09:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709cee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d116d0 0x562709d13b40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce78f0 0x562709cedc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1306506392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709ced6c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 >> 172.21.15.37:0/1306506392 conn(0x562709ce18f0 msgr2=0x562709ce3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0a85600640 0 client.15810 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 shutdown_connections 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.193+0000 7f0aa5641740 1 -- 172.21.15.37:0/1306506392 wait complete. 2024-03-21T09:13:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.194+0000 7f0aa5641740 20 client.15810 trim_cache size 0 max 16384 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.223+0000 7f0aa5641740 1 Processor -- start 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.223+0000 7f0aa5641740 1 -- start start 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709cefdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709ce78f0 0x562709cfba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709cf06c0 0x562709cfbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa5641740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c012210 con 0x562709ce6fd0 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa5641740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00fd10 con 0x562709cf06c0 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa5641740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00ddb0 con 0x562709ce78f0 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709cefdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0aa0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709cf06c0 0x562709cfbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709cefdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38092/0 (socket says 172.21.15.37:38092) 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.224+0000 7f0a9be00640 1 -- 172.21.15.37:0/1023453101 learned_addr learned my addr 172.21.15.37:0/1023453101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0aa0a00640 1 -- 172.21.15.37:0/1023453101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709ce78f0 msgr2=0x562709cfba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709ce78f0 0x562709cfba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0aa0a00640 1 -- 172.21.15.37:0/1023453101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 msgr2=0x562709cefdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709cefdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0aa0a00640 1 -- 172.21.15.37:0/1023453101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562709d141c0 con 0x562709cf06c0 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709cefdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709cf06c0 0x562709cfbf70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009fd0 tx=0x7f0a94012d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed7555adaef0e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0a87e00640 1 -- 172.21.15.37:0/1023453101 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a940121d0 con 0x562709cf06c0 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0a87e00640 1 -- 172.21.15.37:0/1023453101 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a94003e50 con 0x562709cf06c0 2024-03-21T09:13:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.225+0000 7f0a87e00640 1 -- 172.21.15.37:0/1023453101 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94004340 con 0x562709cf06c0 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 -- 172.21.15.37:0/1023453101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709cf06c0 msgr2=0x562709cfbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709cf06c0 0x562709cfbf70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009fd0 tx=0x7f0a94012d20 comp rx=0 tx=0).stop 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 -- 172.21.15.37:0/1023453101 shutdown_connections 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709cf06c0 0x562709cfbf70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709ce78f0 0x562709cfba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1023453101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce6fd0 0x562709cefdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 -- 172.21.15.37:0/1023453101 >> 172.21.15.37:0/1023453101 conn(0x562709ce18f0 msgr2=0x562709ce1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 -- 172.21.15.37:0/1023453101 shutdown_connections 2024-03-21T09:13:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.226+0000 7f0aa5641740 1 -- 172.21.15.37:0/1023453101 wait complete. 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.227+0000 7f0aa5641740 1 -- start start 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.227+0000 7f0aa5641740 1 Processor -- start 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.227+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 0x562709cea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.227+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce78f0 0x562709ceae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0aa5641740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709d64fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0aa5641740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00e3b0 con 0x562709ce78f0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0aa5641740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00f6b0 con 0x562709ce6fd0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0aa5641740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00e9f0 con 0x562709cf06c0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0aa5641740 10 client.0 _ll_register_callbacks cb 0x7f0a8c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 0x562709cea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 0x562709cea940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:50904/0 (socket says 172.21.15.37:50904) 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 -- 172.21.15.37:0/1679217282 learned_addr learned my addr 172.21.15.37:0/1679217282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0aa0a00640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709d64fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9b400640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce78f0 0x562709ceae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 -- 172.21.15.37:0/1679217282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 msgr2=0x562709d64fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709d64fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 -- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce78f0 msgr2=0x562709ceae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce78f0 0x562709ceae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.228+0000 7f0a9be00640 1 -- 172.21.15.37:0/1679217282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562709d657d0 con 0x562709ce6fd0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.229+0000 7f0a9be00640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 0x562709cea940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c820 tx=0x7f0a8c02ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9066d1465d699c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.229+0000 7f0a85600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.229+0000 7f0a85600640 1 -- 172.21.15.37:0/1679217282 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0bd740 con 0x562709ce6fd0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.229+0000 7f0a85600640 1 -- 172.21.15.37:0/1679217282 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0a8c0b9950 con 0x562709ce6fd0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.229+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a94003e50 con 0x562709ce6fd0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0a85600640 1 -- 172.21.15.37:0/1679217282 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a8c0be300 con 0x562709ce6fd0 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 10 client.25964 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562709d6dc40 con 0x562709ce6fd0 2024-03-21T09:13:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0a85600640 1 -- 172.21.15.37:0/1679217282 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a94003e50 con 0x562709ce6fd0 2024-03-21T09:13:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0a85600640 1 -- 172.21.15.37:0/1679217282 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0a8c0b9b50 con 0x562709ce6fd0 2024-03-21T09:13:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 10 client.25964 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 10 client.25964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 20 client.25964 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d6e170 0x562709d70560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 4 client.25964 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:13:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.230+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562709c530c0 con 0x562709d6e170 2024-03-21T09:13:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.231+0000 7f0a9b400640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d6e170 0x562709d70560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.232+0000 7f0a9b400640 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d6e170 0x562709d70560 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8557319829977241 server_cookie=d3c02f500dc706d1 in_seq=0 out_seq=0 2024-03-21T09:13:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.232+0000 7f0a85600640 10 client.25964 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0a85600640 1 -- 172.21.15.37:0/1679217282 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562709c530c0 con 0x562709d6e170 2024-03-21T09:13:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0a85600640 10 client.25964 handle_command_reply: tid=0 2024-03-21T09:13:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 client.25964 shutdown 2024-03-21T09:13:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 msgr2=0x562709cea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 0x562709cea940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c820 tx=0x7f0a8c02ca90 comp rx=0 tx=0).stop 2024-03-21T09:13:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 shutdown_connections 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562709cf06c0 0x562709d64fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562709ce78f0 0x562709ceae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562709d6e170 0x562709d70560 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 --2- 172.21.15.37:0/1679217282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562709ce6fd0 0x562709cea940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 >> 172.21.15.37:0/1679217282 conn(0x562709ce18f0 msgr2=0x562709ce00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.236+0000 7f0a85600640 0 client.25964 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.237+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 shutdown_connections 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.237+0000 7f0aa5641740 1 -- 172.21.15.37:0/1679217282 wait complete. 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.237+0000 7f0aa5641740 20 client.25964 trim_cache size 0 max 16384 2024-03-21T09:13:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:13:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:13:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:13:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0a800a3220 msgr2=0x7f0a800a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0a800a3220 0x7f0a800a56e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a88016350 tx=0x7f0a88003700 comp rx=0 tx=0).stop 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 msgr2=0x7f0a9c1961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 0x7f0a9c1961b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c170 tx=0x7f0a8c004360 comp rx=0 tx=0).stop 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 shutdown_connections 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a9c10dfe0 0x7f0a9c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0a800a3220 0x7f0a800a56e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a9c100de0 0x7f0a9c1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 --2- 172.21.15.37:0/971026577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c100430 0x7f0a9c1961b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 >> 172.21.15.37:0/971026577 conn(0x7f0a9c0fc040 msgr2=0x7f0a9c10f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 shutdown_connections 2024-03-21T09:13:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:33.239+0000 7f0aa2a00640 1 -- 172.21.15.37:0/971026577 wait complete. 2024-03-21T09:13:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:34 smithi071 ceph-mon[36597]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:34.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:34 smithi037 ceph-mon[31830]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:34 smithi053 ceph-mon[36169]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:36.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:36 smithi071 ceph-mon[36597]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:13:36.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:36 smithi037 ceph-mon[31830]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:13:36.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:36 smithi053 ceph-mon[36169]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:13:36.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:13:36.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 -- 172.21.15.37:0/754720326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 msgr2=0x7fc3d0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 --2- 172.21.15.37:0/754720326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 0x7fc3d0101920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00dd40 tx=0x7fc3bc034180 comp rx=0 tx=0).stop 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 -- 172.21.15.37:0/754720326 shutdown_connections 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 --2- 172.21.15.37:0/754720326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0101e60 0x7fc3d010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 --2- 172.21.15.37:0/754720326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 0x7fc3d0101920 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 --2- 172.21.15.37:0/754720326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0107560 0x7fc3d0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 -- 172.21.15.37:0/754720326 >> 172.21.15.37:0/754720326 conn(0x7fc3d00fcfb0 msgr2=0x7fc3d00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 -- 172.21.15.37:0/754720326 shutdown_connections 2024-03-21T09:13:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.161+0000 7fc3d5000640 1 -- 172.21.15.37:0/754720326 wait complete. 2024-03-21T09:13:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.162+0000 7fc3d5000640 1 Processor -- start 2024-03-21T09:13:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.162+0000 7fc3d5000640 1 -- start start 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3d5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 0x7fc3d019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3d5000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 0x7fc3d019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3d5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0107560 0x7fc3d01958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3d5000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d010f700 con 0x7fc3d0107560 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3d5000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d010f580 con 0x7fc3d0101e60 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3d5000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d010f880 con 0x7fc3d01014a0 2024-03-21T09:13:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3cea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 0x7fc3d019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3cea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 0x7fc3d019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44822/0 (socket says 172.21.15.37:44822) 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3cea00640 1 -- 172.21.15.37:0/1204176933 learned_addr learned my addr 172.21.15.37:0/1204176933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.163+0000 7fc3cf400640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 0x7fc3d019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0107560 0x7fc3d01958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cea00640 1 -- 172.21.15.37:0/1204176933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 msgr2=0x7fc3d019b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cea00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 0x7fc3d019b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cea00640 1 -- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0107560 msgr2=0x7fc3d01958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cea00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0107560 0x7fc3d01958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cea00640 1 -- 172.21.15.37:0/1204176933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0196040 con 0x7fc3d0101e60 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.164+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0107560 0x7fc3d01958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.165+0000 7fc3cea00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 0x7fc3d019bc50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00dd10 tx=0x7fc3bc004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.165+0000 7fc3c7e00640 1 -- 172.21.15.37:0/1204176933 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0051f0 con 0x7fc3d0101e60 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.165+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d01962d0 con 0x7fc3d0101e60 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.165+0000 7fc3c7e00640 1 -- 172.21.15.37:0/1204176933 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc04a070 con 0x7fc3d0101e60 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.165+0000 7fc3c7e00640 1 -- 172.21.15.37:0/1204176933 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0459c0 con 0x7fc3d0101e60 2024-03-21T09:13:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.165+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d01a25d0 con 0x7fc3d0101e60 2024-03-21T09:13:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.169+0000 7fc3d9250740 1 Processor -- start 2024-03-21T09:13:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 -- start start 2024-03-21T09:13:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611ae5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611ae6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611af1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558611a4a250 con 0x558611aef460 2024-03-21T09:13:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586119f2a30 con 0x558611ae6dd0 2024-03-21T09:13:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.170+0000 7fc3d9250740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558611a462f0 con 0x558611ae60f0 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611ae6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611ae6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45414/0 (socket says 172.21.15.37:45414) 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 -- 172.21.15.37:0/2149831477 learned_addr learned my addr 172.21.15.37:0/2149831477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611ae5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 -- 172.21.15.37:0/2149831477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 msgr2=0x558611ae5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611ae5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 -- 172.21.15.37:0/2149831477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 msgr2=0x558611af1850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611af1850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 -- 172.21.15.37:0/2149831477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586119dd690 con 0x558611ae60f0 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3c7e00640 1 -- 172.21.15.37:0/1204176933 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fc3bc005390 con 0x7fc3d0101e60 2024-03-21T09:13:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611ae5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.171+0000 7fc3cf400640 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611ae6570 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009c80 tx=0x7fc3c0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=218fca524819176d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.172+0000 7fc3c4c00640 1 -- 172.21.15.37:0/2149831477 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001e070 con 0x558611ae60f0 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.172+0000 7fc3c4c00640 1 -- 172.21.15.37:0/2149831477 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3c0019b10 con 0x558611ae60f0 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.172+0000 7fc3c4c00640 1 -- 172.21.15.37:0/2149831477 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000ea30 con 0x558611ae60f0 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.172+0000 7fc3d9250740 1 -- 172.21.15.37:0/2149831477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 msgr2=0x558611ae6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.172+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611ae6570 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009c80 tx=0x7fc3c0011810 comp rx=0 tx=0).stop 2024-03-21T09:13:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3c7e00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc3b40a3230 0x7fc3b40a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3c7e00640 1 -- 172.21.15.37:0/1204176933 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc3bc0f9e20 con 0x7fc3d0101e60 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 -- 172.21.15.37:0/2149831477 shutdown_connections 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611af1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611ae6570 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2149831477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611ae5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 -- 172.21.15.37:0/2149831477 >> 172.21.15.37:0/2149831477 conn(0x558611aad200 msgr2=0x558611ae0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc3b40a3230 0x7fc3b40a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 -- 172.21.15.37:0/2149831477 shutdown_connections 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.173+0000 7fc3d9250740 1 -- 172.21.15.37:0/2149831477 wait complete. 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.174+0000 7fc3d9250740 1 -- start start 2024-03-21T09:13:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.174+0000 7fc3d9250740 1 Processor -- start 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc3b40a3230 0x7fc3b40a56f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8007e90 tx=0x7fc3c8007420 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611aeebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae6dd0 0x558611aebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611aef460 0x558611aec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558611af22d0 con 0x558611ae6dd0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558611a05e80 con 0x558611aef460 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558611af2450 con 0x558611ae60f0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3d9250740 10 client.0 _ll_register_callbacks cb 0x558611a18130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611aeebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611aeebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45418/0 (socket says 172.21.15.37:45418) 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3cf400640 1 -- 172.21.15.37:0/3289838908 learned_addr learned my addr 172.21.15.37:0/3289838908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.175+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611aef460 0x558611aec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cea00640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae6dd0 0x558611aebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cf400640 1 -- 172.21.15.37:0/3289838908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611aef460 msgr2=0x558611aec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cf400640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611aef460 0x558611aec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cf400640 1 -- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae6dd0 msgr2=0x558611aebf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cf400640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae6dd0 0x558611aebf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cf400640 1 -- 172.21.15.37:0/3289838908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586118287d0 con 0x558611ae60f0 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611aef460 0x558611aec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3cf400640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611aeebc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005bc0 tx=0x7fc3c0019430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ebff2ee5cfe865 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.176+0000 7fc3b9600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.177+0000 7fc3b9600640 1 -- 172.21.15.37:0/3289838908 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001e070 con 0x558611ae60f0 2024-03-21T09:13:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.177+0000 7fc3b9600640 1 -- 172.21.15.37:0/3289838908 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc3c0005030 con 0x558611ae60f0 2024-03-21T09:13:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3c0019b10 con 0x558611ae60f0 2024-03-21T09:13:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3b9600640 1 -- 172.21.15.37:0/3289838908 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c0005230 con 0x558611ae60f0 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3d9250740 10 client.25996 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55861175e3f0 con 0x558611ae60f0 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3b9600640 1 -- 172.21.15.37:0/3289838908 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3c0019b10 con 0x558611ae60f0 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3b9600640 1 -- 172.21.15.37:0/3289838908 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc3c00047e0 con 0x558611ae60f0 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3d9250740 10 client.25996 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.179+0000 7fc3d9250740 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.180+0000 7fc3d9250740 20 client.25996 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.180+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611aff2d0 0x558611b016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.180+0000 7fc3d9250740 4 client.25996 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.180+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558611a57790 con 0x558611aff2d0 2024-03-21T09:13:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.180+0000 7fc3cea00640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611aff2d0 0x558611b016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.181+0000 7fc3cea00640 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611aff2d0 0x558611b016c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9710fb995baaad5f server_cookie=b378393d660dc98f in_seq=0 out_seq=0 2024-03-21T09:13:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.181+0000 7fc3b9600640 10 client.25996 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3b9600640 1 -- 172.21.15.37:0/3289838908 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558611a57790 con 0x558611aff2d0 2024-03-21T09:13:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3b9600640 10 client.25996 handle_command_reply: tid=0 2024-03-21T09:13:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 client.25996 shutdown 2024-03-21T09:13:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 msgr2=0x558611aeebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611aeebc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005bc0 tx=0x7fc3c0019430 comp rx=0 tx=0).stop 2024-03-21T09:13:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 shutdown_connections 2024-03-21T09:13:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611aff2d0 0x558611b016c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611aef460 0x558611aec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae6dd0 0x558611aebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3289838908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae60f0 0x558611aeebc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 >> 172.21.15.37:0/3289838908 conn(0x558611aad200 msgr2=0x558611ae2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.182+0000 7fc3b9600640 0 client.25996 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.183+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 shutdown_connections 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.183+0000 7fc3d9250740 1 -- 172.21.15.37:0/3289838908 wait complete. 2024-03-21T09:13:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.183+0000 7fc3d9250740 20 client.25996 trim_cache size 0 max 16384 2024-03-21T09:13:37.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 Processor -- start 2024-03-21T09:13:37.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 -- start start 2024-03-21T09:13:37.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 0x558611aee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611aeebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611aef460 0x558611af0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01aeba0 con 0x558611ae60f0 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01ac6a0 con 0x558611ae6dd0 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3d9250740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01aa740 con 0x558611aef460 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.217+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 0x558611aee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 0x558611aee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49488/0 (socket says 172.21.15.37:49488) 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 -- 172.21.15.37:0/3302584246 learned_addr learned my addr 172.21.15.37:0/3302584246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611aef460 0x558611af0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cea00640 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611aeebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 -- 172.21.15.37:0/3302584246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611aef460 msgr2=0x558611af0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611aef460 0x558611af0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 -- 172.21.15.37:0/3302584246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 msgr2=0x558611aeebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611aeebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 -- 172.21.15.37:0/3302584246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55861171e0f0 con 0x558611ae60f0 2024-03-21T09:13:37.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.218+0000 7fc3cf400640 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 0x558611aee680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc3c001c040 tx=0x7fc3c0005120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcc58fb229c8899 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.219+0000 7fc3bbe00640 1 -- 172.21.15.37:0/3302584246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0036020 con 0x558611ae60f0 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.219+0000 7fc3bbe00640 1 -- 172.21.15.37:0/3302584246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3c0029070 con 0x558611ae60f0 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.219+0000 7fc3bbe00640 1 -- 172.21.15.37:0/3302584246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001e040 con 0x558611ae60f0 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 -- 172.21.15.37:0/3302584246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 msgr2=0x558611aee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 0x558611aee680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc3c001c040 tx=0x7fc3c0005120 comp rx=0 tx=0).stop 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 -- 172.21.15.37:0/3302584246 shutdown_connections 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611aef460 0x558611af0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae6dd0 0x558611aeebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 --2- 172.21.15.37:0/3302584246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611ae60f0 0x558611aee680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 -- 172.21.15.37:0/3302584246 >> 172.21.15.37:0/3302584246 conn(0x558611aad200 msgr2=0x558611ae0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 -- 172.21.15.37:0/3302584246 shutdown_connections 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.220+0000 7fc3d9250740 1 -- 172.21.15.37:0/3302584246 wait complete. 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.221+0000 7fc3d9250740 1 -- start start 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.221+0000 7fc3d9250740 1 Processor -- start 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 0x558611ae91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611ae9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae9c50 0x558611af9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01aad40 con 0x558611aef460 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01ac040 con 0x558611ae9c50 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01ab380 con 0x558611ae6dd0 2024-03-21T09:13:37.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3d9250740 10 client.0 _ll_register_callbacks cb 0x7fc3c001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 0x558611ae91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 0x558611ae91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45430/0 (socket says 172.21.15.37:45430) 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cf400640 1 -- 172.21.15.37:0/2165210945 learned_addr learned my addr 172.21.15.37:0/2165210945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cfe00640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae9c50 0x558611af9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cf400640 1 -- 172.21.15.37:0/2165210945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae9c50 msgr2=0x558611af9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cf400640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae9c50 0x558611af9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.222+0000 7fc3cf400640 1 -- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 msgr2=0x558611ae9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3cea00640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611ae9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3cf400640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611ae9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3cf400640 1 -- 172.21.15.37:0/2165210945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558611afa100 con 0x558611ae6dd0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3cea00640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611ae9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3cf400640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 0x558611ae91d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009f50 tx=0x7fc3c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e1a6a7b5828e19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3b9600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3b9600640 1 -- 172.21.15.37:0/2165210945 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001e070 con 0x558611ae6dd0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.223+0000 7fc3b9600640 1 -- 172.21.15.37:0/2165210945 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc3c000e670 con 0x558611ae6dd0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3c0029070 con 0x558611ae6dd0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3b9600640 1 -- 172.21.15.37:0/2165210945 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c0036050 con 0x558611ae6dd0 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 10 client.26002 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558611b5afe0 con 0x558611ae6dd0 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3b9600640 1 -- 172.21.15.37:0/2165210945 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3c0029070 con 0x558611ae6dd0 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3b9600640 1 -- 172.21.15.37:0/2165210945 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc3c002e540 con 0x558611ae6dd0 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 10 client.26002 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 20 client.26002 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611b5b540 0x558611b5d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 4 client.26002 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:13:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.224+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558611a51ae0 con 0x558611b5b540 2024-03-21T09:13:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.227+0000 7fc3cea00640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611b5b540 0x558611b5d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.228+0000 7fc3b9600640 10 client.26002 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.236+0000 7fc3cea00640 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611b5b540 0x558611b5d930 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d347110c70e803 server_cookie=ab559767d33ab900 in_seq=0 out_seq=0 2024-03-21T09:13:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.236+0000 7fc3b9600640 1 -- 172.21.15.37:0/2165210945 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558611a51ae0 con 0x558611b5b540 2024-03-21T09:13:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.236+0000 7fc3b9600640 10 client.26002 handle_command_reply: tid=0 2024-03-21T09:13:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.236+0000 7fc3d9250740 1 client.26002 shutdown 2024-03-21T09:13:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.236+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 msgr2=0x558611ae91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.236+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 0x558611ae91d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009f50 tx=0x7fc3c0009b50 comp rx=0 tx=0).stop 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 shutdown_connections 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558611b5b540 0x558611b5d930 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558611ae9c50 0x558611af9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558611aef460 0x558611ae9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 --2- 172.21.15.37:0/2165210945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558611ae6dd0 0x558611ae91d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3b9600640 0 client.26002 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 >> 172.21.15.37:0/2165210945 conn(0x558611aad200 msgr2=0x558611aea6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 shutdown_connections 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 1 -- 172.21.15.37:0/2165210945 wait complete. 2024-03-21T09:13:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.237+0000 7fc3d9250740 20 client.26002 trim_cache size 0 max 16384 2024-03-21T09:13:37.233 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:13:37.233 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:13:37.233 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:13:37.233 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:13:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc3b40a3230 msgr2=0x7fc3b40a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc3b40a3230 0x7fc3b40a56f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8007e90 tx=0x7fc3c8007420 comp rx=0 tx=0).stop 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 msgr2=0x7fc3d019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 0x7fc3d019bc50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00dd10 tx=0x7fc3bc004d30 comp rx=0 tx=0).stop 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 shutdown_connections 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0107560 0x7fc3d01958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc3b40a3230 0x7fc3b40a56f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3d0101e60 0x7fc3d019bc50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 --2- 172.21.15.37:0/1204176933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3d01014a0 0x7fc3d019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.238+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 >> 172.21.15.37:0/1204176933 conn(0x7fc3d00fcfb0 msgr2=0x7fc3d00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.239+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 shutdown_connections 2024-03-21T09:13:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.239+0000 7fc3d5000640 1 -- 172.21.15.37:0/1204176933 wait complete. 2024-03-21T09:13:37.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:13:37.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 -- 172.21.15.37:0/1509309717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc244102700 msgr2=0x7fc244102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 --2- 172.21.15.37:0/1509309717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc244102700 0x7fc244102ae0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc238009900 tx=0x7fc23802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 -- 172.21.15.37:0/1509309717 shutdown_connections 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 --2- 172.21.15.37:0/1509309717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24410bb20 0x7fc24410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 --2- 172.21.15.37:0/1509309717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc244103020 0x7fc24410b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 --2- 172.21.15.37:0/1509309717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc244102700 0x7fc244102ae0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 -- 172.21.15.37:0/1509309717 >> 172.21.15.37:0/1509309717 conn(0x7fc2440fbf00 msgr2=0x7fc2440fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 -- 172.21.15.37:0/1509309717 shutdown_connections 2024-03-21T09:13:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.514+0000 7fc24a200640 1 -- 172.21.15.37:0/1509309717 wait complete. 2024-03-21T09:13:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.516+0000 7fc24a200640 1 Processor -- start 2024-03-21T09:13:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.516+0000 7fc24a200640 1 -- start start 2024-03-21T09:13:37.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.516+0000 7fc24a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc244102700 0x7fc24419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.516+0000 7fc24a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 0x7fc24419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc24a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24410bb20 0x7fc24410a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc24a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc244111930 con 0x7fc244102700 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc24a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2441117b0 con 0x7fc244103020 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc24a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc244111ab0 con 0x7fc24410bb20 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc243e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 0x7fc24419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc243e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 0x7fc24419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44864/0 (socket says 172.21.15.37:44864) 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc243e00640 1 -- 172.21.15.37:0/3065328708 learned_addr learned my addr 172.21.15.37:0/3065328708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc249800640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24410bb20 0x7fc24410a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.517+0000 7fc248e00640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc244102700 0x7fc24419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc249800640 1 -- 172.21.15.37:0/3065328708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 msgr2=0x7fc24419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc249800640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 0x7fc24419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc249800640 1 -- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc244102700 msgr2=0x7fc24419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc249800640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc244102700 0x7fc24419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc249800640 1 -- 172.21.15.37:0/3065328708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24410ae40 con 0x7fc24410bb20 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc243e00640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 0x7fc24419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc249800640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24410bb20 0x7fc24410a580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00cab0 tx=0x7fc23c00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00ae20 con 0x7fc24410bb20 2024-03-21T09:13:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.518+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24410b130 con 0x7fc24410bb20 2024-03-21T09:13:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.519+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24419e710 con 0x7fc24410bb20 2024-03-21T09:13:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.519+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c00f930 con 0x7fc24410bb20 2024-03-21T09:13:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.519+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c018b30 con 0x7fc24410bb20 2024-03-21T09:13:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.521+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224005290 con 0x7fc24410bb20 2024-03-21T09:13:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.524+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fc23c00fae0 con 0x7fc24410bb20 2024-03-21T09:13:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.526+0000 7fc241600640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc22c0a32a0 0x7fc22c0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.526+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc23c0d03b0 con 0x7fc24410bb20 2024-03-21T09:13:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.526+0000 7fc248e00640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc22c0a32a0 0x7fc22c0a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.527+0000 7fc248e00640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc22c0a32a0 0x7fc22c0a5760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc238000c00 tx=0x7fc238004f10 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:37.533+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc23c014030 con 0x7fc24410bb20 2024-03-21T09:13:38.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.011+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc224005850 con 0x7fc24410bb20 2024-03-21T09:13:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.012+0000 7fc241600640 1 -- 172.21.15.37:0/3065328708 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fc23c09c850 con 0x7fc24410bb20 2024-03-21T09:13:38.006 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:13:38.006 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:11:48.827236+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:13:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-03-21T09:13:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc22c0a32a0 msgr2=0x7fc22c0a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc22c0a32a0 0x7fc22c0a5760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc238000c00 tx=0x7fc238004f10 comp rx=0 tx=0).stop 2024-03-21T09:13:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24410bb20 msgr2=0x7fc24410a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24410bb20 0x7fc24410a580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00cab0 tx=0x7fc23c00cf70 comp rx=0 tx=0).stop 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 shutdown_connections 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24410bb20 0x7fc24410a580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc22c0a32a0 0x7fc22c0a5760 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc244103020 0x7fc24419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 --2- 172.21.15.37:0/3065328708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc244102700 0x7fc24419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 >> 172.21.15.37:0/3065328708 conn(0x7fc2440fbf00 msgr2=0x7fc24410dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.015+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 shutdown_connections 2024-03-21T09:13:38.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.016+0000 7fc24a200640 1 -- 172.21.15.37:0/3065328708 wait complete. 2024-03-21T09:13:38.034 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:13:38.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:13:38.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.252+0000 7fda2fe00640 1 -- 172.21.15.37:0/144213287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30074390 msgr2=0x7fda3010d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.252+0000 7fda2fe00640 1 --2- 172.21.15.37:0/144213287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30074390 0x7fda3010d7e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009900 tx=0x7fda1c02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 -- 172.21.15.37:0/144213287 shutdown_connections 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 --2- 172.21.15.37:0/144213287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda3010dd20 0x7fda30110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 --2- 172.21.15.37:0/144213287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30074390 0x7fda3010d7e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 --2- 172.21.15.37:0/144213287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30073a70 0x7fda30073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 -- 172.21.15.37:0/144213287 >> 172.21.15.37:0/144213287 conn(0x7fda300fcfb0 msgr2=0x7fda300ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 -- 172.21.15.37:0/144213287 shutdown_connections 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.253+0000 7fda2fe00640 1 -- 172.21.15.37:0/144213287 wait complete. 2024-03-21T09:13:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.254+0000 7fda2fe00640 1 Processor -- start 2024-03-21T09:13:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.254+0000 7fda2fe00640 1 -- start start 2024-03-21T09:13:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda30073a70 0x7fda30199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30074390 0x7fda3019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 0x7fda3019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30113a80 con 0x7fda30074390 2024-03-21T09:13:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30113900 con 0x7fda3010dd20 2024-03-21T09:13:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30113c00 con 0x7fda30073a70 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30074390 0x7fda3019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 0x7fda3019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30074390 0x7fda3019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49550/0 (socket says 172.21.15.37:49550) 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2e000640 1 -- 172.21.15.37:0/2827194171 learned_addr learned my addr 172.21.15.37:0/2827194171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.255+0000 7fda2f400640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 0x7fda3019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44878/0 (socket says 172.21.15.37:44878) 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2ea00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda30073a70 0x7fda30199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2f400640 1 -- 172.21.15.37:0/2827194171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda30073a70 msgr2=0x7fda30199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2f400640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda30073a70 0x7fda30199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2f400640 1 -- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30074390 msgr2=0x7fda3019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2f400640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30074390 0x7fda3019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2f400640 1 -- 172.21.15.37:0/2827194171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3019ed80 con 0x7fda3010dd20 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda2f400640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 0x7fda3019e680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda28005fd0 tx=0x7fda2800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda34a00640 1 -- 172.21.15.37:0/2827194171 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800cdf0 con 0x7fda3010dd20 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda34a00640 1 -- 172.21.15.37:0/2827194171 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda28005020 con 0x7fda3010dd20 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.256+0000 7fda34a00640 1 -- 172.21.15.37:0/2827194171 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28010670 con 0x7fda3010dd20 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.257+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3019f070 con 0x7fda3010dd20 2024-03-21T09:13:38.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.257+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3006b0c0 con 0x7fda3010dd20 2024-03-21T09:13:38.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.260+0000 7fda34a00640 1 -- 172.21.15.37:0/2827194171 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fda28005540 con 0x7fda3010dd20 2024-03-21T09:13:38.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.261+0000 7fda38a16740 1 Processor -- start 2024-03-21T09:13:38.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.261+0000 7fda34a00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fda140a3230 0x7fda140a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 -- start start 2024-03-21T09:13:38.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda34a00640 1 -- 172.21.15.37:0/2827194171 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda280d01e0 con 0x7fda3010dd20 2024-03-21T09:13:38.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e22de0 0x55c765e25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c765d7d2c0 con 0x55c765e22de0 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c765d25c90 con 0x55c765e1a260 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda38a16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c765d79360 con 0x55c765e198b0 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda2e000640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fda140a3230 0x7fda140a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.262+0000 7fda2e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e22810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44894/0 (socket says 172.21.15.37:44894) 2024-03-21T09:13:38.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 -- 172.21.15.37:0/1193317909 learned_addr learned my addr 172.21.15.37:0/1193317909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2f400640 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e22de0 0x55c765e25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 -- 172.21.15.37:0/1193317909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 msgr2=0x55c765e19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 -- 172.21.15.37:0/1193317909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e22de0 msgr2=0x55c765e25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e22de0 0x55c765e25230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2e000640 1 -- 172.21.15.37:0/1193317909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c765d4a080 con 0x55c765e1a260 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.263+0000 7fda2f400640 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e22de0 0x55c765e25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.264+0000 7fda2e000640 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e22810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda1c004040 tx=0x7fda1c004070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a668d5093f53e77c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.264+0000 7fda2e000640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fda140a3230 0x7fda140a56f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0040e0 tx=0x7fda1c03b000 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.264+0000 7fda1be00640 1 -- 172.21.15.37:0/1193317909 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c005d50 con 0x55c765e1a260 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.264+0000 7fda1be00640 1 -- 172.21.15.37:0/1193317909 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda1c051070 con 0x55c765e1a260 2024-03-21T09:13:38.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.264+0000 7fda1be00640 1 -- 172.21.15.37:0/1193317909 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0035f0 con 0x55c765e1a260 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 -- 172.21.15.37:0/1193317909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 msgr2=0x55c765e22810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e22810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda1c004040 tx=0x7fda1c004070 comp rx=0 tx=0).stop 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 -- 172.21.15.37:0/1193317909 shutdown_connections 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e22de0 0x55c765e25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e22810 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 --2- 172.21.15.37:0/1193317909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 -- 172.21.15.37:0/1193317909 >> 172.21.15.37:0/1193317909 conn(0x55c765e13f80 msgr2=0x55c765e14360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 -- 172.21.15.37:0/1193317909 shutdown_connections 2024-03-21T09:13:38.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.265+0000 7fda38a16740 1 -- 172.21.15.37:0/1193317909 wait complete. 2024-03-21T09:13:38.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.266+0000 7fda38a16740 1 -- start start 2024-03-21T09:13:38.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.266+0000 7fda38a16740 1 Processor -- start 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e1a260 0x55c765e1fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e20260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c765e25cf0 con 0x55c765e198b0 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c765d390d0 con 0x55c765e22de0 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c765e25e70 con 0x55c765e1a260 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda38a16740 10 client.0 _ll_register_callbacks cb 0x55c765d3de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49556/0 (socket says 172.21.15.37:49556) 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda2ea00640 1 -- 172.21.15.37:0/1191842948 learned_addr learned my addr 172.21.15.37:0/1191842948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.267+0000 7fda2e000640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e1a260 0x55c765e1fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e20260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 -- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e1a260 msgr2=0x55c765e1fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e1a260 0x55c765e1fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 -- 172.21.15.37:0/1191842948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 msgr2=0x55c765e227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 -- 172.21.15.37:0/1191842948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c765a913f0 con 0x55c765e22de0 2024-03-21T09:13:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2e000640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e1a260 0x55c765e1fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2ea00640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.268+0000 7fda2f400640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e20260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda280cd3b0 tx=0x7fda2809b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23161e2e75d76df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda18c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda18c00640 1 -- 172.21.15.37:0/1191842948 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280a4990 con 0x55c765e22de0 2024-03-21T09:13:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda18c00640 1 -- 172.21.15.37:0/1191842948 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda280985f0 con 0x55c765e22de0 2024-03-21T09:13:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda1c051070 con 0x55c765e22de0 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda18c00640 1 -- 172.21.15.37:0/1191842948 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda280cd690 con 0x55c765e22de0 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda38a16740 10 client.25991 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c765b5b7d0 con 0x55c765e22de0 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.269+0000 7fda18c00640 1 -- 172.21.15.37:0/1191842948 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda1c051070 con 0x55c765e22de0 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda18c00640 1 -- 172.21.15.37:0/1191842948 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fda2801f020 con 0x55c765e22de0 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda38a16740 10 client.25991 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda38a16740 10 client.25991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda38a16740 20 client.25991 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda38a16740 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e3b420 0x55c765e3d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda38a16740 4 client.25991 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.270+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c765d8a800 con 0x55c765e3b420 2024-03-21T09:13:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.271+0000 7fda2ea00640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e3b420 0x55c765e3d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.271+0000 7fda2ea00640 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e3b420 0x55c765e3d810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f32547965c74b55 server_cookie=d6fa687747dac60c in_seq=0 out_seq=0 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.271+0000 7fda18c00640 10 client.25991 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda18c00640 1 -- 172.21.15.37:0/1191842948 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c765d8a800 con 0x55c765e3b420 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda18c00640 10 client.25991 handle_command_reply: tid=0 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 client.25991 shutdown 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 msgr2=0x55c765e20260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e20260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda280cd3b0 tx=0x7fda2809b9f0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 shutdown_connections 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e20260 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e1a260 0x55c765e1fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e3b420 0x55c765e3d810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 --2- 172.21.15.37:0/1191842948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 >> 172.21.15.37:0/1191842948 conn(0x55c765e13f80 msgr2=0x55c765e15cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda18c00640 0 client.25991 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 shutdown_connections 2024-03-21T09:13:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.272+0000 7fda38a16740 1 -- 172.21.15.37:0/1191842948 wait complete. 2024-03-21T09:13:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.273+0000 7fda38a16740 20 client.25991 trim_cache size 0 max 16384 2024-03-21T09:13:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 Processor -- start 2024-03-21T09:13:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 -- start start 2024-03-21T09:13:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e1eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e12fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e22de0 0x55c765e222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30073080 con 0x55c765e198b0 2024-03-21T09:13:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30070b80 con 0x55c765e1a260 2024-03-21T09:13:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.287+0000 7fda38a16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3006ec80 con 0x55c765e22de0 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e12fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e12fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44910/0 (socket says 172.21.15.37:44910) 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 -- 172.21.15.37:0/2567931538 learned_addr learned my addr 172.21.15.37:0/2567931538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2f400640 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e22de0 0x55c765e222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2ea00640 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e1eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 -- 172.21.15.37:0/2567931538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e22de0 msgr2=0x55c765e222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e22de0 0x55c765e222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 -- 172.21.15.37:0/2567931538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 msgr2=0x55c765e1eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e1eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2e000640 1 -- 172.21.15.37:0/2567931538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c765e0f600 con 0x55c765e1a260 2024-03-21T09:13:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.288+0000 7fda2ea00640 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e1eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.289+0000 7fda2e000640 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e12fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0054a0 tx=0x7fda1c02df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6eebcfbcbbb01cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.289+0000 7fda1b400640 1 -- 172.21.15.37:0/2567931538 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c005d50 con 0x55c765e1a260 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.289+0000 7fda1b400640 1 -- 172.21.15.37:0/2567931538 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda1c051040 con 0x55c765e1a260 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.289+0000 7fda1b400640 1 -- 172.21.15.37:0/2567931538 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c003850 con 0x55c765e1a260 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 -- 172.21.15.37:0/2567931538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 msgr2=0x55c765e12fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e12fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0054a0 tx=0x7fda1c02df90 comp rx=0 tx=0).stop 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 -- 172.21.15.37:0/2567931538 shutdown_connections 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e22de0 0x55c765e222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e1a260 0x55c765e12fe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 --2- 172.21.15.37:0/2567931538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e198b0 0x55c765e1eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 -- 172.21.15.37:0/2567931538 >> 172.21.15.37:0/2567931538 conn(0x55c765e13f80 msgr2=0x55c765e128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 -- 172.21.15.37:0/2567931538 shutdown_connections 2024-03-21T09:13:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 -- 172.21.15.37:0/2567931538 wait complete. 2024-03-21T09:13:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.290+0000 7fda38a16740 1 -- start start 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 Processor -- start 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e1a260 0x55c765e33550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e30920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3006f280 con 0x55c765e1a260 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30070520 con 0x55c765e22de0 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3006f880 con 0x55c765e198b0 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda38a16740 10 client.0 _ll_register_callbacks cb 0x7fda28005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.291+0000 7fda2f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e30920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e1a260 0x55c765e33550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e33010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45512/0 (socket says 172.21.15.37:45512) 2024-03-21T09:13:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e30920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44920/0 (socket says 172.21.15.37:44920) 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2ea00640 1 -- 172.21.15.37:0/3889516757 learned_addr learned my addr 172.21.15.37:0/3889516757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 -- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 msgr2=0x55c765e33010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e33010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 -- 172.21.15.37:0/3889516757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e1a260 msgr2=0x55c765e33550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e1a260 0x55c765e33550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 -- 172.21.15.37:0/3889516757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c765e310b0 con 0x55c765e22de0 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda2f400640 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e30920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fda28098950 tx=0x7fda28005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355a70e533eaccde server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.292+0000 7fda18c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:38.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda18c00640 1 -- 172.21.15.37:0/3889516757 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280a4990 con 0x55c765e22de0 2024-03-21T09:13:38.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda18c00640 1 -- 172.21.15.37:0/3889516757 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda2809b3d0 con 0x55c765e22de0 2024-03-21T09:13:38.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda1c051040 con 0x55c765e22de0 2024-03-21T09:13:38.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda18c00640 1 -- 172.21.15.37:0/3889516757 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda28084050 con 0x55c765e22de0 2024-03-21T09:13:38.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda38a16740 10 client.25997 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c765e966c0 con 0x55c765e22de0 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda18c00640 1 -- 172.21.15.37:0/3889516757 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda1c051040 con 0x55c765e22de0 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.293+0000 7fda18c00640 1 -- 172.21.15.37:0/3889516757 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fda2801f070 con 0x55c765e22de0 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda38a16740 10 client.25997 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda38a16740 10 client.25997 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda38a16740 20 client.25997 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda38a16740 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e96d60 0x55c765e99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda38a16740 4 client.25997 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c765d84b50 con 0x55c765e96d60 2024-03-21T09:13:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.294+0000 7fda2ea00640 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e96d60 0x55c765e99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda18c00640 10 client.25997 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda2ea00640 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e96d60 0x55c765e99180 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf0595453302d9c9 server_cookie=bbe60230039dffcd in_seq=0 out_seq=0 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda18c00640 1 -- 172.21.15.37:0/3889516757 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c765d84b50 con 0x55c765e96d60 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda18c00640 10 client.25997 handle_command_reply: tid=0 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 client.25997 shutdown 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 msgr2=0x55c765e30920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e30920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fda28098950 tx=0x7fda28005940 comp rx=0 tx=0).stop 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 shutdown_connections 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c765e22de0 0x55c765e30920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c765e96d60 0x55c765e99180 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c765e1a260 0x55c765e33550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 --2- 172.21.15.37:0/3889516757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c765e198b0 0x55c765e33010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 >> 172.21.15.37:0/3889516757 conn(0x55c765e13f80 msgr2=0x55c765e13160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.296+0000 7fda18c00640 0 client.25997 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:13:38.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.297+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 shutdown_connections 2024-03-21T09:13:38.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.297+0000 7fda38a16740 1 -- 172.21.15.37:0/3889516757 wait complete. 2024-03-21T09:13:38.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.297+0000 7fda38a16740 20 client.25997 trim_cache size 0 max 16384 2024-03-21T09:13:38.293 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fda140a3230 msgr2=0x7fda140a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fda140a3230 0x7fda140a56f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0040e0 tx=0x7fda1c03b000 comp rx=0 tx=0).stop 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 msgr2=0x7fda3019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 0x7fda3019e680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda28005fd0 tx=0x7fda2800ef10 comp rx=0 tx=0).stop 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 shutdown_connections 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fda140a3230 0x7fda140a56f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda3010dd20 0x7fda3019e680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda30074390 0x7fda3019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 --2- 172.21.15.37:0/2827194171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda30073a70 0x7fda30199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 >> 172.21.15.37:0/2827194171 conn(0x7fda300fcfb0 msgr2=0x7fda3010f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 shutdown_connections 2024-03-21T09:13:38.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.298+0000 7fda2fe00640 1 -- 172.21.15.37:0/2827194171 wait complete. 2024-03-21T09:13:38.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:38 smithi037 ceph-mon[31830]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:38.311 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3065328708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:13:38.312 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:13:38.312 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:13:38.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:38 smithi053 ceph-mon[36169]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:38.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3065328708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:13:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 -- 172.21.15.37:0/2081265903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c1014a0 msgr2=0x7f277c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 --2- 172.21.15.37:0/2081265903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c1014a0 0x7f277c101920 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2768009930 tx=0x7f276802bc20 comp rx=0 tx=0).stop 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 -- 172.21.15.37:0/2081265903 shutdown_connections 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 --2- 172.21.15.37:0/2081265903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f277c101e60 0x7f277c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 --2- 172.21.15.37:0/2081265903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c1014a0 0x7f277c101920 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 --2- 172.21.15.37:0/2081265903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277c107560 0x7f277c107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 -- 172.21.15.37:0/2081265903 >> 172.21.15.37:0/2081265903 conn(0x7f277c0fcfb0 msgr2=0x7f277c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 -- 172.21.15.37:0/2081265903 shutdown_connections 2024-03-21T09:13:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.536+0000 7f277be00640 1 -- 172.21.15.37:0/2081265903 wait complete. 2024-03-21T09:13:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.537+0000 7f277be00640 1 Processor -- start 2024-03-21T09:13:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.537+0000 7f277be00640 1 -- start start 2024-03-21T09:13:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.537+0000 7f277be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277c1014a0 0x7f277c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.537+0000 7f277be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 0x7f277c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.537+0000 7f277be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f277c107560 0x7f277c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c111870 con 0x7f277c1014a0 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1116f0 con 0x7f277c107560 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1119f0 con 0x7f277c101e60 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 0x7f277c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277c1014a0 0x7f277c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 0x7f277c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45524/0 (socket says 172.21.15.37:45524) 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 -- 172.21.15.37:0/3647804051 learned_addr learned my addr 172.21.15.37:0/3647804051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277b400640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f277c107560 0x7f277c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 -- 172.21.15.37:0/3647804051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f277c107560 msgr2=0x7f277c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f277c107560 0x7f277c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 -- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277c1014a0 msgr2=0x7f277c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277c1014a0 0x7f277c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.538+0000 7f277a000640 1 -- 172.21.15.37:0/3647804051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c19eee0 con 0x7f277c101e60 2024-03-21T09:13:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.539+0000 7f277a000640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 0x7f277c19a430 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2768005ab0 tx=0x7f27680042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.539+0000 7f2773400640 1 -- 172.21.15.37:0/3647804051 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768043020 con 0x7f277c101e60 2024-03-21T09:13:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.539+0000 7f2773400640 1 -- 172.21.15.37:0/3647804051 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27680044d0 con 0x7f277c101e60 2024-03-21T09:13:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.539+0000 7f2773400640 1 -- 172.21.15.37:0/3647804051 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768003b30 con 0x7f277c101e60 2024-03-21T09:13:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.539+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c19f170 con 0x7f277c101e60 2024-03-21T09:13:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.539+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c1a6a60 con 0x7f277c101e60 2024-03-21T09:13:38.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 Processor -- start 2024-03-21T09:13:38.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 -- start start 2024-03-21T09:13:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd479dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd47a6de0 0x55acd47a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd47012c0 con 0x55acd479d8b0 2024-03-21T09:13:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd46a9c90 con 0x55acd479e260 2024-03-21T09:13:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.544+0000 7f27843a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd46fd360 con 0x55acd47a6de0 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f2773400640 1 -- 172.21.15.37:0/3647804051 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f2768046020 con 0x7f277c101e60 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44928/0 (socket says 172.21.15.37:44928) 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 -- 172.21.15.37:0/1634213803 learned_addr learned my addr 172.21.15.37:0/1634213803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277a000640 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd47a6de0 0x55acd47a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 -- 172.21.15.37:0/1634213803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd47a6de0 msgr2=0x55acd47a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd47a6de0 0x55acd47a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 -- 172.21.15.37:0/1634213803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 msgr2=0x55acd479dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd479dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.545+0000 7f277aa00640 1 -- 172.21.15.37:0/1634213803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acd46ce080 con 0x55acd479e260 2024-03-21T09:13:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.546+0000 7f277aa00640 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f276c00ccc0 tx=0x7f276c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811d0523ac06f6be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.546+0000 7f2767400640 1 -- 172.21.15.37:0/1634213803 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c017bd0 con 0x55acd479e260 2024-03-21T09:13:38.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.546+0000 7f2767400640 1 -- 172.21.15.37:0/1634213803 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f276c017d70 con 0x55acd479e260 2024-03-21T09:13:38.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.546+0000 7f2767400640 1 -- 172.21.15.37:0/1634213803 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c01e380 con 0x55acd479e260 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.547+0000 7f2773400640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27600a32b0 0x7f27600a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.547+0000 7f27843a7740 1 -- 172.21.15.37:0/1634213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 msgr2=0x55acd47a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.547+0000 7f27843a7740 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f276c00ccc0 tx=0x7f276c00e720 comp rx=0 tx=0).stop 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.547+0000 7f2773400640 1 -- 172.21.15.37:0/3647804051 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f27680f1d30 con 0x7f277c101e60 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 -- 172.21.15.37:0/1634213803 shutdown_connections 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd47a6de0 0x55acd47a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6810 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 --2- 172.21.15.37:0/1634213803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd479dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 -- 172.21.15.37:0/1634213803 >> 172.21.15.37:0/1634213803 conn(0x55acd4797f80 msgr2=0x55acd4798360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f277b400640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27600a32b0 0x7f27600a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 -- 172.21.15.37:0/1634213803 shutdown_connections 2024-03-21T09:13:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.548+0000 7f27843a7740 1 -- 172.21.15.37:0/1634213803 wait complete. 2024-03-21T09:13:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.549+0000 7f277b400640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27600a32b0 0x7f27600a5770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27740099a0 tx=0x7f2774004a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.549+0000 7f27843a7740 1 -- start start 2024-03-21T09:13:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.549+0000 7f27843a7740 1 Processor -- start 2024-03-21T09:13:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f27843a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd47a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f27843a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 0x55acd47a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f27843a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f277a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 0x55acd47a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f277a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 0x55acd47a3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45530/0 (socket says 172.21.15.37:45530) 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f277aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd47a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f277aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd47a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49608/0 (socket says 172.21.15.37:49608) 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f27843a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd47a9d50 con 0x55acd479d8b0 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f277a000640 1 -- 172.21.15.37:0/1803300033 learned_addr learned my addr 172.21.15.37:0/1803300033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f277b400640 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd46bd0d0 con 0x55acd47a6de0 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.550+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd47a9ed0 con 0x55acd479e260 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f277a000640 1 -- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 msgr2=0x55acd47a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f277a000640 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f277a000640 1 -- 172.21.15.37:0/1803300033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 msgr2=0x55acd47a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f277a000640 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd47a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f277a000640 1 -- 172.21.15.37:0/1803300033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acd44153f0 con 0x55acd479e260 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f27843a7740 10 client.0 _ll_register_callbacks cb 0x55acd46c1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f276c017d70 con 0x55acd479e260 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f277a000640 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 0x55acd47a3af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f276802c810 tx=0x7f2768009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4126a2b79aabf91 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.551+0000 7f275be00640 10 client.26038 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f275be00640 1 -- 172.21.15.37:0/1803300033 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680bd540 con 0x55acd479e260 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f275be00640 1 -- 172.21.15.37:0/1803300033 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f276802cb10 con 0x55acd479e260 2024-03-21T09:13:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f275be00640 1 -- 172.21.15.37:0/1803300033 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27680be2e0 con 0x55acd479e260 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 10 client.26038 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x55acd44df7d0 con 0x55acd479e260 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f275be00640 1 -- 172.21.15.37:0/1803300033 <== mon.1 v2:172.21.15.71:3300/0 4 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f27680f7070 con 0x55acd479e260 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 10 client.26038 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 10 client.26038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 20 client.26038 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd47b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 4 client.26038 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.552+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acd470e800 con 0x55acd47b6a90 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.553+0000 7f277aa00640 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd47b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.553+0000 7f275be00640 10 client.26038 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.553+0000 7f277aa00640 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd47b8ea0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d43e7f9d5561e63a server_cookie=30dad1189840affe in_seq=0 out_seq=0 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.554+0000 7f275be00640 1 -- 172.21.15.37:0/1803300033 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55acd470e800 con 0x55acd47b6a90 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.554+0000 7f275be00640 10 client.26038 handle_command_reply: tid=0 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.554+0000 7f27843a7740 1 client.26038 shutdown 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.554+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 msgr2=0x55acd47a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.554+0000 7f27843a7740 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 0x55acd47a3af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f276802c810 tx=0x7f2768009620 comp rx=0 tx=0).stop 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 shutdown_connections 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd47b8ea0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f275be00640 0 client.26038 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479e260 0x55acd47a3af0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 --2- 172.21.15.37:0/1803300033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479d8b0 0x55acd47a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 >> 172.21.15.37:0/1803300033 conn(0x55acd4797f80 msgr2=0x55acd4799cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 shutdown_connections 2024-03-21T09:13:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 1 -- 172.21.15.37:0/1803300033 wait complete. 2024-03-21T09:13:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.555+0000 7f27843a7740 20 client.26038 trim_cache size 0 max 16384 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.568+0000 7f27843a7740 1 Processor -- start 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.568+0000 7f27843a7740 1 -- start start 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f27843a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd4796fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f27843a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f27843a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd47a6de0 0x55acd47a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f27843a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1aead0 con 0x55acd47a6de0 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f27843a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1ac5d0 con 0x55acd479e260 2024-03-21T09:13:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f27843a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1aa6b0 con 0x55acd479d8b0 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd4796fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44932/0 (socket says 172.21.15.37:44932) 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277a000640 1 -- 172.21.15.37:0/2539320147 learned_addr learned my addr 172.21.15.37:0/2539320147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277aa00640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd4796fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45546/0 (socket says 172.21.15.37:45546) 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277a000640 1 -- 172.21.15.37:0/2539320147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 msgr2=0x55acd4796fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277b400640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd47a6de0 0x55acd47a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277a000640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd4796fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.569+0000 7f277a000640 1 -- 172.21.15.37:0/2539320147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd47a6de0 msgr2=0x55acd47a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f277a000640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd47a6de0 0x55acd47a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f277a000640 1 -- 172.21.15.37:0/2539320147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acd47938b0 con 0x55acd479e260 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f277aa00640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd4796fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f277b400640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd47a6de0 0x55acd47a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:38.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f277a000640 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f27680bdb70 tx=0x7f276804a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98730f0af22f8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f2766a00640 1 -- 172.21.15.37:0/2539320147 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680bd540 con 0x55acd479e260 2024-03-21T09:13:38.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f2766a00640 1 -- 172.21.15.37:0/2539320147 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f276c017d70 con 0x55acd479e260 2024-03-21T09:13:38.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.570+0000 7f2766a00640 1 -- 172.21.15.37:0/2539320147 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680bd9b0 con 0x55acd479e260 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 -- 172.21.15.37:0/2539320147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 msgr2=0x55acd47a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f27680bdb70 tx=0x7f276804a000 comp rx=0 tx=0).stop 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 -- 172.21.15.37:0/2539320147 shutdown_connections 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd47a6de0 0x55acd47a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd479e260 0x55acd47a6220 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 --2- 172.21.15.37:0/2539320147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd4796fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 -- 172.21.15.37:0/2539320147 >> 172.21.15.37:0/2539320147 conn(0x55acd4797f80 msgr2=0x55acd47968d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 -- 172.21.15.37:0/2539320147 shutdown_connections 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.571+0000 7f27843a7740 1 -- 172.21.15.37:0/2539320147 wait complete. 2024-03-21T09:13:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 -- start start 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 Processor -- start 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd47aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 0x55acd47aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1aacb0 con 0x55acd479e260 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1abf70 con 0x55acd47a6de0 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1ab2b0 con 0x55acd479d8b0 2024-03-21T09:13:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.572+0000 7f27843a7740 10 client.0 _ll_register_callbacks cb 0x7f27680ef0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 0x55acd47aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 0x55acd47aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49622/0 (socket says 172.21.15.37:49622) 2024-03-21T09:13:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 -- 172.21.15.37:0/2736368585 learned_addr learned my addr 172.21.15.37:0/2736368585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277aa00640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd47aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 -- 172.21.15.37:0/2736368585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 msgr2=0x55acd47aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277b400640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd47aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 -- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 msgr2=0x55acd47acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 -- 172.21.15.37:0/2736368585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acd47ad730 con 0x55acd479e260 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277aa00640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd47aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277b400640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f277a000640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 0x55acd47aefe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27680bd8e0 tx=0x7f276802bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d235d05ccd3139ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.573+0000 7f275be00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:13:38.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f275be00640 1 -- 172.21.15.37:0/2736368585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680bd9b0 con 0x55acd479e260 2024-03-21T09:13:38.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f275be00640 1 -- 172.21.15.37:0/2736368585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f27680bd030 con 0x55acd479e260 2024-03-21T09:13:38.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f276c017d70 con 0x55acd479e260 2024-03-21T09:13:38.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f275be00640 1 -- 172.21.15.37:0/2736368585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27680a3050 con 0x55acd479e260 2024-03-21T09:13:38.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f27843a7740 10 client.15852 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55acd47ad730 con 0x55acd479e260 2024-03-21T09:13:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.574+0000 7f275be00640 1 -- 172.21.15.37:0/2736368585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f276c017d70 con 0x55acd479e260 2024-03-21T09:13:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f275be00640 1 -- 172.21.15.37:0/2736368585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f27680f7040 con 0x55acd479e260 2024-03-21T09:13:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f27843a7740 10 client.15852 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f27843a7740 10 client.15852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:13:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f27843a7740 20 client.15852 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f27843a7740 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd4815120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f27843a7740 4 client.15852 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:13:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acd4708b50 con 0x55acd47b6a90 2024-03-21T09:13:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.575+0000 7f277aa00640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd4815120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f275be00640 10 client.15852 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f277aa00640 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd4815120 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4444e688fe1f066 server_cookie=e30cbe15a6f7adbf in_seq=0 out_seq=0 2024-03-21T09:13:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f275be00640 1 -- 172.21.15.37:0/2736368585 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acd4708b50 con 0x55acd47b6a90 2024-03-21T09:13:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f275be00640 10 client.15852 handle_command_reply: tid=0 2024-03-21T09:13:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f27843a7740 1 client.15852 shutdown 2024-03-21T09:13:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 msgr2=0x55acd47aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.576+0000 7f27843a7740 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 0x55acd47aefe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27680bd8e0 tx=0x7f276802bc20 comp rx=0 tx=0).stop 2024-03-21T09:13:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 shutdown_connections 2024-03-21T09:13:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55acd47b6a90 0x55acd4815120 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55acd47a6de0 0x55acd47acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55acd479e260 0x55acd47aefe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 --2- 172.21.15.37:0/2736368585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acd479d8b0 0x55acd47aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 >> 172.21.15.37:0/2736368585 conn(0x55acd4797f80 msgr2=0x55acd4797160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f275be00640 0 client.15852 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:13:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 shutdown_connections 2024-03-21T09:13:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 1 -- 172.21.15.37:0/2736368585 wait complete. 2024-03-21T09:13:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.577+0000 7f27843a7740 20 client.15852 trim_cache size 0 max 16384 2024-03-21T09:13:38.579 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:13:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27600a32b0 msgr2=0x7f27600a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27600a32b0 0x7f27600a5770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27740099a0 tx=0x7f2774004a40 comp rx=0 tx=0).stop 2024-03-21T09:13:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 msgr2=0x7f277c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 0x7f277c19a430 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2768005ab0 tx=0x7f27680042e0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 shutdown_connections 2024-03-21T09:13:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27600a32b0 0x7f27600a5770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f277c107560 0x7f277c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f277c101e60 0x7f277c19a430 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 --2- 172.21.15.37:0/3647804051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277c1014a0 0x7f277c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 >> 172.21.15.37:0/3647804051 conn(0x7f277c0fcfb0 msgr2=0x7f277c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 shutdown_connections 2024-03-21T09:13:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.578+0000 7f277be00640 1 -- 172.21.15.37:0/3647804051 wait complete. 2024-03-21T09:13:38.596 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:13:38.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:13:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:38 smithi071 ceph-mon[36597]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:13:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3065328708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:13:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 -- 172.21.15.37:0/4157315482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d41014a0 msgr2=0x7f57d4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 --2- 172.21.15.37:0/4157315482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d41014a0 0x7f57d4101920 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009930 tx=0x7f57c002bc20 comp rx=0 tx=0).stop 2024-03-21T09:13:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 -- 172.21.15.37:0/4157315482 shutdown_connections 2024-03-21T09:13:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 --2- 172.21.15.37:0/4157315482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 0x7f57d410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 --2- 172.21.15.37:0/4157315482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d41014a0 0x7f57d4101920 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 --2- 172.21.15.37:0/4157315482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d4107560 0x7f57d4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 -- 172.21.15.37:0/4157315482 >> 172.21.15.37:0/4157315482 conn(0x7f57d40fcfb0 msgr2=0x7f57d40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 -- 172.21.15.37:0/4157315482 shutdown_connections 2024-03-21T09:13:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.804+0000 7f57da200640 1 -- 172.21.15.37:0/4157315482 wait complete. 2024-03-21T09:13:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.805+0000 7f57da200640 1 Processor -- start 2024-03-21T09:13:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.805+0000 7f57da200640 1 -- start start 2024-03-21T09:13:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57da200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41014a0 0x7f57d40740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57da200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 0x7f57d406c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57da200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 0x7f57d406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57da200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d406baf0 con 0x7f57d41014a0 2024-03-21T09:13:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57da200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d406b970 con 0x7f57d4107560 2024-03-21T09:13:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57da200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d406bc70 con 0x7f57d4101e60 2024-03-21T09:13:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57d9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 0x7f57d406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57d9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 0x7f57d406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44954/0 (socket says 172.21.15.37:44954) 2024-03-21T09:13:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57d9800640 1 -- 172.21.15.37:0/2048211076 learned_addr learned my addr 172.21.15.37:0/2048211076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57d8e00640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41014a0 0x7f57d40740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 0x7f57d406c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.806+0000 7f57d9800640 1 -- 172.21.15.37:0/2048211076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 msgr2=0x7f57d406c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d9800640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 0x7f57d406c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d9800640 1 -- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41014a0 msgr2=0x7f57d40740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d9800640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41014a0 0x7f57d40740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d9800640 1 -- 172.21.15.37:0/2048211076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d406cd30 con 0x7f57d4107560 2024-03-21T09:13:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 0x7f57d406c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d9800640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 0x7f57d406c7f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00eb10 tx=0x7f57cc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d1600640 1 -- 172.21.15.37:0/2048211076 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc004b20 con 0x7f57d4107560 2024-03-21T09:13:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d1600640 1 -- 172.21.15.37:0/2048211076 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57cc004cc0 con 0x7f57d4107560 2024-03-21T09:13:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57d1600640 1 -- 172.21.15.37:0/2048211076 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc010760 con 0x7f57d4107560 2024-03-21T09:13:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d406d020 con 0x7f57d4107560 2024-03-21T09:13:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.807+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d41bbe30 con 0x7f57d4107560 2024-03-21T09:13:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.812+0000 7f57de589740 1 Processor -- start 2024-03-21T09:13:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.812+0000 7f57d1600640 1 -- 172.21.15.37:0/2048211076 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f57cc010900 con 0x7f57d4107560 2024-03-21T09:13:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 -- start start 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd298c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd2a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fd1fc2c0 con 0x55b6fd2988b0 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fd1a4c90 con 0x55b6fd299260 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.813+0000 7f57de589740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fd1f8360 con 0x55b6fd2a1de0 2024-03-21T09:13:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.814+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.814+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45594/0 (socket says 172.21.15.37:45594) 2024-03-21T09:13:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.814+0000 7f57d8e00640 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd2a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.814+0000 7f57d3e00640 1 -- 172.21.15.37:0/4000530908 learned_addr learned my addr 172.21.15.37:0/4000530908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.814+0000 7f57d3e00640 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd298c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d1600640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f57bc0a31b0 0x7f57bc0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d8e00640 1 -- 172.21.15.37:0/4000530908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 msgr2=0x55b6fd2a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d1600640 1 -- 172.21.15.37:0/2048211076 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f57cc0d05d0 con 0x7f57d4107560 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d8e00640 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d8e00640 1 -- 172.21.15.37:0/4000530908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 msgr2=0x55b6fd298c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d8e00640 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd298c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d8e00640 1 -- 172.21.15.37:0/4000530908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fd1c9080 con 0x55b6fd299260 2024-03-21T09:13:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d9800640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f57bc0a31b0 0x7f57bc0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.815+0000 7f57d8e00640 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd2a1810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f57c800cad0 tx=0x7f57c800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee6ed529a348188 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.816+0000 7f57c6000640 1 -- 172.21.15.37:0/4000530908 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8014070 con 0x55b6fd299260 2024-03-21T09:13:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.816+0000 7f57c6000640 1 -- 172.21.15.37:0/4000530908 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c8005020 con 0x55b6fd299260 2024-03-21T09:13:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.816+0000 7f57c6000640 1 -- 172.21.15.37:0/4000530908 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8004bc0 con 0x55b6fd299260 2024-03-21T09:13:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.816+0000 7f57d9800640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f57bc0a31b0 0x7f57bc0a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0cd860 tx=0x7f57cc098040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 -- 172.21.15.37:0/4000530908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 msgr2=0x55b6fd2a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd2a1810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f57c800cad0 tx=0x7f57c800cf90 comp rx=0 tx=0).stop 2024-03-21T09:13:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 -- 172.21.15.37:0/4000530908 shutdown_connections 2024-03-21T09:13:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd2a1810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 --2- 172.21.15.37:0/4000530908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd298c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 -- 172.21.15.37:0/4000530908 >> 172.21.15.37:0/4000530908 conn(0x55b6fd292f80 msgr2=0x55b6fd293360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 -- 172.21.15.37:0/4000530908 shutdown_connections 2024-03-21T09:13:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.817+0000 7f57de589740 1 -- 172.21.15.37:0/4000530908 wait complete. 2024-03-21T09:13:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.818+0000 7f57de589740 1 -- start start 2024-03-21T09:13:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.818+0000 7f57de589740 1 Processor -- start 2024-03-21T09:13:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd29ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2a1de0 0x55b6fd29efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fd2a4d50 con 0x55b6fd2a1de0 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fd1b80d0 con 0x55b6fd2988b0 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fd2a4ed0 con 0x55b6fd299260 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57de589740 10 client.0 _ll_register_callbacks cb 0x55b6fd1bce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44968/0 (socket says 172.21.15.37:44968) 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57d3e00640 1 -- 172.21.15.37:0/2961101403 learned_addr learned my addr 172.21.15.37:0/2961101403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2a1de0 0x55b6fd29efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.819+0000 7f57d8e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd29ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57d3e00640 1 -- 172.21.15.37:0/2961101403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 msgr2=0x55b6fd29ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd29ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57d3e00640 1 -- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2a1de0 msgr2=0x55b6fd29efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2a1de0 0x55b6fd29efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57d3e00640 1 -- 172.21.15.37:0/2961101403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6fcf103f0 con 0x55b6fd2988b0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a16d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f57c002c810 tx=0x7f57c002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2095be3a0bcada9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57baa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57baa00640 1 -- 172.21.15.37:0/2961101403 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c002ca80 con 0x55b6fd2988b0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c8005020 con 0x55b6fd2988b0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.820+0000 7f57baa00640 1 -- 172.21.15.37:0/2961101403 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f57c002cc20 con 0x55b6fd2988b0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57baa00640 1 -- 172.21.15.37:0/2961101403 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57c0054050 con 0x55b6fd2988b0 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 10 client.26024 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:38.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b6fcfda7d0 con 0x55b6fd2988b0 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57baa00640 1 -- 172.21.15.37:0/2961101403 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c8005020 con 0x55b6fd2988b0 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57baa00640 1 -- 172.21.15.37:0/2961101403 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f57c0043870 con 0x55b6fd2988b0 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 10 client.26024 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 20 client.26024 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd2b1be0 0x55b6fd2b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 4 client.26024 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.821+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6fd209800 con 0x55b6fd2b1be0 2024-03-21T09:13:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.822+0000 7f57d8e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd2b1be0 0x55b6fd2b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.824+0000 7f57d8e00640 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd2b1be0 0x55b6fd2b3fd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1c030e01a9ba8a server_cookie=bcf3f7d4765a2fef in_seq=0 out_seq=0 2024-03-21T09:13:38.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.824+0000 7f57baa00640 10 client.26024 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57baa00640 1 -- 172.21.15.37:0/2961101403 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b6fd209800 con 0x55b6fd2b1be0 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57baa00640 10 client.26024 handle_command_reply: tid=0 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 client.26024 shutdown 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 msgr2=0x55b6fd2a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a16d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f57c002c810 tx=0x7f57c002e040 comp rx=0 tx=0).stop 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 shutdown_connections 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2a1de0 0x55b6fd29efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd29ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd2b1be0 0x55b6fd2b3fd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57baa00640 0 client.26024 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 --2- 172.21.15.37:0/2961101403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a16d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.825+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 >> 172.21.15.37:0/2961101403 conn(0x55b6fd292f80 msgr2=0x55b6fd294cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.826+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 shutdown_connections 2024-03-21T09:13:38.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.826+0000 7f57de589740 1 -- 172.21.15.37:0/2961101403 wait complete. 2024-03-21T09:13:38.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.826+0000 7f57de589740 20 client.26024 trim_cache size 0 max 16384 2024-03-21T09:13:38.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.839+0000 7f57de589740 1 Processor -- start 2024-03-21T09:13:38.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.839+0000 7f57de589740 1 -- start start 2024-03-21T09:13:38.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57de589740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd294c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57de589740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd291fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57de589740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57de589740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d411d2f0 con 0x55b6fd2988b0 2024-03-21T09:13:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57de589740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d411adf0 con 0x55b6fd299260 2024-03-21T09:13:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57de589740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4118ec0 con 0x55b6fd2a1de0 2024-03-21T09:13:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57d8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd291fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57d8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd291fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44984/0 (socket says 172.21.15.37:44984) 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57d8e00640 1 -- 172.21.15.37:0/2335400116 learned_addr learned my addr 172.21.15.37:0/2335400116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.840+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd294c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.841+0000 7f57d3e00640 1 -- 172.21.15.37:0/2335400116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 msgr2=0x55b6fd291fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.841+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd291fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.841+0000 7f57d3e00640 1 -- 172.21.15.37:0/2335400116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 msgr2=0x55b6fd294c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.841+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd294c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.841+0000 7f57d3e00640 1 -- 172.21.15.37:0/2335400116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fd29f350 con 0x55b6fd2a1de0 2024-03-21T09:13:38.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.841+0000 7f57d3e00640 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a1190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f57c0052040 tx=0x7f57c0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1023316f2c78694a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.844+0000 7f57c5600640 1 -- 172.21.15.37:0/2335400116 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c002cc80 con 0x55b6fd2a1de0 2024-03-21T09:13:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.844+0000 7f57c5600640 1 -- 172.21.15.37:0/2335400116 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c0047070 con 0x55b6fd2a1de0 2024-03-21T09:13:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.844+0000 7f57c5600640 1 -- 172.21.15.37:0/2335400116 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00046c0 con 0x55b6fd2a1de0 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 -- 172.21.15.37:0/2335400116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 msgr2=0x55b6fd2a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a1190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f57c0052040 tx=0x7f57c0000bc0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 -- 172.21.15.37:0/2335400116 shutdown_connections 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a1190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd299260 0x55b6fd291fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 --2- 172.21.15.37:0/2335400116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd294c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 -- 172.21.15.37:0/2335400116 >> 172.21.15.37:0/2335400116 conn(0x55b6fd292f80 msgr2=0x55b6fd2918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 -- 172.21.15.37:0/2335400116 shutdown_connections 2024-03-21T09:13:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 -- 172.21.15.37:0/2335400116 wait complete. 2024-03-21T09:13:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.845+0000 7f57de589740 1 -- start start 2024-03-21T09:13:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 Processor -- start 2024-03-21T09:13:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd2a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4119490 con 0x55b6fd2988b0 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d411a790 con 0x55b6fd2a1de0 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4119ad0 con 0x55b6fd299260 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.846+0000 7f57de589740 10 client.0 _ll_register_callbacks cb 0x7f57c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49692/0 (socket says 172.21.15.37:49692) 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 -- 172.21.15.37:0/1654460372 learned_addr learned my addr 172.21.15.37:0/1654460372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 -- 172.21.15.37:0/1654460372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 msgr2=0x55b6fd2a7fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:13:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd2a7fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 -- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 msgr2=0x55b6fd2a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 -- 172.21.15.37:0/1654460372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6fd2a8ac0 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57d3e00640 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a8500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f57c0004650 tx=0x7f57c0005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa340cf7d5cb085 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.847+0000 7f57baa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57baa00640 1 -- 172.21.15.37:0/1654460372 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c002cc80 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c0047070 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57baa00640 1 -- 172.21.15.37:0/1654460372 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f57c0005040 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57baa00640 1 -- 172.21.15.37:0/1654460372 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57c0066050 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57de589740 10 client.26030 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b6fd30e200 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57baa00640 1 -- 172.21.15.37:0/1654460372 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c0047070 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57baa00640 1 -- 172.21.15.37:0/1654460372 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f57c0004ad0 con 0x55b6fd2a1de0 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57de589740 10 client.26030 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.848+0000 7f57de589740 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.849+0000 7f57de589740 20 client.26030 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.849+0000 7f57de589740 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd30e820 0x55b6fd310c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.849+0000 7f57de589740 4 client.26030 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:13:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.849+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6fd203b50 con 0x55b6fd30e820 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.849+0000 7f57d8e00640 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd30e820 0x55b6fd310c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.850+0000 7f57d8e00640 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd30e820 0x55b6fd310c40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33571aea69f47f3f server_cookie=712b1f62c78877f2 in_seq=0 out_seq=0 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.850+0000 7f57baa00640 10 client.26030 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57baa00640 1 -- 172.21.15.37:0/1654460372 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6fd203b50 con 0x55b6fd30e820 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57baa00640 10 client.26030 handle_command_reply: tid=0 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 client.26030 shutdown 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2a1de0 msgr2=0x55b6fd2a8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a8500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f57c0004650 tx=0x7f57c0005b50 comp rx=0 tx=0).stop 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 shutdown_connections 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b6fd30e820 0x55b6fd310c40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6fd2a1de0 0x55b6fd2a8500 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6fd299260 0x55b6fd2a7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 --2- 172.21.15.37:0/1654460372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6fd2988b0 0x55b6fd2a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 >> 172.21.15.37:0/1654460372 conn(0x55b6fd292f80 msgr2=0x55b6fd2918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57baa00640 0 client.26030 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:38.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.851+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 shutdown_connections 2024-03-21T09:13:38.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.852+0000 7f57de589740 1 -- 172.21.15.37:0/1654460372 wait complete. 2024-03-21T09:13:38.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.852+0000 7f57de589740 20 client.26030 trim_cache size 0 max 16384 2024-03-21T09:13:38.848 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:13:38.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.852+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f57bc0a31b0 msgr2=0x7f57bc0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.852+0000 7f57da200640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f57bc0a31b0 0x7f57bc0a5670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0cd860 tx=0x7f57cc098040 comp rx=0 tx=0).stop 2024-03-21T09:13:38.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.852+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 msgr2=0x7f57d406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.852+0000 7f57da200640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 0x7f57d406c7f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00eb10 tx=0x7f57cc00c2d0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 shutdown_connections 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f57bc0a31b0 0x7f57bc0a5670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57d4107560 0x7f57d406c7f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101e60 0x7f57d406c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 --2- 172.21.15.37:0/2048211076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41014a0 0x7f57d40740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 >> 172.21.15.37:0/2048211076 conn(0x7f57d40fcfb0 msgr2=0x7f57d410d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 shutdown_connections 2024-03-21T09:13:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:38.853+0000 7f57da200640 1 -- 172.21.15.37:0/2048211076 wait complete. 2024-03-21T09:13:39.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:13:39.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:13:39.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:13:40.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 -- 172.21.15.37:0/1566886309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4564105690 msgr2=0x7f456410d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 --2- 172.21.15.37:0/1566886309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4564105690 0x7f456410d260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f455c009900 tx=0x7f455c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:13:40.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 -- 172.21.15.37:0/1566886309 shutdown_connections 2024-03-21T09:13:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 --2- 172.21.15.37:0/1566886309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4564105690 0x7f456410d260 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 --2- 172.21.15.37:0/1566886309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100fc0 0x7f4564104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 --2- 172.21.15.37:0/1566886309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564100610 0x7f45641009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.106+0000 7f456a000640 1 -- 172.21.15.37:0/1566886309 >> 172.21.15.37:0/1566886309 conn(0x7f4564077130 msgr2=0x7f45640fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.107+0000 7f456a000640 1 -- 172.21.15.37:0/1566886309 shutdown_connections 2024-03-21T09:13:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.107+0000 7f456a000640 1 -- 172.21.15.37:0/1566886309 wait complete. 2024-03-21T09:13:40.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.108+0000 7f456a000640 1 Processor -- start 2024-03-21T09:13:40.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.108+0000 7f456a000640 1 -- start start 2024-03-21T09:13:40.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f456a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 0x7f4564197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f456a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564105690 0x7f45641982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f456a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f456419c5e0 0x7f456419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f456a000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564111890 con 0x7f4564105690 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f456a000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564111710 con 0x7f4564100610 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f456a000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564111a10 con 0x7f456419c5e0 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f4568c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 0x7f4564197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f4568c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 0x7f4564197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45018/0 (socket says 172.21.15.37:45018) 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f4568c00640 1 -- 172.21.15.37:0/189226435 learned_addr learned my addr 172.21.15.37:0/189226435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f4563e00640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564105690 0x7f45641982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.109+0000 7f4569600640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f456419c5e0 0x7f456419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4568c00640 1 -- 172.21.15.37:0/189226435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f456419c5e0 msgr2=0x7f456419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4568c00640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f456419c5e0 0x7f456419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4568c00640 1 -- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564105690 msgr2=0x7f45641982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4568c00640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564105690 0x7f45641982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4568c00640 1 -- 172.21.15.37:0/189226435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456419d0d0 con 0x7f4564100610 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4569600640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f456419c5e0 0x7f456419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4563e00640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564105690 0x7f45641982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4568c00640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 0x7f4564197d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f455400eab0 tx=0x7f455400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4561600640 1 -- 172.21.15.37:0/189226435 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400ce40 con 0x7f4564100610 2024-03-21T09:13:40.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4561600640 1 -- 172.21.15.37:0/189226435 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4554011950 con 0x7f4564100610 2024-03-21T09:13:40.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f4561600640 1 -- 172.21.15.37:0/189226435 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554010ae0 con 0x7f4564100610 2024-03-21T09:13:40.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.110+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456419d3c0 con 0x7f4564100610 2024-03-21T09:13:40.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.111+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456410ddf0 con 0x7f4564100610 2024-03-21T09:13:40.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.115+0000 7f456e3be740 1 Processor -- start 2024-03-21T09:13:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.115+0000 7f456e3be740 1 -- start start 2024-03-21T09:13:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f456e3be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec078670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f456e3be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec081250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f456e3be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec083b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f456e3be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558debfdceb0 con 0x558dec078290 2024-03-21T09:13:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f456e3be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558debf85a60 con 0x558dec081790 2024-03-21T09:13:40.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f456e3be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558debfd8f50 con 0x558dec078bb0 2024-03-21T09:13:40.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f4561600640 1 -- 172.21.15.37:0/189226435 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f4554011b00 con 0x7f4564100610 2024-03-21T09:13:40.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f4563e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec083b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f4563e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec083b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45028/0 (socket says 172.21.15.37:45028) 2024-03-21T09:13:40.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f4563e00640 1 -- 172.21.15.37:0/3853951772 learned_addr learned my addr 172.21.15.37:0/3853951772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.116+0000 7f4569600640 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec078670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4563e00640 1 -- 172.21.15.37:0/3853951772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 msgr2=0x558dec081250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4563e00640 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec081250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4563e00640 1 -- 172.21.15.37:0/3853951772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 msgr2=0x558dec078670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4563e00640 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec078670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4563e00640 1 -- 172.21.15.37:0/3853951772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558debdb97e0 con 0x558dec081790 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4569600640 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4563e00640 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec083b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f454c00af20 tx=0x7f454c00f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3716d3c29b2b0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4559c00640 1 -- 172.21.15.37:0/3853951772 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c01d070 con 0x558dec081790 2024-03-21T09:13:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4559c00640 1 -- 172.21.15.37:0/3853951772 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f454c010de0 con 0x558dec081790 2024-03-21T09:13:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.117+0000 7f4559c00640 1 -- 172.21.15.37:0/3853951772 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c018e40 con 0x558dec081790 2024-03-21T09:13:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.118+0000 7f456e3be740 1 -- 172.21.15.37:0/3853951772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 msgr2=0x558dec083b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.118+0000 7f456e3be740 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec083b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f454c00af20 tx=0x7f454c00f850 comp rx=0 tx=0).stop 2024-03-21T09:13:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 -- 172.21.15.37:0/3853951772 shutdown_connections 2024-03-21T09:13:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec083b80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec081250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 --2- 172.21.15.37:0/3853951772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 -- 172.21.15.37:0/3853951772 >> 172.21.15.37:0/3853951772 conn(0x558dec0729c0 msgr2=0x558dec072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 -- 172.21.15.37:0/3853951772 shutdown_connections 2024-03-21T09:13:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f456e3be740 1 -- 172.21.15.37:0/3853951772 wait complete. 2024-03-21T09:13:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.119+0000 7f4561600640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f45480a3140 0x7f45480a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.120+0000 7f456e3be740 1 -- start start 2024-03-21T09:13:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.120+0000 7f4561600640 1 -- 172.21.15.37:0/189226435 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f45540d0400 con 0x7f4564100610 2024-03-21T09:13:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.120+0000 7f456e3be740 1 Processor -- start 2024-03-21T09:13:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.120+0000 7f4569600640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f45480a3140 0x7f45480a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.121+0000 7f4569600640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f45480a3140 0x7f45480a5600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f455c006010 tx=0x7f455c02df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 0x558dec081100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078bb0 0x558dec07e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dec084580 con 0x558dec078bb0 2024-03-21T09:13:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558debf98eb0 con 0x558dec081790 2024-03-21T09:13:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dec084700 con 0x558dec078290 2024-03-21T09:13:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f456e3be740 10 client.0 _ll_register_callbacks cb 0x558debf8d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f4563e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078bb0 0x558dec07e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f4569600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f4568c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 0x558dec081100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f4569600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45030/0 (socket says 172.21.15.37:45030) 2024-03-21T09:13:40.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.123+0000 7f4569600640 1 -- 172.21.15.37:0/775964566 learned_addr learned my addr 172.21.15.37:0/775964566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f4568c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 0x558dec081100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45660/0 (socket says 172.21.15.37:45660) 2024-03-21T09:13:40.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.122+0000 7f4563e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078bb0 0x558dec07e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49724/0 (socket says 172.21.15.37:49724) 2024-03-21T09:13:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4568c00640 1 -- 172.21.15.37:0/775964566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 msgr2=0x558dec07e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4568c00640 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4568c00640 1 -- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078bb0 msgr2=0x558dec07e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4568c00640 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078bb0 0x558dec07e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4568c00640 1 -- 172.21.15.37:0/775964566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558debe19f70 con 0x558dec078290 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4569600640 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4568c00640 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 0x558dec081100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45540cd630 tx=0x7f455409bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6af660e0fef72a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4552000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.124+0000 7f4552000640 1 -- 172.21.15.37:0/775964566 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455409c660 con 0x558dec078290 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f4552000640 1 -- 172.21.15.37:0/775964566 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f45540a4780 con 0x558dec078290 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f454c010de0 con 0x558dec078290 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f4552000640 1 -- 172.21.15.37:0/775964566 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45540a4a00 con 0x558dec078290 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f456e3be740 10 client.26062 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558debcb1170 con 0x558dec078290 2024-03-21T09:13:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f4552000640 1 -- 172.21.15.37:0/775964566 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f454c010de0 con 0x558dec078290 2024-03-21T09:13:40.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.125+0000 7f4552000640 1 -- 172.21.15.37:0/775964566 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f455400f040 con 0x558dec078290 2024-03-21T09:13:40.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f456e3be740 10 client.26062 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:40.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f456e3be740 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:40.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f456e3be740 20 client.26062 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:40.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f456e3be740 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec099f40 0x558dec09c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f456e3be740 4 client.26062 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558debfea3f0 con 0x558dec099f40 2024-03-21T09:13:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.126+0000 7f4563e00640 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec099f40 0x558dec09c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.127+0000 7f4563e00640 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec099f40 0x558dec09c330 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6178d76403281b22 server_cookie=6b1b3a1364987a3f in_seq=0 out_seq=0 2024-03-21T09:13:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.127+0000 7f4552000640 10 client.26062 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.128+0000 7f4552000640 1 -- 172.21.15.37:0/775964566 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558debfea3f0 con 0x558dec099f40 2024-03-21T09:13:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f4552000640 10 client.26062 handle_command_reply: tid=0 2024-03-21T09:13:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 client.26062 shutdown 2024-03-21T09:13:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 msgr2=0x558dec081100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 0x558dec081100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45540cd630 tx=0x7f455409bd40 comp rx=0 tx=0).stop 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 shutdown_connections 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec099f40 0x558dec09c330 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078bb0 0x558dec07e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 --2- 172.21.15.37:0/775964566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078290 0x558dec081100 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 >> 172.21.15.37:0/775964566 conn(0x558dec0729c0 msgr2=0x558dec074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.129+0000 7f4552000640 0 client.26062 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.130+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 shutdown_connections 2024-03-21T09:13:40.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.130+0000 7f456e3be740 1 -- 172.21.15.37:0/775964566 wait complete. 2024-03-21T09:13:40.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.130+0000 7f456e3be740 20 client.26062 trim_cache size 0 max 16384 2024-03-21T09:13:40.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.164+0000 7f456e3be740 1 Processor -- start 2024-03-21T09:13:40.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.164+0000 7f456e3be740 1 -- start start 2024-03-21T09:13:40.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f456e3be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 0x558dec074500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f456e3be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec080bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f456e3be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec081790 0x558dec081100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f456e3be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641aeaf0 con 0x558dec081790 2024-03-21T09:13:40.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f456e3be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641ac5f0 con 0x558dec078290 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f456e3be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641aa690 con 0x558dec078bb0 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f4568c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 0x558dec074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f4568c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 0x558dec074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45032/0 (socket says 172.21.15.37:45032) 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.165+0000 7f4568c00640 1 -- 172.21.15.37:0/844279949 learned_addr learned my addr 172.21.15.37:0/844279949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4563e00640 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec080bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4569600640 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec081790 0x558dec081100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4568c00640 1 -- 172.21.15.37:0/844279949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 msgr2=0x558dec080bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4568c00640 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec080bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4568c00640 1 -- 172.21.15.37:0/844279949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec081790 msgr2=0x558dec081100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4568c00640 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec081790 0x558dec081100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4568c00640 1 -- 172.21.15.37:0/844279949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dec06cc40 con 0x558dec078290 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.166+0000 7f4568c00640 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 0x558dec074500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f455400c700 tx=0x7f45540d0770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5f38d8599fa1f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.167+0000 7f4559200640 1 -- 172.21.15.37:0/844279949 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540a0120 con 0x558dec078290 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.167+0000 7f4559200640 1 -- 172.21.15.37:0/844279949 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45540a4c10 con 0x558dec078290 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.167+0000 7f4559200640 1 -- 172.21.15.37:0/844279949 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540a49e0 con 0x558dec078290 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.167+0000 7f456e3be740 1 -- 172.21.15.37:0/844279949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 msgr2=0x558dec074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.167+0000 7f456e3be740 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 0x558dec074500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f455400c700 tx=0x7f45540d0770 comp rx=0 tx=0).stop 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 -- 172.21.15.37:0/844279949 shutdown_connections 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec081790 0x558dec081100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec080bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 --2- 172.21.15.37:0/844279949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec078290 0x558dec074500 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 -- 172.21.15.37:0/844279949 >> 172.21.15.37:0/844279949 conn(0x558dec0729c0 msgr2=0x558dec073df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 -- 172.21.15.37:0/844279949 shutdown_connections 2024-03-21T09:13:40.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.168+0000 7f456e3be740 1 -- 172.21.15.37:0/844279949 wait complete. 2024-03-21T09:13:40.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 -- start start 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 Processor -- start 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec07ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641aac90 con 0x558dec078290 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641abf90 con 0x558dec081790 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641ab2d0 con 0x558dec078bb0 2024-03-21T09:13:40.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.169+0000 7f456e3be740 10 client.0 _ll_register_callbacks cb 0x7f45540984a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4569600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4569600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45042/0 (socket says 172.21.15.37:45042) 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4569600640 1 -- 172.21.15.37:0/85273498 learned_addr learned my addr 172.21.15.37:0/85273498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4563e00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4568c00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec07ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4563e00640 1 -- 172.21.15.37:0/85273498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 msgr2=0x558dec07b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4563e00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4563e00640 1 -- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 msgr2=0x558dec07ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4563e00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec07ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4563e00640 1 -- 172.21.15.37:0/85273498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dec0f4750 con 0x558dec078bb0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.170+0000 7f4568c00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec07ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f4563e00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec07b1c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f454c00aef0 tx=0x7f454c00aab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ae11707f3efe9b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f4552000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:40.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f4552000640 1 -- 172.21.15.37:0/85273498 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c01d040 con 0x558dec078bb0 2024-03-21T09:13:40.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f4552000640 1 -- 172.21.15.37:0/85273498 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f454c018e00 con 0x558dec078bb0 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45540a4c10 con 0x558dec078bb0 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f4552000640 1 -- 172.21.15.37:0/85273498 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454c03b050 con 0x558dec078bb0 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f456e3be740 10 client.26068 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.171+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558dec0f5990 con 0x558dec078bb0 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f4552000640 1 -- 172.21.15.37:0/85273498 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45540a4c10 con 0x558dec078bb0 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f4552000640 1 -- 172.21.15.37:0/85273498 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f454c005040 con 0x558dec078bb0 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f456e3be740 10 client.26068 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f456e3be740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f456e3be740 20 client.26068 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f456e3be740 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec0f5e10 0x558dec0f8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f456e3be740 4 client.26068 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558debfe4740 con 0x558dec0f5e10 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.172+0000 7f4568c00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec0f5e10 0x558dec0f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.173+0000 7f4552000640 10 client.26068 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.173+0000 7f4568c00640 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec0f5e10 0x558dec0f8230 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed3f1b97df67f74 server_cookie=499f49ffcaea7478 in_seq=0 out_seq=0 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f4552000640 1 -- 172.21.15.37:0/85273498 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558debfe4740 con 0x558dec0f5e10 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f4552000640 10 client.26068 handle_command_reply: tid=0 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 client.26068 shutdown 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 msgr2=0x558dec07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec07b1c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f454c00aef0 tx=0x7f454c00aab0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 shutdown_connections 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558dec0f5e10 0x558dec0f8230 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dec081790 0x558dec07b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558dec078bb0 0x558dec07b1c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 --2- 172.21.15.37:0/85273498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558dec078290 0x558dec07ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 >> 172.21.15.37:0/85273498 conn(0x558dec0729c0 msgr2=0x558dec074680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.175+0000 7f4552000640 0 client.26068 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.176+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 shutdown_connections 2024-03-21T09:13:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.176+0000 7f456e3be740 1 -- 172.21.15.37:0/85273498 wait complete. 2024-03-21T09:13:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.176+0000 7f456e3be740 20 client.26068 trim_cache size 0 max 16384 2024-03-21T09:13:40.172 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:13:40.172 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:13:40.172 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "5e07ddc0-4ae5-4e06-809e-0a17b9c94e48", 2024-03-21T09:13:40.172 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:13:40.172 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:13:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f45480a3140 msgr2=0x7f45480a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f45480a3140 0x7f45480a5600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f455c006010 tx=0x7f455c02df90 comp rx=0 tx=0).stop 2024-03-21T09:13:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 msgr2=0x7f4564197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 0x7f4564197d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f455400eab0 tx=0x7f455400ef70 comp rx=0 tx=0).stop 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 shutdown_connections 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f456419c5e0 0x7f456419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4564105690 0x7f45641982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f45480a3140 0x7f45480a5600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 --2- 172.21.15.37:0/189226435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4564100610 0x7f4564197d70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 >> 172.21.15.37:0/189226435 conn(0x7f4564077130 msgr2=0x7f4564103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 shutdown_connections 2024-03-21T09:13:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.179+0000 7f456a000640 1 -- 172.21.15.37:0/189226435 wait complete. 2024-03-21T09:13:40.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:13:40.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:13:40.262 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:40 smithi037 ceph-mon[31830]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.401+0000 7fb82e600640 1 -- 172.21.15.37:0/3118631340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 msgr2=0x7fb828109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.401+0000 7fb82e600640 1 --2- 172.21.15.37:0/3118631340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 0x7fb828109a70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009930 tx=0x7fb81c02bc20 comp rx=0 tx=0).stop 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 -- 172.21.15.37:0/3118631340 shutdown_connections 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 --2- 172.21.15.37:0/3118631340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 0x7fb828110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 --2- 172.21.15.37:0/3118631340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 --2- 172.21.15.37:0/3118631340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 0x7fb828109a70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 -- 172.21.15.37:0/3118631340 >> 172.21.15.37:0/3118631340 conn(0x7fb8280ff190 msgr2=0x7fb8281015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 -- 172.21.15.37:0/3118631340 shutdown_connections 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.402+0000 7fb82e600640 1 -- 172.21.15.37:0/3118631340 wait complete. 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.404+0000 7fb82e600640 1 Processor -- start 2024-03-21T09:13:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.404+0000 7fb82e600640 1 -- start start 2024-03-21T09:13:40.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.404+0000 7fb82e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.404+0000 7fb82e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 0x7fb82819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.404+0000 7fb82e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 0x7fb82819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828113ab0 con 0x7fb828103680 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82e600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828113930 con 0x7fb828104040 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828113c30 con 0x7fb828109690 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49756/0 (socket says 172.21.15.37:49756) 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82d200640 1 -- 172.21.15.37:0/1976779565 learned_addr learned my addr 172.21.15.37:0/1976779565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb827e00640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 0x7fb82819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82dc00640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 0x7fb82819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb827e00640 1 -- 172.21.15.37:0/1976779565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 msgr2=0x7fb82819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb827e00640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 0x7fb82819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb827e00640 1 -- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 msgr2=0x7fb828199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb827e00640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb827e00640 1 -- 172.21.15.37:0/1976779565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82819efa0 con 0x7fb828104040 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.405+0000 7fb82d200640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.406+0000 7fb827e00640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 0x7fb82819a4f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb81400d6d0 tx=0x7fb81400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.406+0000 7fb825600640 1 -- 172.21.15.37:0/1976779565 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814017070 con 0x7fb828104040 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.406+0000 7fb825600640 1 -- 172.21.15.37:0/1976779565 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb814009820 con 0x7fb828104040 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.406+0000 7fb825600640 1 -- 172.21.15.37:0/1976779565 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814004d00 con 0x7fb828104040 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.406+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82819f290 con 0x7fb828104040 2024-03-21T09:13:40.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.406+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8281a6b20 con 0x7fb828104040 2024-03-21T09:13:40.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.411+0000 7fb832a01740 1 Processor -- start 2024-03-21T09:13:40.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.411+0000 7fb832a01740 1 -- start start 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb832a01740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b01c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb832a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 0x55e29b01cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb832a01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b025c70 0x55e29b028060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb832a01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29af78250 con 0x55e29b025c70 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb832a01740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29af20a30 con 0x55e29b01d5e0 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb832a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29af742f0 con 0x55e29b01c900 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb82d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 0x55e29b01cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb82d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 0x55e29b01cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45690/0 (socket says 172.21.15.37:45690) 2024-03-21T09:13:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.412+0000 7fb82d200640 1 -- 172.21.15.37:0/125956540 learned_addr learned my addr 172.21.15.37:0/125956540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82dc00640 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b01c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82d200640 1 -- 172.21.15.37:0/125956540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 msgr2=0x55e29b01c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82d200640 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b01c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82d200640 1 -- 172.21.15.37:0/125956540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b025c70 msgr2=0x55e29b028060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82d200640 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b025c70 0x55e29b028060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82d200640 1 -- 172.21.15.37:0/125956540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29af0b690 con 0x55e29b01c900 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb825600640 1 -- 172.21.15.37:0/1976779565 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fb8140099d0 con 0x7fb828104040 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.413+0000 7fb82d200640 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 0x55e29b01cd80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb81c036040 tx=0x7fb81c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c23e70a419be986 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.414+0000 7fb81a000640 1 -- 172.21.15.37:0/125956540 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c045070 con 0x55e29b01c900 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.414+0000 7fb81a000640 1 -- 172.21.15.37:0/125956540 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb81c004a70 con 0x55e29b01c900 2024-03-21T09:13:40.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.414+0000 7fb81a000640 1 -- 172.21.15.37:0/125956540 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c005140 con 0x55e29b01c900 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 -- 172.21.15.37:0/125956540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 msgr2=0x55e29b01cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 0x55e29b01cd80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb81c036040 tx=0x7fb81c005e80 comp rx=0 tx=0).stop 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 -- 172.21.15.37:0/125956540 shutdown_connections 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b025c70 0x55e29b028060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01c900 0x55e29b01cd80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 --2- 172.21.15.37:0/125956540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b01c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 -- 172.21.15.37:0/125956540 >> 172.21.15.37:0/125956540 conn(0x55e29afdb200 msgr2=0x55e29b017350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.415+0000 7fb832a01740 1 -- 172.21.15.37:0/125956540 shutdown_connections 2024-03-21T09:13:40.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.416+0000 7fb832a01740 1 -- 172.21.15.37:0/125956540 wait complete. 2024-03-21T09:13:40.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.416+0000 7fb825600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8100a3140 0x7fb8100a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.416+0000 7fb832a01740 1 -- start start 2024-03-21T09:13:40.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.416+0000 7fb825600640 1 -- 172.21.15.37:0/1976779565 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb8140cf5b0 con 0x7fb828104040 2024-03-21T09:13:40.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.416+0000 7fb832a01740 1 Processor -- start 2024-03-21T09:13:40.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.417+0000 7fb832a01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b025390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb832a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01d5e0 0x55e29b022720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb82d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b025390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb82d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b025390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49764/0 (socket says 172.21.15.37:49764) 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb82d200640 1 -- 172.21.15.37:0/405495790 learned_addr learned my addr 172.21.15.37:0/405495790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b025c70 0x55e29b022c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29b028a80 con 0x55e29b01c900 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb827e00640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01d5e0 0x55e29b022720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.418+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29af33e80 con 0x55e29b025c70 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb82dc00640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b025c70 0x55e29b022c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29b028c00 con 0x55e29b01d5e0 2024-03-21T09:13:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb82d200640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8100a3140 0x7fb8100a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb827e00640 1 -- 172.21.15.37:0/405495790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b025c70 msgr2=0x55e29b022c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb827e00640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b025c70 0x55e29b022c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb827e00640 1 -- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 msgr2=0x55e29b025390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb827e00640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b025390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb827e00640 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e29ad567d0 con 0x55e29b01d5e0 2024-03-21T09:13:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb82dc00640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b025c70 0x55e29b022c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.419+0000 7fb827e00640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01d5e0 0x55e29b022720 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb8140cc7e0 tx=0x7fb81409bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50141f09f1b46fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.420+0000 7fb80ea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:40.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.420+0000 7fb80ea00640 1 -- 172.21.15.37:0/405495790 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140b4060 con 0x55e29b01d5e0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.420+0000 7fb80ea00640 1 -- 172.21.15.37:0/405495790 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb814097a20 con 0x55e29b01d5e0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.420+0000 7fb832a01740 10 client.0 _ll_register_callbacks cb 0x55e29af46130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.420+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81c004a70 con 0x55e29b01d5e0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb82d200640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8100a3140 0x7fb8100a5600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb81c004e00 tx=0x7fb81c04c040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb80ea00640 1 -- 172.21.15.37:0/405495790 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814020040 con 0x55e29b01d5e0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb832a01740 10 client.26080 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x55e29ac8c3f0 con 0x55e29b01d5e0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb80ea00640 1 -- 172.21.15.37:0/405495790 <== mon.1 v2:172.21.15.71:3300/0 4 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb8140cf1b0 con 0x55e29b01d5e0 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb832a01740 10 client.26080 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:40.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb832a01740 10 client.26080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb832a01740 20 client.26080 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.421+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b037b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.422+0000 7fb832a01740 4 client.26080 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.422+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e29af85790 con 0x55e29b035710 2024-03-21T09:13:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.422+0000 7fb82d200640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b037b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.423+0000 7fb82d200640 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b037b60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=883c5efc4610c021 server_cookie=9c7259e8bd12a45f in_seq=0 out_seq=0 2024-03-21T09:13:40.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.423+0000 7fb80ea00640 10 client.26080 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.424+0000 7fb80ea00640 1 -- 172.21.15.37:0/405495790 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e29af85790 con 0x55e29b035710 2024-03-21T09:13:40.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.424+0000 7fb80ea00640 10 client.26080 handle_command_reply: tid=0 2024-03-21T09:13:40.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.424+0000 7fb832a01740 1 client.26080 shutdown 2024-03-21T09:13:40.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.425+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01d5e0 msgr2=0x55e29b022720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.425+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01d5e0 0x55e29b022720 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb8140cc7e0 tx=0x7fb81409bab0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.425+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 shutdown_connections 2024-03-21T09:13:40.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.425+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b037b60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb80ea00640 0 client.26080 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b025c70 0x55e29b022c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b01d5e0 0x55e29b022720 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 1 --2- 172.21.15.37:0/405495790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b025390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 >> 172.21.15.37:0/405495790 conn(0x55e29afdb200 msgr2=0x55e29b0190a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 shutdown_connections 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 1 -- 172.21.15.37:0/405495790 wait complete. 2024-03-21T09:13:40.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.426+0000 7fb832a01740 20 client.26080 trim_cache size 0 max 16384 2024-03-21T09:13:40.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.460+0000 7fb832a01740 1 Processor -- start 2024-03-21T09:13:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.460+0000 7fb832a01740 1 -- start start 2024-03-21T09:13:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb832a01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b024e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb832a01740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b025390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb832a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b027060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb832a01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281aeb10 con 0x55e29b01c900 2024-03-21T09:13:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb832a01740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281ac610 con 0x55e29b01d5e0 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb832a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281aa6d0 con 0x55e29b025c70 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb827e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b025390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb82d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b024e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb827e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b025390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45078/0 (socket says 172.21.15.37:45078) 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb827e00640 1 -- 172.21.15.37:0/1621415951 learned_addr learned my addr 172.21.15.37:0/1621415951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.461+0000 7fb82dc00640 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b027060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb827e00640 1 -- 172.21.15.37:0/1621415951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 msgr2=0x55e29b027060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb827e00640 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b027060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb827e00640 1 -- 172.21.15.37:0/1621415951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 msgr2=0x55e29b024e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb827e00640 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b024e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb827e00640 1 -- 172.21.15.37:0/1621415951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29ac4c0f0 con 0x55e29b01d5e0 2024-03-21T09:13:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb82dc00640 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b027060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb82d200640 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b024e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb827e00640 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b025390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb814009610 tx=0x7fb8140cf000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df929f90a81e38e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.462+0000 7fb819600640 1 -- 172.21.15.37:0/1621415951 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140b4040 con 0x55e29b01d5e0 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.463+0000 7fb819600640 1 -- 172.21.15.37:0/1621415951 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb81c004a70 con 0x55e29b01d5e0 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.463+0000 7fb819600640 1 -- 172.21.15.37:0/1621415951 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140ccac0 con 0x55e29b01d5e0 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.463+0000 7fb832a01740 1 -- 172.21.15.37:0/1621415951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 msgr2=0x55e29b025390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.463+0000 7fb832a01740 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b025390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb814009610 tx=0x7fb8140cf000 comp rx=0 tx=0).stop 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 -- 172.21.15.37:0/1621415951 shutdown_connections 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b027060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b025390 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 --2- 172.21.15.37:0/1621415951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b024e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 -- 172.21.15.37:0/1621415951 >> 172.21.15.37:0/1621415951 conn(0x55e29afdb200 msgr2=0x55e29b00dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 -- 172.21.15.37:0/1621415951 shutdown_connections 2024-03-21T09:13:40.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.464+0000 7fb832a01740 1 -- 172.21.15.37:0/1621415951 wait complete. 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.465+0000 7fb832a01740 1 -- start start 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.465+0000 7fb832a01740 1 Processor -- start 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b020390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b08e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b08e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281aacd0 con 0x55e29b01c900 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281abfb0 con 0x55e29b01d5e0 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281ab2f0 con 0x55e29b025c70 2024-03-21T09:13:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb832a01740 10 client.0 _ll_register_callbacks cb 0x7fb814004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb827e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b08e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb827e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b08e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45084/0 (socket says 172.21.15.37:45084) 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb827e00640 1 -- 172.21.15.37:0/669834273 learned_addr learned my addr 172.21.15.37:0/669834273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.466+0000 7fb82dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b08e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb82d200640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b020390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb827e00640 1 -- 172.21.15.37:0/669834273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 msgr2=0x55e29b08e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb827e00640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b08e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb827e00640 1 -- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 msgr2=0x55e29b020390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb827e00640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b020390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb827e00640 1 -- 172.21.15.37:0/669834273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e29b08edd0 con 0x55e29b01d5e0 2024-03-21T09:13:40.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb82d200640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b020390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb827e00640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b08e020 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb814097d80 tx=0x7fb814097a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=595d383888b88c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb80ea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.467+0000 7fb80ea00640 1 -- 172.21.15.37:0/669834273 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140b4040 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81c004a70 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb80ea00640 1 -- 172.21.15.37:0/669834273 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb8140cccc0 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb80ea00640 1 -- 172.21.15.37:0/669834273 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8140a3dc0 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 10 client.26063 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e29b08edd0 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb80ea00640 1 -- 172.21.15.37:0/669834273 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb81c004a70 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb80ea00640 1 -- 172.21.15.37:0/669834273 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb81409a270 con 0x55e29b01d5e0 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 10 client.26063 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 20 client.26063 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b02d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 4 client.26063 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:13:40.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.468+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e29af7fae0 con 0x55e29b035710 2024-03-21T09:13:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.469+0000 7fb82d200640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b02d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.470+0000 7fb82d200640 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b02d230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e14b0d148b1297 server_cookie=293383aac544fc58 in_seq=0 out_seq=0 2024-03-21T09:13:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.470+0000 7fb80ea00640 10 client.26063 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.470+0000 7fb80ea00640 1 -- 172.21.15.37:0/669834273 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e29af7fae0 con 0x55e29b035710 2024-03-21T09:13:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.470+0000 7fb80ea00640 10 client.26063 handle_command_reply: tid=0 2024-03-21T09:13:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 client.26063 shutdown 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 msgr2=0x55e29b08e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b08e020 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb814097d80 tx=0x7fb814097a20 comp rx=0 tx=0).stop 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 shutdown_connections 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29b025c70 0x55e29b08e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e29b035710 0x55e29b02d230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e29b01d5e0 0x55e29b08e020 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 --2- 172.21.15.37:0/669834273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e29b01c900 0x55e29b020390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb80ea00640 0 client.26063 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.471+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 >> 172.21.15.37:0/669834273 conn(0x55e29afdb200 msgr2=0x55e29b018560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.472+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 shutdown_connections 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.472+0000 7fb832a01740 1 -- 172.21.15.37:0/669834273 wait complete. 2024-03-21T09:13:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.472+0000 7fb832a01740 20 client.26063 trim_cache size 0 max 16384 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: "5e07ddc0-4ae5-4e06-809e-0a17b9c94e48": { 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "5e07ddc0-4ae5-4e06-809e-0a17b9c94e48", 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:13:40.469 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.473+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8100a3140 msgr2=0x7fb8100a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.473+0000 7fb82e600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8100a3140 0x7fb8100a5600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb81c004e00 tx=0x7fb81c04c040 comp rx=0 tx=0).stop 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.473+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 msgr2=0x7fb82819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.473+0000 7fb82e600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 0x7fb82819a4f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb81400d6d0 tx=0x7fb81400db90 comp rx=0 tx=0).stop 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 shutdown_connections 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828109690 0x7fb82819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb8100a3140 0x7fb8100a5600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828104040 0x7fb82819a4f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 --2- 172.21.15.37:0/1976779565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb828103680 0x7fb828199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 >> 172.21.15.37:0/1976779565 conn(0x7fb8280ff190 msgr2=0x7fb8280ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 shutdown_connections 2024-03-21T09:13:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:13:40.474+0000 7fb82e600640 1 -- 172.21.15.37:0/1976779565 wait complete. 2024-03-21T09:13:40.493 INFO:tasks.cephfs.filesystem:scrub status for tag:5e07ddc0-4ae5-4e06-809e-0a17b9c94e48 - {'path': '/', 'tag': '5e07ddc0-4ae5-4e06-809e-0a17b9c94e48', 'options': 'recursive,force'} 2024-03-21T09:13:40.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:40 smithi071 ceph-mon[36597]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:40 smithi053 ceph-mon[36169]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:41.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:41 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:13:41.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:41 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:41.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:41 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:13:41.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:41 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:41.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:41 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:13:41.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:41 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:41.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:41 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:13:41.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:41 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:41.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:41 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:13:41.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:41 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:41.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:41 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:13:41.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:41 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:13:42.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:42 smithi071 ceph-mon[36597]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:42.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:42 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:13:42.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:42 smithi037 ceph-mon[31830]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:42.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:42 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:13:42.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:42 smithi053 ceph-mon[36169]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-21T09:13:42.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:42 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:13:44.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:44 smithi071 ceph-mon[36597]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-21T09:13:44.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:44 smithi071 ceph-mon[36597]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-21T09:13:44.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:44 smithi037 ceph-mon[31830]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-21T09:13:44.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:44 smithi037 ceph-mon[31830]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-21T09:13:44.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:44 smithi053 ceph-mon[36169]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-03-21T09:13:44.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:44 smithi053 ceph-mon[36169]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-21T09:13:45.786 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 55% (6832/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 92% (11368/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 -- 172.21.15.37:0/3130422107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c107770 msgr2=0x7f7d6c107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 --2- 172.21.15.37:0/3130422107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c107770 0x7f7d6c107b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009930 tx=0x7f7d6002bcb0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 -- 172.21.15.37:0/3130422107 shutdown_connections 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 --2- 172.21.15.37:0/3130422107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6c06a650 0x7f7d6c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 --2- 172.21.15.37:0/3130422107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d6c069cb0 0x7f7d6c06a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 --2- 172.21.15.37:0/3130422107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c107770 0x7f7d6c107b50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 -- 172.21.15.37:0/3130422107 >> 172.21.15.37:0/3130422107 conn(0x7f7d6c0773f0 msgr2=0x7f7d6c077800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 -- 172.21.15.37:0/3130422107 shutdown_connections 2024-03-21T09:13:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.386+0000 7f7d74600640 1 -- 172.21.15.37:0/3130422107 wait complete. 2024-03-21T09:13:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.387+0000 7f7d74600640 1 Processor -- start 2024-03-21T09:13:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 -- start start 2024-03-21T09:13:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d6c069cb0 0x7f7d6c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 0x7f7d6c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6c107770 0x7f7d6c19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c111ae0 con 0x7f7d6c069cb0 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c111960 con 0x7f7d6c107770 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d74600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c111c60 con 0x7f7d6c06a650 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d71c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d6c069cb0 0x7f7d6c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d71200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 0x7f7d6c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d71200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 0x7f7d6c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35508/0 (socket says 172.21.15.37:35508) 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d71200640 1 -- 172.21.15.37:0/1072397108 learned_addr learned my addr 172.21.15.37:0/1072397108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.388+0000 7f7d72600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6c107770 0x7f7d6c19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d71200640 1 -- 172.21.15.37:0/1072397108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6c107770 msgr2=0x7f7d6c19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d71200640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6c107770 0x7f7d6c19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d71200640 1 -- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d6c069cb0 msgr2=0x7f7d6c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d71200640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d6c069cb0 0x7f7d6c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d71200640 1 -- 172.21.15.37:0/1072397108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c19f150 con 0x7f7d6c06a650 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d71200640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 0x7f7d6c19a6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800e9f0 tx=0x7f7d5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d6a000640 1 -- 172.21.15.37:0/1072397108 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800cd80 con 0x7f7d6c06a650 2024-03-21T09:13:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d6a000640 1 -- 172.21.15.37:0/1072397108 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d58010910 con 0x7f7d6c06a650 2024-03-21T09:13:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d6a000640 1 -- 172.21.15.37:0/1072397108 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58018990 con 0x7f7d6c06a650 2024-03-21T09:13:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.389+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c19f440 con 0x7f7d6c06a650 2024-03-21T09:13:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.390+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c1a6cd0 con 0x7f7d6c06a650 2024-03-21T09:13:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.393+0000 7f7d773ab740 1 Processor -- start 2024-03-21T09:13:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.393+0000 7f7d6a000640 1 -- 172.21.15.37:0/1072397108 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7d58010ac0 con 0x7f7d6c06a650 2024-03-21T09:13:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.393+0000 7f7d773ab740 1 -- start start 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.393+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.393+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa3628f0 0x561efa36b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa36b6c0 0x561efa36dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d773ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efa3243b0 con 0x561efa36b6c0 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d773ab740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efa2c6830 con 0x561efa3628f0 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d773ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efa26f5d0 con 0x561efa361fd0 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35518/0 (socket says 172.21.15.37:35518) 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 -- 172.21.15.37:0/3609314632 learned_addr learned my addr 172.21.15.37:0/3609314632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d71c00640 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa3628f0 0x561efa36b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d71200640 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa36b6c0 0x561efa36dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 -- 172.21.15.37:0/3609314632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa3628f0 msgr2=0x561efa36b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa3628f0 0x561efa36b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 -- 172.21.15.37:0/3609314632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa36b6c0 msgr2=0x561efa36dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa36b6c0 0x561efa36dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.394+0000 7f7d72600640 1 -- 172.21.15.37:0/3609314632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efa2872c0 con 0x561efa361fd0 2024-03-21T09:13:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d72600640 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3623b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009f40 tx=0x7f7d640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc8d4a6e0e15aaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d6a000640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7d540a3100 0x7f7d540a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d5ea00640 1 -- 172.21.15.37:0/3609314632 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64004d20 con 0x561efa361fd0 2024-03-21T09:13:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d5ea00640 1 -- 172.21.15.37:0/3609314632 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d64004050 con 0x561efa361fd0 2024-03-21T09:13:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d5ea00640 1 -- 172.21.15.37:0/3609314632 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64005220 con 0x561efa361fd0 2024-03-21T09:13:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d6a000640 1 -- 172.21.15.37:0/1072397108 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7d580cfe70 con 0x7f7d6c06a650 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.395+0000 7f7d71c00640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7d540a3100 0x7f7d540a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 -- 172.21.15.37:0/3609314632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 msgr2=0x561efa3623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3623b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009f40 tx=0x7f7d640118d0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 -- 172.21.15.37:0/3609314632 shutdown_connections 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa36b6c0 0x561efa36dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa3628f0 0x561efa36b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 --2- 172.21.15.37:0/3609314632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3623b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 -- 172.21.15.37:0/3609314632 >> 172.21.15.37:0/3609314632 conn(0x561efa35c8f0 msgr2=0x561efa35ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 -- 172.21.15.37:0/3609314632 shutdown_connections 2024-03-21T09:13:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d71c00640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7d540a3100 0x7f7d540a55c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d6002e040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.396+0000 7f7d773ab740 1 -- 172.21.15.37:0/3609314632 wait complete. 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 -- start start 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 Processor -- start 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa368c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa36b6c0 0x561efa369160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efa282a20 con 0x561efa3628f0 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efa2c28d0 con 0x561efa36b6c0 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efa36e2f0 con 0x561efa361fd0 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.397+0000 7f7d773ab740 10 client.0 _ll_register_callbacks cb 0x561efa2bf1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35526/0 (socket says 172.21.15.37:35526) 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 -- 172.21.15.37:0/1012733454 learned_addr learned my addr 172.21.15.37:0/1012733454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d72600640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa36b6c0 0x561efa369160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71200640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa368c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 -- 172.21.15.37:0/1012733454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa36b6c0 msgr2=0x561efa369160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa36b6c0 0x561efa369160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 -- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 msgr2=0x561efa368c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa368c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 -- 172.21.15.37:0/1012733454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ef9fdb210 con 0x561efa361fd0 2024-03-21T09:13:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71200640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa368c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d71c00640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3686e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d60006010 tx=0x7f7d6002bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e39694f8d7c5f4ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.398+0000 7f7d53400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d53400640 1 -- 172.21.15.37:0/1012733454 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60051020 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d53400640 1 -- 172.21.15.37:0/1012733454 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7d600048d0 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d64004050 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d53400640 1 -- 172.21.15.37:0/1012733454 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6005c050 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d773ab740 10 client.26098 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561efa0a7620 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d53400640 1 -- 172.21.15.37:0/1012733454 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d64004050 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.399+0000 7f7d53400640 1 -- 172.21.15.37:0/1012733454 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7d600036f0 con 0x561efa361fd0 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d773ab740 10 client.26098 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d773ab740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d773ab740 20 client.26098 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d773ab740 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa37b740 0x561efa37db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d773ab740 4 client.26098 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561efa2d3d70 con 0x561efa37b740 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.400+0000 7f7d71200640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa37b740 0x561efa37db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.401+0000 7f7d71200640 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa37b740 0x561efa37db30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c037e3de29d9ce7 server_cookie=17dc1869061d237a in_seq=0 out_seq=0 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.401+0000 7f7d53400640 10 client.26098 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.401+0000 7f7d53400640 1 -- 172.21.15.37:0/1012733454 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561efa2d3d70 con 0x561efa37b740 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.401+0000 7f7d53400640 10 client.26098 handle_command_reply: tid=0 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 client.26098 shutdown 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 msgr2=0x561efa3686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3686e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d60006010 tx=0x7f7d6002bcb0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 shutdown_connections 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa37b740 0x561efa37db30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa36b6c0 0x561efa369160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa368c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d53400640 0 client.26098 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 --2- 172.21.15.37:0/1012733454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa361fd0 0x561efa3686e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 >> 172.21.15.37:0/1012733454 conn(0x561efa35c8f0 msgr2=0x561efa35e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 shutdown_connections 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 1 -- 172.21.15.37:0/1012733454 wait complete. 2024-03-21T09:13:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.402+0000 7f7d773ab740 20 client.26098 trim_cache size 0 max 16384 2024-03-21T09:13:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.433+0000 7f7d773ab740 1 Processor -- start 2024-03-21T09:13:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 -- start start 2024-03-21T09:13:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa36ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa376a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa376f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1b3060 con 0x561efa3628f0 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1b0b60 con 0x561efa361fd0 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d773ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1aec00 con 0x561efa36b6c0 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d71c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa36ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d72600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa376f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d71c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa36ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:59574/0 (socket says 172.21.15.37:59574) 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.434+0000 7f7d71c00640 1 -- 172.21.15.37:0/14841180 learned_addr learned my addr 172.21.15.37:0/14841180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d71200640 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa376a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d72600640 1 -- 172.21.15.37:0/14841180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 msgr2=0x561efa36ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d72600640 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa36ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d72600640 1 -- 172.21.15.37:0/14841180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 msgr2=0x561efa376a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d72600640 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa376a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d72600640 1 -- 172.21.15.37:0/14841180 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efa362720 con 0x561efa36b6c0 2024-03-21T09:13:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d71200640 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa376a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:13:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d71c00640 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa36ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d72600640 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa376f40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009250 tx=0x7f7d64003a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa12a876b5a611b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d5e000640 1 -- 172.21.15.37:0/14841180 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64003e30 con 0x561efa36b6c0 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d5e000640 1 -- 172.21.15.37:0/14841180 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d64004530 con 0x561efa36b6c0 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.435+0000 7f7d5e000640 1 -- 172.21.15.37:0/14841180 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d640052a0 con 0x561efa36b6c0 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.436+0000 7f7d773ab740 1 -- 172.21.15.37:0/14841180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 msgr2=0x561efa376f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.436+0000 7f7d773ab740 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa376f40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009250 tx=0x7f7d64003a40 comp rx=0 tx=0).stop 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 -- 172.21.15.37:0/14841180 shutdown_connections 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa376f40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa376a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 --2- 172.21.15.37:0/14841180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa36ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 -- 172.21.15.37:0/14841180 >> 172.21.15.37:0/14841180 conn(0x561efa35c8f0 msgr2=0x561efa35b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 -- 172.21.15.37:0/14841180 shutdown_connections 2024-03-21T09:13:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 -- 172.21.15.37:0/14841180 wait complete. 2024-03-21T09:13:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.437+0000 7f7d773ab740 1 -- start start 2024-03-21T09:13:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 Processor -- start 2024-03-21T09:13:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa372fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa3657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa365ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1af200 con 0x561efa3628f0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1b0500 con 0x561efa361fd0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1af840 con 0x561efa36b6c0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.438+0000 7f7d773ab740 10 client.0 _ll_register_callbacks cb 0x7f7d60044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa3657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d72600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa365ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa372fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d72600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa365ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35532/0 (socket says 172.21.15.37:35532) 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d72600640 1 -- 172.21.15.37:0/181341769 learned_addr learned my addr 172.21.15.37:0/181341769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa3657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48262/0 (socket says 172.21.15.37:48262) 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 -- 172.21.15.37:0/181341769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 msgr2=0x561efa365ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa365ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 -- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 msgr2=0x561efa3657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa3657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 -- 172.21.15.37:0/181341769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561efa366220 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d72600640 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa365ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d71c00640 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa372fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d60003b10 tx=0x7f7d60003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5177307a8a440d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.439+0000 7f7d53400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d53400640 1 -- 172.21.15.37:0/181341769 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60064070 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d53400640 1 -- 172.21.15.37:0/181341769 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7d600048d0 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d64004530 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d53400640 1 -- 172.21.15.37:0/181341769 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d60070050 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d773ab740 10 client.26081 fetch_fsmap learned FSMap version 24 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561efa3d7c50 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.440+0000 7f7d53400640 1 -- 172.21.15.37:0/181341769 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d64004530 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d53400640 1 -- 172.21.15.37:0/181341769 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7d6004d480 con 0x561efa361fd0 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d773ab740 10 client.26081 fetch_fsmap finished waiting for FSMap version 24 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d773ab740 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d773ab740 20 client.26081 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d773ab740 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa3d8260 0x561efa3da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d773ab740 4 client.26081 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:13:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561efa2ce0c0 con 0x561efa3d8260 2024-03-21T09:13:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.441+0000 7f7d71200640 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa3d8260 0x561efa3da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.442+0000 7f7d71200640 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa3d8260 0x561efa3da650 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589e81bf3a37e15b server_cookie=6f69629ca1ee5143 in_seq=0 out_seq=0 2024-03-21T09:13:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.442+0000 7f7d53400640 10 client.26081 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.446+0000 7f7d53400640 1 -- 172.21.15.37:0/181341769 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561efa2ce0c0 con 0x561efa3d8260 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.446+0000 7f7d53400640 10 client.26081 handle_command_reply: tid=0 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 client.26081 shutdown 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 msgr2=0x561efa372fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa372fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d60003b10 tx=0x7f7d60003f30 comp rx=0 tx=0).stop 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 shutdown_connections 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561efa36b6c0 0x561efa365ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561efa3628f0 0x561efa3657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561efa3d8260 0x561efa3da650 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 --2- 172.21.15.37:0/181341769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561efa361fd0 0x561efa372fe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d53400640 0 client.26081 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 >> 172.21.15.37:0/181341769 conn(0x561efa35c8f0 msgr2=0x561efa35b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 shutdown_connections 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 1 -- 172.21.15.37:0/181341769 wait complete. 2024-03-21T09:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.447+0000 7f7d773ab740 20 client.26081 trim_cache size 0 max 16384 2024-03-21T09:13:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:13:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:13:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:13:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7d540a3100 msgr2=0x7f7d540a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7d540a3100 0x7f7d540a55c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d6002e040 comp rx=0 tx=0).stop 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 msgr2=0x7f7d6c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 0x7f7d6c19a6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800e9f0 tx=0x7f7d5800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 shutdown_connections 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7d540a3100 0x7f7d540a55c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6c107770 0x7f7d6c19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d6c06a650 0x7f7d6c19a6a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 --2- 172.21.15.37:0/1072397108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d6c069cb0 0x7f7d6c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 >> 172.21.15.37:0/1072397108 conn(0x7f7d6c0773f0 msgr2=0x7f7d6c10cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 shutdown_connections 2024-03-21T09:13:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:46.448+0000 7f7d74600640 1 -- 172.21.15.37:0/1072397108 wait complete. 2024-03-21T09:13:46.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:46 smithi071 ceph-mon[36597]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:46.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:46 smithi037 ceph-mon[31830]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-03-21T09:13:46.684 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.685 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-03-21T09:13:46.685 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.685 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T09:13:46.685 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-21T09:13:46.685 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T09:13:46.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:46 smithi053 ceph-mon[36169]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:46.694 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T09:13:46.749 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T09:13:46.756 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T09:13:46.756 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T09:13:47.677 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T09:13:47.806 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T09:13:47.894 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T09:13:47.900 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T09:13:47.900 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T09:13:47.985 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T09:13:47.996 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:13:47.996 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T09:13:48.047 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-03-21T09:13:48.048 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-03-21T09:13:48.048 DEBUG:teuthology.orchestra.run.smithi037:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-03-21T09:13:48.113 INFO:tasks.workunit.client.0.smithi037.stdout:test large (16MB) dio write 2024-03-21T09:13:48.113 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 'test large (16MB) dio write' 2024-03-21T09:13:48.113 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-03-21T09:13:48.456 INFO:tasks.workunit.client.0.smithi037.stderr:1+0 records in 2024-03-21T09:13:48.457 INFO:tasks.workunit.client.0.smithi037.stderr:1+0 records out 2024-03-21T09:13:48.457 INFO:tasks.workunit.client.0.smithi037.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.33744 s, 49.7 MB/s 2024-03-21T09:13:48.459 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-03-21T09:13:48.459 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo OK 2024-03-21T09:13:48.460 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T09:13:48.460 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T09:13:48.550 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:48 smithi037 ceph-mon[31830]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-21T09:13:48.582 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-03-21T09:13:48.582 DEBUG:teuthology.orchestra.run.smithi037:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-03-21T09:13:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:48 smithi071 ceph-mon[36597]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-21T09:13:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:48 smithi053 ceph-mon[36169]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-21T09:13:50.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:50 smithi071 ceph-mon[36597]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:50.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:50 smithi037 ceph-mon[31830]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:50.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:50 smithi053 ceph-mon[36169]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:52 smithi071 ceph-mon[36597]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:52.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:52 smithi037 ceph-mon[31830]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:52.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:52 smithi053 ceph-mon[36169]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-21T09:13:53.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:53 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:13:53.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:53 smithi071 ceph-mon[36597]: pgmap v54: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 297 op/s 2024-03-21T09:13:53.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:53 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:13:53.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:53 smithi037 ceph-mon[31830]: pgmap v54: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 297 op/s 2024-03-21T09:13:53.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:53 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:13:53.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:53 smithi053 ceph-mon[36169]: pgmap v54: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 297 op/s 2024-03-21T09:13:56.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:56 smithi037 ceph-mon[31830]: pgmap v55: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-03-21T09:13:56.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:56 smithi053 ceph-mon[36169]: pgmap v55: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-03-21T09:13:56.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:56 smithi071 ceph-mon[36597]: pgmap v55: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-03-21T09:13:57.993 INFO:tasks.workunit.client.0.smithi037.stdout:.......... 2024-03-21T09:13:57.993 INFO:tasks.workunit.client.0.smithi037.stdout:do_o_direct_reads: SUCCESS 2024-03-21T09:13:57.996 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T09:13:57.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T09:13:58.115 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-03-21T09:13:58.115 DEBUG:teuthology.orchestra.run.smithi037:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-03-21T09:13:58.197 INFO:tasks.workunit.client.0.smithi037.stdout:test read from hole 2024-03-21T09:13:58.198 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 'test read from hole' 2024-03-21T09:13:58.198 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-03-21T09:13:58.199 INFO:tasks.workunit.client.0.smithi037.stderr:0+0 records in 2024-03-21T09:13:58.200 INFO:tasks.workunit.client.0.smithi037.stderr:0+0 records out 2024-03-21T09:13:58.200 INFO:tasks.workunit.client.0.smithi037.stderr:0 bytes copied, 9.66e-05 s, 0.0 kB/s 2024-03-21T09:13:58.200 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-03-21T09:13:58.205 INFO:tasks.workunit.client.0.smithi037.stderr:2+0 records in 2024-03-21T09:13:58.205 INFO:tasks.workunit.client.0.smithi037.stderr:2+0 records out 2024-03-21T09:13:58.205 INFO:tasks.workunit.client.0.smithi037.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00247237 s, 414 kB/s 2024-03-21T09:13:58.206 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-03-21T09:13:58.209 INFO:tasks.workunit.client.0.smithi037.stderr:2+0 records in 2024-03-21T09:13:58.209 INFO:tasks.workunit.client.0.smithi037.stderr:2+0 records out 2024-03-21T09:13:58.209 INFO:tasks.workunit.client.0.smithi037.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00011127 s, 9.2 MB/s 2024-03-21T09:13:58.210 INFO:tasks.workunit.client.0.smithi037.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-03-21T09:13:58.213 INFO:tasks.workunit.client.0.smithi037.stdout:other thing 2024-03-21T09:13:58.213 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 'other thing' 2024-03-21T09:13:58.213 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-03-21T09:13:58.215 INFO:tasks.workunit.client.0.smithi037.stderr:1+0 records in 2024-03-21T09:13:58.215 INFO:tasks.workunit.client.0.smithi037.stderr:1+0 records out 2024-03-21T09:13:58.215 INFO:tasks.workunit.client.0.smithi037.stderr:500 bytes copied, 0.000124238 s, 4.0 MB/s 2024-03-21T09:13:58.216 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-03-21T09:13:58.224 INFO:tasks.workunit.client.0.smithi037.stderr:0+1 records in 2024-03-21T09:13:58.224 INFO:tasks.workunit.client.0.smithi037.stderr:0+1 records out 2024-03-21T09:13:58.224 INFO:tasks.workunit.client.0.smithi037.stderr:500 bytes copied, 0.00605177 s, 82.6 kB/s 2024-03-21T09:13:58.225 INFO:tasks.workunit.client.0.smithi037.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-03-21T09:13:58.236 INFO:tasks.workunit.client.0.smithi037.stderr:0+1 records in 2024-03-21T09:13:58.236 INFO:tasks.workunit.client.0.smithi037.stderr:0+1 records out 2024-03-21T09:13:58.236 INFO:tasks.workunit.client.0.smithi037.stderr:500 bytes copied, 0.00905835 s, 55.2 kB/s 2024-03-21T09:13:58.236 INFO:tasks.workunit.client.0.smithi037.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-03-21T09:13:58.240 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-03-21T09:13:58.241 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo OK 2024-03-21T09:13:58.241 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T09:13:58.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T09:13:58.342 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-03-21T09:13:58.342 DEBUG:teuthology.orchestra.run.smithi037:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-03-21T09:13:58.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:58 smithi037 ceph-mon[31830]: pgmap v56: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 724 op/s 2024-03-21T09:13:58.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:13:58 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:13:58.423 INFO:tasks.workunit.client.0.smithi037.stdout:writing first 3 bytes of 10k file 2024-03-21T09:13:58.423 INFO:tasks.workunit.client.0.smithi037.stdout:reading O_DIRECT 2024-03-21T09:13:58.423 INFO:tasks.workunit.client.0.smithi037.stdout:got 10000 2024-03-21T09:13:58.424 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T09:13:58.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T09:13:58.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:58 smithi053 ceph-mon[36169]: pgmap v56: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 724 op/s 2024-03-21T09:13:58.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:13:58 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:13:58.521 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-03-21T09:13:58.521 DEBUG:teuthology.orchestra.run.smithi037:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-03-21T09:13:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:58 smithi071 ceph-mon[36597]: pgmap v56: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 724 op/s 2024-03-21T09:13:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:13:58 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:13:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.583+0000 7fd4e7200640 1 -- 172.21.15.37:0/1866391522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e0069cb0 msgr2=0x7fd4e006a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.583+0000 7fd4e7200640 1 --2- 172.21.15.37:0/1866391522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e0069cb0 0x7fd4e006a090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009900 tx=0x7fd4d002bc30 comp rx=0 tx=0).stop 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 -- 172.21.15.37:0/1866391522 shutdown_connections 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 --2- 172.21.15.37:0/1866391522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4e010e3a0 0x7fd4e0110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 --2- 172.21.15.37:0/1866391522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e006a5d0 0x7fd4e010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 --2- 172.21.15.37:0/1866391522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e0069cb0 0x7fd4e006a090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 -- 172.21.15.37:0/1866391522 >> 172.21.15.37:0/1866391522 conn(0x7fd4e0077360 msgr2=0x7fd4e0077770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 -- 172.21.15.37:0/1866391522 shutdown_connections 2024-03-21T09:13:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.584+0000 7fd4e7200640 1 -- 172.21.15.37:0/1866391522 wait complete. 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 Processor -- start 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 -- start start 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e0069cb0 0x7fd4e019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 0x7fd4e019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4e010e3a0 0x7fd4e019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0113d40 con 0x7fd4e0069cb0 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0113bc0 con 0x7fd4e010e3a0 2024-03-21T09:13:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.585+0000 7fd4e7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0113ec0 con 0x7fd4e006a5d0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 0x7fd4e019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 0x7fd4e019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:53600/0 (socket says 172.21.15.37:53600) 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 -- 172.21.15.37:0/2267898382 learned_addr learned my addr 172.21.15.37:0/2267898382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4e5200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4e010e3a0 0x7fd4e019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e0069cb0 0x7fd4e019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 -- 172.21.15.37:0/2267898382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4e010e3a0 msgr2=0x7fd4e019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4e010e3a0 0x7fd4e019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 -- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e0069cb0 msgr2=0x7fd4e019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e0069cb0 0x7fd4e019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 -- 172.21.15.37:0/2267898382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e019f1a0 con 0x7fd4e006a5d0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.586+0000 7fd4df400640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 0x7fd4e019a740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00d950 tx=0x7fd4cc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.587+0000 7fd4dcc00640 1 -- 172.21.15.37:0/2267898382 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc017030 con 0x7fd4e006a5d0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.587+0000 7fd4dcc00640 1 -- 172.21.15.37:0/2267898382 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4cc004020 con 0x7fd4e006a5d0 2024-03-21T09:13:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.587+0000 7fd4dcc00640 1 -- 172.21.15.37:0/2267898382 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0047a0 con 0x7fd4e006a5d0 2024-03-21T09:13:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.587+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e019f490 con 0x7fd4e006a5d0 2024-03-21T09:13:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.587+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e01a6d20 con 0x7fd4e006a5d0 2024-03-21T09:13:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 Processor -- start 2024-03-21T09:13:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 -- start start 2024-03-21T09:13:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4dcc00640 1 -- 172.21.15.37:0/2267898382 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd4cc0041d0 con 0x7fd4e006a5d0 2024-03-21T09:13:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd4013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd4018f0 0x563ffd40a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd40a6c0 0x563ffd40cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffd3c33b0 con 0x563ffd4018f0 2024-03-21T09:13:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffd365830 con 0x563ffd40a6c0 2024-03-21T09:13:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.590+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffd30e5d0 con 0x563ffd400fd0 2024-03-21T09:13:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4df400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd40a6c0 0x563ffd40cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd4013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd4018f0 0x563ffd40a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd4013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:53616/0 (socket says 172.21.15.37:53616) 2024-03-21T09:13:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 -- 172.21.15.37:0/4117987469 learned_addr learned my addr 172.21.15.37:0/4117987469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 -- 172.21.15.37:0/4117987469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd40a6c0 msgr2=0x563ffd40cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd40a6c0 0x563ffd40cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 -- 172.21.15.37:0/4117987469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd4018f0 msgr2=0x563ffd40a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd4018f0 0x563ffd40a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4e5200640 1 -- 172.21.15.37:0/4117987469 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ffd3262c0 con 0x563ffd400fd0 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.591+0000 7fd4df400640 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd40a6c0 0x563ffd40cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.592+0000 7fd4e5200640 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd4013b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009f30 tx=0x7fd4d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b8880842f60bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.592+0000 7fd4d5600640 1 -- 172.21.15.37:0/4117987469 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8004d20 con 0x563ffd400fd0 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.592+0000 7fd4d5600640 1 -- 172.21.15.37:0/4117987469 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4d8004050 con 0x563ffd400fd0 2024-03-21T09:13:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.592+0000 7fd4d5600640 1 -- 172.21.15.37:0/4117987469 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8005260 con 0x563ffd400fd0 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.592+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4117987469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 msgr2=0x563ffd4013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.592+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd4013b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009f30 tx=0x7fd4d80118d0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4117987469 shutdown_connections 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd40a6c0 0x563ffd40cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd4018f0 0x563ffd40a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4117987469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd4013b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4117987469 >> 172.21.15.37:0/4117987469 conn(0x563ffd3fb8f0 msgr2=0x563ffd3fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4117987469 shutdown_connections 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4117987469 wait complete. 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4dcc00640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd4c40a3010 0x7fd4c40a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4dcc00640 1 -- 172.21.15.37:0/2267898382 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd4cc0cf530 con 0x7fd4e006a5d0 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd4c40a3010 0x7fd4c40a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.593+0000 7fd4e9fdb740 1 -- start start 2024-03-21T09:13:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 Processor -- start 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd4c40a3010 0x7fd4c40a54d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4d002c240 tx=0x7fd4d002df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd400fd0 0x563ffd407760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd407ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 0x563ffd4081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffd321a20 con 0x563ffd400fd0 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffd3618d0 con 0x563ffd4018f0 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffd40d2f0 con 0x563ffd40a6c0 2024-03-21T09:13:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e9fdb740 10 client.0 _ll_register_callbacks cb 0x563ffd35e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4df400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd407ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 0x563ffd4081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 0x563ffd4081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:53624/0 (socket says 172.21.15.37:53624) 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4df400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd407ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46306/0 (socket says 172.21.15.37:46306) 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.594+0000 7fd4e5200640 1 -- 172.21.15.37:0/4240796113 learned_addr learned my addr 172.21.15.37:0/4240796113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd400fd0 0x563ffd407760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4e5200640 1 -- 172.21.15.37:0/4240796113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 msgr2=0x563ffd407ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4e5200640 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd407ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4e5200640 1 -- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd400fd0 msgr2=0x563ffd407760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4e5200640 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd400fd0 0x563ffd407760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4e5200640 1 -- 172.21.15.37:0/4240796113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ffd07a210 con 0x563ffd40a6c0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4e5200640 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 0x563ffd4081e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8005c40 tx=0x7fd4d8003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fc9f1f0c7645e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4ca000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.595+0000 7fd4ca000640 1 -- 172.21.15.37:0/4240796113 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8029020 con 0x563ffd40a6c0 2024-03-21T09:13:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4ca000640 1 -- 172.21.15.37:0/4240796113 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd4d8004610 con 0x563ffd40a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4d8004050 con 0x563ffd40a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4ca000640 1 -- 172.21.15.37:0/4240796113 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4d8034050 con 0x563ffd40a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 10 client.26116 fetch_fsmap learned FSMap version 26 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563ffd146620 con 0x563ffd40a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4ca000640 1 -- 172.21.15.37:0/4240796113 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4d8004050 con 0x563ffd40a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4ca000640 1 -- 172.21.15.37:0/4240796113 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd4d8025470 con 0x563ffd40a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 10 client.26116 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 20 client.26116 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd41a6c0 0x563ffd41cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 4 client.26116 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.596+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ffd372d70 con 0x563ffd41a6c0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.597+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd41a6c0 0x563ffd41cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.597+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd41a6c0 0x563ffd41cab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769d306db93a0bb6 server_cookie=f26f55d50d1e9962 in_seq=0 out_seq=0 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.597+0000 7fd4ca000640 10 client.26116 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4ca000640 1 -- 172.21.15.37:0/4240796113 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563ffd372d70 con 0x563ffd41a6c0 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4ca000640 10 client.26116 handle_command_reply: tid=0 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 client.26116 shutdown 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 msgr2=0x563ffd4081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 0x563ffd4081e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8005c40 tx=0x7fd4d8003ea0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 shutdown_connections 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd40a6c0 0x563ffd4081e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd41a6c0 0x563ffd41cab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd407ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/4240796113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd400fd0 0x563ffd407760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4ca000640 0 client.26116 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 >> 172.21.15.37:0/4240796113 conn(0x563ffd3fb8f0 msgr2=0x563ffd3fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 shutdown_connections 2024-03-21T09:13:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.598+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/4240796113 wait complete. 2024-03-21T09:13:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.599+0000 7fd4e9fdb740 20 client.26116 trim_cache size 0 max 16384 2024-03-21T09:13:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.613+0000 7fd4e9fdb740 1 Processor -- start 2024-03-21T09:13:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.613+0000 7fd4e9fdb740 1 -- start start 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd409e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd3fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd415a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01b30f0 con 0x563ffd40a6c0 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01b0bf0 con 0x563ffd4018f0 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01aec90 con 0x563ffd400fd0 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd409e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4df400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd3fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd409e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:53626/0 (socket says 172.21.15.37:53626) 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4dfe00640 1 -- 172.21.15.37:0/57981237 learned_addr learned my addr 172.21.15.37:0/57981237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4e5200640 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd415a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4df400640 1 -- 172.21.15.37:0/57981237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 msgr2=0x563ffd409e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4df400640 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd409e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4df400640 1 -- 172.21.15.37:0/57981237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 msgr2=0x563ffd415a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4df400640 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd415a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.614+0000 7fd4df400640 1 -- 172.21.15.37:0/57981237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ffd3fc270 con 0x563ffd4018f0 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4df400640 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd3fd3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cc880 tx=0x7fd4cc0cf850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef570e2cbbaa247 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4d4c00640 1 -- 172.21.15.37:0/57981237 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0977c0 con 0x563ffd4018f0 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4d4c00640 1 -- 172.21.15.37:0/57981237 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4cc09bc20 con 0x563ffd4018f0 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4d4c00640 1 -- 172.21.15.37:0/57981237 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc09ae80 con 0x563ffd4018f0 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/57981237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 msgr2=0x563ffd3fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd3fd3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cc880 tx=0x7fd4cc0cf850 comp rx=0 tx=0).stop 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/57981237 shutdown_connections 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd415a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd3fd3e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/57981237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd409e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/57981237 >> 172.21.15.37:0/57981237 conn(0x563ffd3fb8f0 msgr2=0x563ffd3fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.615+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/57981237 shutdown_connections 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/57981237 wait complete. 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 -- start start 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 Processor -- start 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd404480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd4049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd474de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01af290 con 0x563ffd40a6c0 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01b0590 con 0x563ffd4018f0 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.616+0000 7fd4e9fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01af8d0 con 0x563ffd400fd0 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4e9fdb740 10 client.0 _ll_register_callbacks cb 0x7fd4d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd4049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd404480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd404480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:53642/0 (socket says 172.21.15.37:53642) 2024-03-21T09:13:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4dfe00640 1 -- 172.21.15.37:0/1473554053 learned_addr learned my addr 172.21.15.37:0/1473554053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd4049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46328/0 (socket says 172.21.15.37:46328) 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4e5200640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd474de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 -- 172.21.15.37:0/1473554053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 msgr2=0x563ffd404480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd404480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 -- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 msgr2=0x563ffd474de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd474de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 -- 172.21.15.37:0/1473554053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ffd475320 con 0x563ffd4018f0 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd404480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4df400640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd4049c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc09bdd0 tx=0x7fd4cc097100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2c8a9d79d072ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.617+0000 7fd4ca000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4ca000640 1 -- 172.21.15.37:0/1473554053 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc09b060 con 0x563ffd4018f0 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4ca000640 1 -- 172.21.15.37:0/1473554053 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd4cc0cf0d0 con 0x563ffd4018f0 2024-03-21T09:13:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4cc09bc20 con 0x563ffd4018f0 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4ca000640 1 -- 172.21.15.37:0/1473554053 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4cc09b2f0 con 0x563ffd4018f0 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 10 client.26099 fetch_fsmap learned FSMap version 26 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563ffd476510 con 0x563ffd4018f0 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4ca000640 1 -- 172.21.15.37:0/1473554053 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4cc09bc20 con 0x563ffd4018f0 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4ca000640 1 -- 172.21.15.37:0/1473554053 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd4cc0afb70 con 0x563ffd4018f0 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 10 client.26099 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 20 client.26099 populate_metadata read hostname 'smithi037' 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd476b90 0x563ffd478fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 4 client.26099 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.618+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563ffd36d0c0 con 0x563ffd476b90 2024-03-21T09:13:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.619+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd476b90 0x563ffd478fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:13:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.619+0000 7fd4dfe00640 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd476b90 0x563ffd478fb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd64cbabd35fb9c6 server_cookie=146f7728723281d2 in_seq=0 out_seq=0 2024-03-21T09:13:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.619+0000 7fd4ca000640 10 client.26099 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.629+0000 7fd4ca000640 1 -- 172.21.15.37:0/1473554053 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563ffd36d0c0 con 0x563ffd476b90 2024-03-21T09:13:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.629+0000 7fd4ca000640 10 client.26099 handle_command_reply: tid=0 2024-03-21T09:13:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.629+0000 7fd4e9fdb740 1 client.26099 shutdown 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 msgr2=0x563ffd4049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd4049c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc09bdd0 tx=0x7fd4cc097100 comp rx=0 tx=0).stop 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 shutdown_connections 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ffd40a6c0 0x563ffd474de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x563ffd476b90 0x563ffd478fb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563ffd4018f0 0x563ffd4049c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 --2- 172.21.15.37:0/1473554053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ffd400fd0 0x563ffd404480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 >> 172.21.15.37:0/1473554053 conn(0x563ffd3fb8f0 msgr2=0x563ffd407f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4ca000640 0 client.26099 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 shutdown_connections 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 1 -- 172.21.15.37:0/1473554053 wait complete. 2024-03-21T09:13:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.630+0000 7fd4e9fdb740 20 client.26099 trim_cache size 0 max 16384 2024-03-21T09:13:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:13:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:13:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:13:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:13:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd4c40a3010 msgr2=0x7fd4c40a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd4c40a3010 0x7fd4c40a54d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4d002c240 tx=0x7fd4d002df90 comp rx=0 tx=0).stop 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 msgr2=0x7fd4e019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 0x7fd4e019a740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00d950 tx=0x7fd4cc00de10 comp rx=0 tx=0).stop 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 shutdown_connections 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd4c40a3010 0x7fd4c40a54d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4e010e3a0 0x7fd4e019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4e006a5d0 0x7fd4e019a740 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 --2- 172.21.15.37:0/2267898382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e0069cb0 0x7fd4e019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 >> 172.21.15.37:0/2267898382 conn(0x7fd4e0077360 msgr2=0x7fd4e010f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 shutdown_connections 2024-03-21T09:13:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:13:59.631+0000 7fd4e7200640 1 -- 172.21.15.37:0/2267898382 wait complete. 2024-03-21T09:14:00.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:00 smithi037 ceph-mon[31830]: pgmap v57: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-03-21T09:14:00.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:00 smithi053 ceph-mon[36169]: pgmap v57: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-03-21T09:14:00.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:00 smithi071 ceph-mon[36597]: pgmap v57: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-03-21T09:14:02.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:02 smithi071 ceph-mon[36597]: pgmap v58: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-03-21T09:14:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:02 smithi037 ceph-mon[31830]: pgmap v58: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-03-21T09:14:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:02 smithi053 ceph-mon[36169]: pgmap v58: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2024-03-21T09:14:04.192 INFO:tasks.workunit.client.0.smithi037.stdout:writing pattern 2024-03-21T09:14:04.192 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-03-21T09:14:04.192 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-21T09:14:04.192 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-03-21T09:14:04.193 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:04.194 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:04.195 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:04.196 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:04.197 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:04.198 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:04.199 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:04.288 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:04.289 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:04.290 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:04.291 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:04.292 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:04.293 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:04.294 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:04.295 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:04.338 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:04.338 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:04.338 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:04.338 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:04.339 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:04.339 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:04.339 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:04.339 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:04.339 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:04.339 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:04.340 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:04.340 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:04.340 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:04.340 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:04.340 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:04.340 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:04.341 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:04.341 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:04.341 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:04.341 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:04.341 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:04.341 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:04.342 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:04.342 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:04.342 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:04.342 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:04.342 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:04.342 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:04.343 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:04.344 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:04.345 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:04.346 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:04.346 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:04.346 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:04.346 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:04.346 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:04.346 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:04.347 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:04.348 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:04.348 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:04.348 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:04.348 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:04.348 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:04.348 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:04.349 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:04.350 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:04.395 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:04.395 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:04.395 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:04.395 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:04.395 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:04.396 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:04.397 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:04.397 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:04.397 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:04.397 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:04.397 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:04.397 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:04.398 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:04.398 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:04.398 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:04.398 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:04.398 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:04.398 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:04.399 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:04.399 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:04.399 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:04.399 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:04.399 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:04.399 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:04.400 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:04.401 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:04.401 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:04.401 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:04.401 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:04.401 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:04.401 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:04.402 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:04.402 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:04.402 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:04.402 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:04.402 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:04.402 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:04.403 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:04.404 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:04.404 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:04.404 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-03-21T09:14:04.404 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-21T09:14:04.404 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-03-21T09:14:04.404 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-21T09:14:04.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-03-21T09:14:04.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-21T09:14:04.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-03-21T09:14:04.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-21T09:14:04.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-03-21T09:14:04.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-21T09:14:04.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-03-21T09:14:04.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-21T09:14:04.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-03-21T09:14:04.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-21T09:14:04.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-03-21T09:14:04.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-21T09:14:04.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-03-21T09:14:04.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-21T09:14:04.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-03-21T09:14:04.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-21T09:14:04.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:04.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:04.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:04.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:04 smithi037 ceph-mon[31830]: pgmap v59: 129 pgs: 129 active+clean; 4.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 825 op/s 2024-03-21T09:14:04.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:04 smithi053 ceph-mon[36169]: pgmap v59: 129 pgs: 129 active+clean; 4.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 825 op/s 2024-03-21T09:14:04.484 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:04.485 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:04.486 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:04.487 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:04.488 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:04.489 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:04.490 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:04.490 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:04.490 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:04.490 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:04.490 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:04.490 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:04.491 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:04.492 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:04.492 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:04.492 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:04.492 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:04.564 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:04.565 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:04.566 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:04.567 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:04.568 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:04.569 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:04.570 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:04.570 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:04.570 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:04.570 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:04.570 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:04.570 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:04.571 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:04.571 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:04.571 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:04.571 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:04.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:04 smithi071 ceph-mon[36597]: pgmap v59: 129 pgs: 129 active+clean; 4.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 825 op/s 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:04.616 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:04.617 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:04.618 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:04.619 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:04.620 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:04.621 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:04.622 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:04.623 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:04.688 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:04.688 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:04.688 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:04.689 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:04.690 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:04.691 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:04.692 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:04.692 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:04.692 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:04.692 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:04.692 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:04.693 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:04.693 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:04.693 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:04.693 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:04.693 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:04.694 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:04.694 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:04.694 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:04.694 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:04.694 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:04.694 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-03-21T09:14:04.695 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-21T09:14:04.695 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-03-21T09:14:04.695 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-21T09:14:04.695 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-03-21T09:14:04.696 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-21T09:14:04.696 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-03-21T09:14:04.696 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-21T09:14:04.696 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-03-21T09:14:04.696 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-21T09:14:04.696 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-03-21T09:14:04.697 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:04.698 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:04.699 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:04.699 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:04.699 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:04.699 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:04.699 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:04.699 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:04.700 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:04.700 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:04.700 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:04.700 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:04.701 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:04.701 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:04.701 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:04.701 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:04.701 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:04.701 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:04.702 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:04.702 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:04.702 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:04.777 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:04.777 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:04.778 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:04.779 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:04.780 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:04.781 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:04.782 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:04.783 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:04.783 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:04.783 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:04.783 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:04.783 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:04.851 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:04.852 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:04.853 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:04.854 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:04.855 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:04.856 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:04.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:04.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:04.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:04.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:04.904 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:04.905 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:04.975 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:04.976 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-21T09:14:04.977 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:04.978 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:04.979 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:04.980 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:05.135 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:05.136 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:05.137 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:05.138 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:05.139 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:05.140 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:05.141 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:05.141 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:05.141 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:05.141 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:05.141 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:05.141 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:05.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:05.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:05.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:05.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:05.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:05.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:05.184 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:05.185 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:05.186 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:05.187 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:05.188 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:05.255 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:05.255 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:05.255 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:05.256 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-03-21T09:14:05.257 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:05.258 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:05.259 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:05.260 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:05.261 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:05.261 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:05.261 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:05.261 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:05.350 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:05.350 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:05.350 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:05.351 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:05.352 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:05.353 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:05.354 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:05.357 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:05.357 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:05.358 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:05.359 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:05.359 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:05.359 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:05.359 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:05.359 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:05.405 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:05.406 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:05.407 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:05.408 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:05.409 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:05.410 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:05.411 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:05.452 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:05.454 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:05.455 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:05.457 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:05.457 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:05.457 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:05.530 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:05.530 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:05.530 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-03-21T09:14:05.531 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:05.532 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:05.533 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:05.630 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:05.631 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:05.633 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:05.634 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:05.636 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:05.636 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:05.636 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:05.676 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:05.678 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:05.679 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:05.681 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:05.719 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:05.720 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:05.721 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:05.723 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:05.725 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-21T09:14:05.806 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:05.808 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:05.810 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:05.811 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:05.812 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:05.896 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:05.898 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:05.898 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:05.898 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:05.898 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:05.898 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:05.898 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:05.899 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:05.899 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:05.899 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:05.899 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:05.899 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:05.899 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:05.901 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:05.903 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:05.942 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:05.943 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:05.944 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:05.945 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:05.946 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:05.948 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:05.993 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:05.994 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:05.996 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-03-21T09:14:05.997 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-21T09:14:05.998 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-03-21T09:14:05.998 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-21T09:14:05.998 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-03-21T09:14:05.998 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-21T09:14:05.998 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-03-21T09:14:05.998 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-21T09:14:06.084 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-03-21T09:14:06.084 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-21T09:14:06.084 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-03-21T09:14:06.084 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-21T09:14:06.084 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:06.085 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:06.088 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:06.090 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:06.090 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:06.090 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:06.090 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:06.090 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:06.090 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:06.178 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:06.179 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:06.181 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:06.182 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:06.183 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:06.222 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:06.225 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:06.227 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:06.228 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:06.228 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:06.228 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:06.228 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:06.228 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:06.228 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:06.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:06 smithi037 ceph-mon[31830]: pgmap v60: 129 pgs: 129 active+clean; 4.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 36 KiB/s wr, 535 op/s 2024-03-21T09:14:06.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:06 smithi053 ceph-mon[36169]: pgmap v60: 129 pgs: 129 active+clean; 4.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 36 KiB/s wr, 535 op/s 2024-03-21T09:14:06.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:06 smithi071 ceph-mon[36597]: pgmap v60: 129 pgs: 129 active+clean; 4.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 36 KiB/s wr, 535 op/s 2024-03-21T09:14:06.890 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:06.890 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:06.890 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:06.890 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:06.891 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:06.892 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:06.893 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-21T09:14:06.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-21T09:14:06.895 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:08.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:08 smithi037 ceph-mon[31830]: pgmap v61: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 741 KiB/s wr, 687 op/s 2024-03-21T09:14:08.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:08 smithi053 ceph-mon[36169]: pgmap v61: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 741 KiB/s wr, 687 op/s 2024-03-21T09:14:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:08 smithi071 ceph-mon[36597]: pgmap v61: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 741 KiB/s wr, 687 op/s 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:09.315 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-21T09:14:09.317 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:09.318 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:09.319 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:10.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:10 smithi037 ceph-mon[31830]: pgmap v62: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 737 KiB/s wr, 256 op/s 2024-03-21T09:14:10.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:10 smithi053 ceph-mon[36169]: pgmap v62: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 737 KiB/s wr, 256 op/s 2024-03-21T09:14:10.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:14:10.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:14:10.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:10 smithi071 ceph-mon[36597]: pgmap v62: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 737 KiB/s wr, 256 op/s 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-21T09:14:10.709 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:10.711 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:10.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:10.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:10.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:10.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 -- 172.21.15.37:0/554390854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8109870 msgr2=0x7f26d8109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 --2- 172.21.15.37:0/554390854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8109870 0x7f26d8109c50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009900 tx=0x7f26c802bc30 comp rx=0 tx=0).stop 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 -- 172.21.15.37:0/554390854 shutdown_connections 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 --2- 172.21.15.37:0/554390854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8104220 0x7f26d8110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 --2- 172.21.15.37:0/554390854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26d8103860 0x7f26d8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 --2- 172.21.15.37:0/554390854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8109870 0x7f26d8109c50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.741+0000 7f26df200640 1 -- 172.21.15.37:0/554390854 >> 172.21.15.37:0/554390854 conn(0x7f26d80ff310 msgr2=0x7f26d8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.742+0000 7f26df200640 1 -- 172.21.15.37:0/554390854 shutdown_connections 2024-03-21T09:14:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.742+0000 7f26df200640 1 -- 172.21.15.37:0/554390854 wait complete. 2024-03-21T09:14:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 Processor -- start 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 -- start start 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 0x7f26d819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8104220 0x7f26d819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26d8109870 0x7f26d819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8113cd0 con 0x7f26d8109870 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8113b50 con 0x7f26d8104220 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.743+0000 7f26df200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8113e50 con 0x7f26d8103860 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 0x7f26d819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 0x7f26d819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52588/0 (socket says 172.21.15.37:52588) 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 -- 172.21.15.37:0/2272470916 learned_addr learned my addr 172.21.15.37:0/2272470916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7400640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8104220 0x7f26d819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26dd200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26d8109870 0x7f26d819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 -- 172.21.15.37:0/2272470916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8104220 msgr2=0x7f26d819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8104220 0x7f26d819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 -- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26d8109870 msgr2=0x7f26d819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26d8109870 0x7f26d819e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 -- 172.21.15.37:0/2272470916 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d819f100 con 0x7f26d8103860 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7400640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8104220 0x7f26d819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.744+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 0x7f26d819a110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26c802efd0 tx=0x7f26c80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.745+0000 7f26d4c00640 1 -- 172.21.15.37:0/2272470916 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8041070 con 0x7f26d8103860 2024-03-21T09:14:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.745+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d819f390 con 0x7f26d8103860 2024-03-21T09:14:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.745+0000 7f26d4c00640 1 -- 172.21.15.37:0/2272470916 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26c802ccc0 con 0x7f26d8103860 2024-03-21T09:14:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.745+0000 7f26d4c00640 1 -- 172.21.15.37:0/2272470916 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8003d40 con 0x7f26d8103860 2024-03-21T09:14:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.745+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d81a6c80 con 0x7f26d8103860 2024-03-21T09:14:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.748+0000 7f26e1faf740 1 Processor -- start 2024-03-21T09:14:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.748+0000 7f26e1faf740 1 -- start start 2024-03-21T09:14:10.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 0x55b004987950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b0049883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b004993680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26e1faf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0048ec250 con 0x55b004991290 2024-03-21T09:14:10.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26e1faf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004894a30 con 0x55b004987f20 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26e1faf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0048e82f0 con 0x55b004988c00 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26dd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 0x55b004987950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26dd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 0x55b004987950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52600/0 (socket says 172.21.15.37:52600) 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26dd200640 1 -- 172.21.15.37:0/2536692555 learned_addr learned my addr 172.21.15.37:0/2536692555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26d7400640 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b004993680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26d4c00640 1 -- 172.21.15.37:0/2272470916 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f26c8044070 con 0x7f26d8103860 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.749+0000 7f26dd200640 1 -- 172.21.15.37:0/2536692555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 msgr2=0x55b0049883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26dd200640 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b0049883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26dd200640 1 -- 172.21.15.37:0/2536692555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 msgr2=0x55b004993680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26dd200640 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b004993680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26dd200640 1 -- 172.21.15.37:0/2536692555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b00487f690 con 0x55b004988c00 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26dd200640 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 0x55b004987950 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009e30 tx=0x7f26d000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07c598205c21763 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26cd600640 1 -- 172.21.15.37:0/2536692555 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0011640 con 0x55b004988c00 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26cd600640 1 -- 172.21.15.37:0/2536692555 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0011cc0 con 0x55b004988c00 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.750+0000 7f26cd600640 1 -- 172.21.15.37:0/2536692555 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d001bd00 con 0x55b004988c00 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 -- 172.21.15.37:0/2536692555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 msgr2=0x55b004987950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 0x55b004987950 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009e30 tx=0x7f26d000f730 comp rx=0 tx=0).stop 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 -- 172.21.15.37:0/2536692555 shutdown_connections 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b004993680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b0049883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2536692555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004988c00 0x55b004987950 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 -- 172.21.15.37:0/2536692555 >> 172.21.15.37:0/2536692555 conn(0x55b00494f200 msgr2=0x55b004982970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:10.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 -- 172.21.15.37:0/2536692555 shutdown_connections 2024-03-21T09:14:10.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.751+0000 7f26e1faf740 1 -- 172.21.15.37:0/2536692555 wait complete. 2024-03-21T09:14:10.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.752+0000 7f26e1faf740 1 -- start start 2024-03-21T09:14:10.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.752+0000 7f26e1faf740 1 Processor -- start 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26d4c00640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26bc0a3030 0x7f26bc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26d4c00640 1 -- 172.21.15.37:0/2272470916 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f26c80f15e0 con 0x7f26d8103860 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 0x55b0049909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00498e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004994100 con 0x55b004987f20 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0048a7e80 con 0x55b004988c00 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004994280 con 0x55b004991290 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.753+0000 7f26e1faf740 10 client.0 _ll_register_callbacks cb 0x55b0048ba130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 0x55b0049909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 0x55b0049909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46674/0 (socket says 172.21.15.37:46674) 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 -- 172.21.15.37:0/2236131328 learned_addr learned my addr 172.21.15.37:0/2236131328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26dd200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26bc0a3030 0x7f26bc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 -- 172.21.15.37:0/2236131328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 msgr2=0x55b00498e290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:14:10.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00498e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 -- 172.21.15.37:0/2236131328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 msgr2=0x55b00498dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7e00640 1 -- 172.21.15.37:0/2236131328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0046ca7d0 con 0x55b004987f20 2024-03-21T09:14:10.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.754+0000 7f26d7400640 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:10.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 0x55b0049909c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26c80bd440 tx=0x7f26c80b99e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c8b8f222a9f602 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:10.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26c6000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26c6000640 1 -- 172.21.15.37:0/2236131328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c804b040 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26dd200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26bc0a3030 0x7f26bc0a54f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26d000fc30 tx=0x7f26d001b840 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26c6000640 1 -- 172.21.15.37:0/2236131328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f26c80bba40 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26d0011cc0 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26c6000640 1 -- 172.21.15.37:0/2236131328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c80bd0a0 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26e1faf740 10 client.15936 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.755+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b0046003f0 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26c6000640 1 -- 172.21.15.37:0/2236131328 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0011cc0 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26c6000640 1 -- 172.21.15.37:0/2236131328 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f26c80f11e0 con 0x55b004987f20 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26e1faf740 10 client.15936 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26e1faf740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26e1faf740 20 client.15936 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:10.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b0049b2170 0x55b0049b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26e1faf740 4 client.15936 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:10.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0048f9790 con 0x55b0049b2170 2024-03-21T09:14:10.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.756+0000 7f26d7400640 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b0049b2170 0x55b0049b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.757+0000 7f26d7400640 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b0049b2170 0x55b0049b4590 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24594c4733d90a1b server_cookie=d44b7a0dfd49dc4c in_seq=0 out_seq=0 2024-03-21T09:14:10.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.757+0000 7f26c6000640 10 client.15936 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26c6000640 1 -- 172.21.15.37:0/2236131328 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b0048f9790 con 0x55b0049b2170 2024-03-21T09:14:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26c6000640 10 client.15936 handle_command_reply: tid=0 2024-03-21T09:14:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 client.15936 shutdown 2024-03-21T09:14:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 msgr2=0x55b0049909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 0x55b0049909c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26c80bd440 tx=0x7f26c80b99e0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 shutdown_connections 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00498e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b0049b2170 0x55b0049b4590 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2236131328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004987f20 0x55b0049909c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 >> 172.21.15.37:0/2236131328 conn(0x55b00494f200 msgr2=0x55b0049846c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.758+0000 7f26c6000640 0 client.15936 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.759+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 shutdown_connections 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.759+0000 7f26e1faf740 1 -- 172.21.15.37:0/2236131328 wait complete. 2024-03-21T09:14:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.759+0000 7f26e1faf740 20 client.15936 trim_cache size 0 max 16384 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 Processor -- start 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 -- start start 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b004990480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004988c00 0x55b0049909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00499ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81aa780 con 0x55b004988c00 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81a8280 con 0x55b004987f20 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.784+0000 7f26e1faf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8072d40 con 0x55b004991290 2024-03-21T09:14:10.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00499ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00499ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52624/0 (socket says 172.21.15.37:52624) 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 -- 172.21.15.37:0/2737204725 learned_addr learned my addr 172.21.15.37:0/2737204725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26d7e00640 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b004990480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26d7400640 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004988c00 0x55b0049909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 -- 172.21.15.37:0/2737204725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 msgr2=0x55b004990480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b004990480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 -- 172.21.15.37:0/2737204725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004988c00 msgr2=0x55b0049909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004988c00 0x55b0049909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 -- 172.21.15.37:0/2737204725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0045c00f0 con 0x55b004991290 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26dd200640 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00499ca00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f26d001b950 tx=0x7f26d0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60574016ff05f6b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.785+0000 7f26ccc00640 1 -- 172.21.15.37:0/2737204725 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0034070 con 0x55b004991290 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.786+0000 7f26ccc00640 1 -- 172.21.15.37:0/2737204725 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0003e40 con 0x55b004991290 2024-03-21T09:14:10.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.786+0000 7f26ccc00640 1 -- 172.21.15.37:0/2737204725 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d002f470 con 0x55b004991290 2024-03-21T09:14:10.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 -- 172.21.15.37:0/2737204725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 msgr2=0x55b00499ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00499ca00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f26d001b950 tx=0x7f26d0020000 comp rx=0 tx=0).stop 2024-03-21T09:14:10.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 -- 172.21.15.37:0/2737204725 shutdown_connections 2024-03-21T09:14:10.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004991290 0x55b00499ca00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004988c00 0x55b0049909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 --2- 172.21.15.37:0/2737204725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004987f20 0x55b004990480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 -- 172.21.15.37:0/2737204725 >> 172.21.15.37:0/2737204725 conn(0x55b00494f200 msgr2=0x55b0049824a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 -- 172.21.15.37:0/2737204725 shutdown_connections 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 -- 172.21.15.37:0/2737204725 wait complete. 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.787+0000 7f26e1faf740 1 -- start start 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 Processor -- start 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 0x55b00498aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b00498ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8073340 con 0x55b004991290 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81a7c20 con 0x55b004988c00 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.788+0000 7f26e1faf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81a6f80 con 0x55b004987f20 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26e1faf740 10 client.0 _ll_register_callbacks cb 0x7f26c80b9320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 0x55b00498aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26dd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b00498ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 0x55b00498aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52630/0 (socket says 172.21.15.37:52630) 2024-03-21T09:14:10.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 -- 172.21.15.37:0/278052778 learned_addr learned my addr 172.21.15.37:0/278052778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:10.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 -- 172.21.15.37:0/278052778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 msgr2=0x55b00498b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 -- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 msgr2=0x55b00498ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b00498ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.789+0000 7f26d7e00640 1 -- 172.21.15.37:0/278052778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b004991670 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26d7e00640 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 0x55b00498aff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f26c80b9ba0 tx=0x7f26c802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535d9a9d7f2ea623 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26c6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26c6000640 1 -- 172.21.15.37:0/278052778 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80da070 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26c6000640 1 -- 172.21.15.37:0/278052778 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f26c80d6570 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0048f3ae0 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26c6000640 1 -- 172.21.15.37:0/278052778 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c80a3050 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 10 client.26137 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b004a0db60 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26c6000640 1 -- 172.21.15.37:0/278052778 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b0048f3ae0 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26c6000640 1 -- 172.21.15.37:0/278052778 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f26c80bcb90 con 0x55b004987f20 2024-03-21T09:14:10.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 10 client.26137 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 20 client.26137 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b004a0e0a0 0x55b004a104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 4 client.26137 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.790+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0048713a0 con 0x55b004a0e0a0 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.791+0000 7f26d7400640 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b004a0e0a0 0x55b004a104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.791+0000 7f26d7400640 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b004a0e0a0 0x55b004a104c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c26aa334c46a073 server_cookie=2dfd7c3f01dad9af in_seq=0 out_seq=0 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.791+0000 7f26c6000640 10 client.26137 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26c6000640 1 -- 172.21.15.37:0/278052778 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b0048713a0 con 0x55b004a0e0a0 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26c6000640 10 client.26137 handle_command_reply: tid=0 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 client.26137 shutdown 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 msgr2=0x55b00498aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 0x55b00498aff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f26c80b9ba0 tx=0x7f26c802bc30 comp rx=0 tx=0).stop 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 shutdown_connections 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b004991290 0x55b00498ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b004a0e0a0 0x55b004a104c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b004988c00 0x55b00498b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 --2- 172.21.15.37:0/278052778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b004987f20 0x55b00498aff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 >> 172.21.15.37:0/278052778 conn(0x55b00494f200 msgr2=0x55b00498c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26c6000640 0 client.26137 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 shutdown_connections 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 1 -- 172.21.15.37:0/278052778 wait complete. 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.792+0000 7f26e1faf740 20 client.26137 trim_cache size 0 max 16384 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:14:10.789 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:14:10.790 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:14:10.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26bc0a3030 msgr2=0x7f26bc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26bc0a3030 0x7f26bc0a54f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26d000fc30 tx=0x7f26d001b840 comp rx=0 tx=0).stop 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 msgr2=0x7f26d819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 0x7f26d819a110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26c802efd0 tx=0x7f26c80042f0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 shutdown_connections 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26d8109870 0x7f26d819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26bc0a3030 0x7f26bc0a54f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26d8104220 0x7f26d819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 --2- 172.21.15.37:0/2272470916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d8103860 0x7f26d819a110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.793+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 >> 172.21.15.37:0/2272470916 conn(0x7f26d80ff310 msgr2=0x7f26d80ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.794+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 shutdown_connections 2024-03-21T09:14:10.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:10.794+0000 7f26df200640 1 -- 172.21.15.37:0/2272470916 wait complete. 2024-03-21T09:14:10.807 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:14:10.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:14:11.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.032+0000 7f02d4400640 1 -- 172.21.15.37:0/3468863548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc075bd0 msgr2=0x7f02cc075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.032+0000 7f02d4400640 1 --2- 172.21.15.37:0/3468863548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc075bd0 0x7f02cc075fb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009890 tx=0x7f02c002bb70 comp rx=0 tx=0).stop 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 -- 172.21.15.37:0/3468863548 shutdown_connections 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 --2- 172.21.15.37:0/3468863548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc112500 0x7f02cc1148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 --2- 172.21.15.37:0/3468863548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 --2- 172.21.15.37:0/3468863548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc075bd0 0x7f02cc075fb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 -- 172.21.15.37:0/3468863548 >> 172.21.15.37:0/3468863548 conn(0x7f02cc0ff2f0 msgr2=0x7f02cc101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 -- 172.21.15.37:0/3468863548 shutdown_connections 2024-03-21T09:14:11.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.033+0000 7f02d4400640 1 -- 172.21.15.37:0/3468863548 wait complete. 2024-03-21T09:14:11.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 Processor -- start 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 -- start start 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 0x7f02cc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 0x7f02cc1a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc118060 con 0x7f02cc075bd0 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc117ee0 con 0x7f02cc0764f0 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.034+0000 7f02d4400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc1181e0 con 0x7f02cc112500 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 0x7f02cc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 0x7f02cc1a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36288/0 (socket says 172.21.15.37:36288) 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 0x7f02cc1a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52656/0 (socket says 172.21.15.37:52656) 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 0x7f02cc19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46708/0 (socket says 172.21.15.37:46708) 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1000640 1 -- 172.21.15.37:0/3848942539 learned_addr learned my addr 172.21.15.37:0/3848942539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:11.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 -- 172.21.15.37:0/3848942539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 msgr2=0x7f02cc1a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 0x7f02cc1a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 -- 172.21.15.37:0/3848942539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 msgr2=0x7f02cc19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1a00640 1 -- 172.21.15.37:0/3848942539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc1a3580 con 0x7f02cc075bd0 2024-03-21T09:14:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d1000640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.035+0000 7f02d2400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 0x7f02cc1a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.036+0000 7f02d1a00640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 0x7f02cc19e5e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009780 tx=0x7f02c0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.036+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0041020 con 0x7f02cc075bd0 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.036+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc1a3810 con 0x7f02cc075bd0 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.036+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c002c9f0 con 0x7f02cc075bd0 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.036+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0039040 con 0x7f02cc075bd0 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.036+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc1ab100 con 0x7f02cc075bd0 2024-03-21T09:14:11.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.038+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc068970 con 0x7f02cc075bd0 2024-03-21T09:14:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.041+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f02c0044020 con 0x7f02cc075bd0 2024-03-21T09:14:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.042+0000 7f02ca000640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f02b40a3110 0x7f02b40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.043+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f02c00f1360 con 0x7f02cc075bd0 2024-03-21T09:14:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.043+0000 7f02d1000640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f02b40a3110 0x7f02b40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.044+0000 7f02d1000640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f02b40a3110 0x7f02b40a55d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02cc1017a0 tx=0x7f02b8009260 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.048+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02c004b020 con 0x7f02cc075bd0 2024-03-21T09:14:11.189 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:11.189 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:11.189 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:11.189 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:11.190 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:11.191 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-21T09:14:11.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:11.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:11.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.451+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02cc19f8e0 con 0x7f02cc075bd0 2024-03-21T09:14:11.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.454+0000 7f02ca000640 1 -- 172.21.15.37:0/3848942539 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11375 (secure 0 0 0) 0x7f02c00bdb60 con 0x7f02cc075bd0 2024-03-21T09:14:11.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:14:11.448 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:13:56.489186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:14:11.448 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-21T09:14:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.459+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f02b40a3110 msgr2=0x7f02b40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.459+0000 7f02d4400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f02b40a3110 0x7f02b40a55d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02cc1017a0 tx=0x7f02b8009260 comp rx=0 tx=0).stop 2024-03-21T09:14:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.459+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 msgr2=0x7f02cc19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.459+0000 7f02d4400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 0x7f02cc19e5e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009780 tx=0x7f02c0004430 comp rx=0 tx=0).stop 2024-03-21T09:14:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 shutdown_connections 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02cc112500 0x7f02cc1a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f02b40a3110 0x7f02b40a55d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02cc0764f0 0x7f02cc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 --2- 172.21.15.37:0/3848942539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc075bd0 0x7f02cc19e5e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 >> 172.21.15.37:0/3848942539 conn(0x7f02cc0ff2f0 msgr2=0x7f02cc113d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 shutdown_connections 2024-03-21T09:14:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.460+0000 7f02d4400640 1 -- 172.21.15.37:0/3848942539 wait complete. 2024-03-21T09:14:11.475 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:14:11.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:14:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 -- 172.21.15.37:0/4107030052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 msgr2=0x7f25941078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 --2- 172.21.15.37:0/4107030052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 0x7f25941078e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2590009cd0 tx=0x7f259002c0f0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 -- 172.21.15.37:0/4107030052 shutdown_connections 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 --2- 172.21.15.37:0/4107030052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101e20 0x7f259410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 --2- 172.21.15.37:0/4107030052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2594101460 0x7f25941018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 --2- 172.21.15.37:0/4107030052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 0x7f25941078e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.709+0000 7f259b200640 1 -- 172.21.15.37:0/4107030052 >> 172.21.15.37:0/4107030052 conn(0x7f25940fcff0 msgr2=0x7f25940ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.710+0000 7f259b200640 1 -- 172.21.15.37:0/4107030052 shutdown_connections 2024-03-21T09:14:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.710+0000 7f259b200640 1 -- 172.21.15.37:0/4107030052 wait complete. 2024-03-21T09:14:11.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 Processor -- start 2024-03-21T09:14:11.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 -- start start 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 0x7f2594197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2594101e20 0x7f25941982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 0x7f259419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594111820 con 0x7f2594101e20 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941116a0 con 0x7f2594101460 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941119a0 con 0x7f2594107500 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f2599e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 0x7f2594197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.711+0000 7f259a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 0x7f259419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 0x7f2594197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36314/0 (socket says 172.21.15.37:36314) 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 -- 172.21.15.37:0/1468283505 learned_addr learned my addr 172.21.15.37:0/1468283505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599400640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2594101e20 0x7f25941982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 -- 172.21.15.37:0/1468283505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 msgr2=0x7f259419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 0x7f259419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 -- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2594101e20 msgr2=0x7f25941982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2594101e20 0x7f25941982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 -- 172.21.15.37:0/1468283505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259419cd90 con 0x7f2594101460 2024-03-21T09:14:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.712+0000 7f2599e00640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 0x7f2594197da0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2590004af0 tx=0x7f2590032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.713+0000 7f258aa00640 1 -- 172.21.15.37:0/1468283505 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590041020 con 0x7f2594101460 2024-03-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.713+0000 7f258aa00640 1 -- 172.21.15.37:0/1468283505 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2590034cd0 con 0x7f2594101460 2024-03-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.713+0000 7f258aa00640 1 -- 172.21.15.37:0/1468283505 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590039040 con 0x7f2594101460 2024-03-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.713+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259419d020 con 0x7f2594101460 2024-03-21T09:14:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.713+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25940738d0 con 0x7f2594101460 2024-03-21T09:14:11.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.716+0000 7f258aa00640 1 -- 172.21.15.37:0/1468283505 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2590044020 con 0x7f2594101460 2024-03-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.717+0000 7f259f3f0740 1 Processor -- start 2024-03-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 -- start start 2024-03-21T09:14:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f258aa00640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f25800a3110 0x7f25800a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f258aa00640 1 -- 172.21.15.37:0/1468283505 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f25900f1560 con 0x7f2594101460 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 0x55cdecd7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7d260 0x55cdecd85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd85de0 0x55cdecd88230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdecce02c0 con 0x55cdecd7d260 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdecc88c90 con 0x55cdecd7c8b0 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259f3f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdeccdc360 con 0x55cdecd85de0 2024-03-21T09:14:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.718+0000 7f259a800640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f25800a3110 0x7f25800a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 0x55cdecd7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 0x55cdecd7cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36318/0 (socket says 172.21.15.37:36318) 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 -- 172.21.15.37:0/3216870537 learned_addr learned my addr 172.21.15.37:0/3216870537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f259a800640 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd85de0 0x55cdecd88230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599400640 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7d260 0x55cdecd85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 -- 172.21.15.37:0/3216870537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd85de0 msgr2=0x55cdecd88230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd85de0 0x55cdecd88230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 -- 172.21.15.37:0/3216870537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7d260 msgr2=0x55cdecd85810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7d260 0x55cdecd85810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 -- 172.21.15.37:0/3216870537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdeccad080 con 0x55cdecd7c8b0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f2599e00640 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 0x55cdecd7cc90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f25900bd650 tx=0x7f25900095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbe40e7a0363ba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f257f000640 1 -- 172.21.15.37:0/3216870537 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900bd020 con 0x55cdecd7c8b0 2024-03-21T09:14:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f259a800640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f25800a3110 0x7f25800a55d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f258c009580 tx=0x7f258c009a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f257f000640 1 -- 172.21.15.37:0/3216870537 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2590034920 con 0x55cdecd7c8b0 2024-03-21T09:14:11.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.719+0000 7f257f000640 1 -- 172.21.15.37:0/3216870537 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900eeb90 con 0x55cdecd7c8b0 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 -- 172.21.15.37:0/3216870537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 msgr2=0x55cdecd7cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 0x55cdecd7cc90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f25900bd650 tx=0x7f25900095c0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 -- 172.21.15.37:0/3216870537 shutdown_connections 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd85de0 0x55cdecd88230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7d260 0x55cdecd85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3216870537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7c8b0 0x55cdecd7cc90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.720+0000 7f259f3f0740 1 -- 172.21.15.37:0/3216870537 >> 172.21.15.37:0/3216870537 conn(0x55cdecd76f80 msgr2=0x55cdecd77360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.721+0000 7f259f3f0740 1 -- 172.21.15.37:0/3216870537 shutdown_connections 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.721+0000 7f259f3f0740 1 -- 172.21.15.37:0/3216870537 wait complete. 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.721+0000 7f259f3f0740 1 -- start start 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.721+0000 7f259f3f0740 1 Processor -- start 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 0x55cdecd85680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7d260 0x55cdecd82a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd85de0 0x55cdecd82f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdecd88b10 con 0x55cdecd85de0 2024-03-21T09:14:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdecc9c0d0 con 0x55cdecd7d260 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdecd88c90 con 0x55cdecd7c8b0 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259f3f0740 10 client.0 _ll_register_callbacks cb 0x55cdecca0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f2599e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 0x55cdecd85680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f2599400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7d260 0x55cdecd82a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f2599e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 0x55cdecd85680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52680/0 (socket says 172.21.15.37:52680) 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f2599e00640 1 -- 172.21.15.37:0/2653247667 learned_addr learned my addr 172.21.15.37:0/2653247667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.722+0000 7f259a800640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd85de0 0x55cdecd82f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599e00640 1 -- 172.21.15.37:0/2653247667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7d260 msgr2=0x55cdecd82a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599e00640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7d260 0x55cdecd82a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599e00640 1 -- 172.21.15.37:0/2653247667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd85de0 msgr2=0x55cdecd82f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599e00640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd85de0 0x55cdecd82f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599e00640 1 -- 172.21.15.37:0/2653247667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdec9f43f0 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599400640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7d260 0x55cdecd82a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2599e00640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 0x55cdecd85680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2590037040 tx=0x7f25900bd590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60852cd60cae668 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2573400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2573400640 1 -- 172.21.15.37:0/2653247667 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900d5070 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f2573400640 1 -- 172.21.15.37:0/2653247667 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f25900f1160 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.723+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2590034920 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f2573400640 1 -- 172.21.15.37:0/2653247667 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25900bd020 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 10 client.26155 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cdecabe7d0 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f2573400640 1 -- 172.21.15.37:0/2653247667 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2590034920 con 0x55cdecd7c8b0 2024-03-21T09:14:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f2573400640 1 -- 172.21.15.37:0/2653247667 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f259003c030 con 0x55cdecd7c8b0 2024-03-21T09:14:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 10 client.26155 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 10 client.26155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:14:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 20 client.26155 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecd95740 0x55cdecd97b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 4 client.26155 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.724+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdecced800 con 0x55cdecd95740 2024-03-21T09:14:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.725+0000 7f2599400640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecd95740 0x55cdecd97b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.726+0000 7f2599400640 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecd95740 0x55cdecd97b30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8933a881982b5236 server_cookie=3582597d2b79d0f7 in_seq=0 out_seq=0 2024-03-21T09:14:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.726+0000 7f2573400640 10 client.26155 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.727+0000 7f2573400640 1 -- 172.21.15.37:0/2653247667 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cdecced800 con 0x55cdecd95740 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.727+0000 7f2573400640 10 client.26155 handle_command_reply: tid=0 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.727+0000 7f259f3f0740 1 client.26155 shutdown 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 msgr2=0x55cdecd85680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 0x55cdecd85680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2590037040 tx=0x7f25900bd590 comp rx=0 tx=0).stop 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 shutdown_connections 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecd95740 0x55cdecd97b30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd85de0 0x55cdecd82f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd7d260 0x55cdecd82a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f2573400640 0 client.26155 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2653247667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7c8b0 0x55cdecd85680 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 >> 172.21.15.37:0/2653247667 conn(0x55cdecd76f80 msgr2=0x55cdecd78cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 shutdown_connections 2024-03-21T09:14:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 1 -- 172.21.15.37:0/2653247667 wait complete. 2024-03-21T09:14:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.728+0000 7f259f3f0740 20 client.26155 trim_cache size 0 max 16384 2024-03-21T09:14:11.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.742+0000 7f259f3f0740 1 Processor -- start 2024-03-21T09:14:11.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 -- start start 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd75fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecd85680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941aeb30 con 0x55cdecd7c8b0 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941ac630 con 0x55cdecd85de0 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259f3f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941aa6d0 con 0x55cdecd7d260 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f2599e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd75fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f2599400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f2599400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd85140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52682/0 (socket says 172.21.15.37:52682) 2024-03-21T09:14:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f2599400640 1 -- 172.21.15.37:0/2190302897 learned_addr learned my addr 172.21.15.37:0/2190302897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:11.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.743+0000 7f259a800640 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecd85680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599400640 1 -- 172.21.15.37:0/2190302897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 msgr2=0x55cdecd85680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599400640 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecd85680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599400640 1 -- 172.21.15.37:0/2190302897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 msgr2=0x55cdecd75fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599400640 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd75fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599400640 1 -- 172.21.15.37:0/2190302897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdecd728b0 con 0x55cdecd7d260 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599e00640 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd75fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f2599400640 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd85140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55cdecd7d660 tx=0x7f2584020a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4b174bdaa36e4d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.744+0000 7f257e600640 1 -- 172.21.15.37:0/2190302897 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840035f0 con 0x55cdecd7d260 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.745+0000 7f257e600640 1 -- 172.21.15.37:0/2190302897 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2584003c70 con 0x55cdecd7d260 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.745+0000 7f257e600640 1 -- 172.21.15.37:0/2190302897 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584013d70 con 0x55cdecd7d260 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 -- 172.21.15.37:0/2190302897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 msgr2=0x55cdecd85140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd85140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55cdecd7d660 tx=0x7f2584020a60 comp rx=0 tx=0).stop 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 -- 172.21.15.37:0/2190302897 shutdown_connections 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecd85680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd85140 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 --2- 172.21.15.37:0/2190302897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd75fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 -- 172.21.15.37:0/2190302897 >> 172.21.15.37:0/2190302897 conn(0x55cdecd76f80 msgr2=0x55cdecd758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 -- 172.21.15.37:0/2190302897 shutdown_connections 2024-03-21T09:14:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 -- 172.21.15.37:0/2190302897 wait complete. 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.746+0000 7f259f3f0740 1 -- start start 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 Processor -- start 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecdeff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941aacd0 con 0x55cdecd7c8b0 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941abfd0 con 0x55cdecd85de0 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.747+0000 7f259f3f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941ab310 con 0x55cdecd7d260 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f259f3f0740 10 client.0 _ll_register_callbacks cb 0x7f2590005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd89f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52688/0 (socket says 172.21.15.37:52688) 2024-03-21T09:14:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 -- 172.21.15.37:0/3514027263 learned_addr learned my addr 172.21.15.37:0/3514027263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599e00640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f259a800640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecdeff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 -- 172.21.15.37:0/3514027263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 msgr2=0x55cdecdeff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecdeff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 -- 172.21.15.37:0/3514027263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 msgr2=0x55cdecd899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.748+0000 7f2599400640 1 -- 172.21.15.37:0/3514027263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdecdf0460 con 0x55cdecd7d260 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f2599400640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd89f20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2584009460 tx=0x7f25840137e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c19ed8eec0f01789 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f2573400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f2573400640 1 -- 172.21.15.37:0/3514027263 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584013bf0 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f2573400640 1 -- 172.21.15.37:0/3514027263 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2584013d90 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2584003c70 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f2573400640 1 -- 172.21.15.37:0/3514027263 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2584039050 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f259f3f0740 10 client.26161 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cdecdf1730 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.749+0000 7f2573400640 1 -- 172.21.15.37:0/3514027263 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2584003c70 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f2573400640 1 -- 172.21.15.37:0/3514027263 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2584010700 con 0x55cdecd7d260 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f259f3f0740 10 client.26161 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f259f3f0740 10 client.26161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f259f3f0740 20 client.26161 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecdf1d80 0x55cdecdf41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f259f3f0740 4 client.26161 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:14:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cdecce7b50 con 0x55cdecdf1d80 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f2599e00640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecdf1d80 0x55cdecdf41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f2599e00640 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecdf1d80 0x55cdecdf41a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12cc70848caca8bc server_cookie=c17ae5f15134219d in_seq=0 out_seq=0 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.750+0000 7f2573400640 10 client.26161 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f2573400640 1 -- 172.21.15.37:0/3514027263 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cdecce7b50 con 0x55cdecdf1d80 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f2573400640 10 client.26161 handle_command_reply: tid=0 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 client.26161 shutdown 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 msgr2=0x55cdecd89f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd89f20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2584009460 tx=0x7f25840137e0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 shutdown_connections 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cdecd85de0 0x55cdecdeff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cdecd7d260 0x55cdecd89f20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cdecdf1d80 0x55cdecdf41a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 --2- 172.21.15.37:0/3514027263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cdecd7c8b0 0x55cdecd899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 >> 172.21.15.37:0/3514027263 conn(0x55cdecd76f80 msgr2=0x55cdecd7df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f2573400640 0 client.26161 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 shutdown_connections 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 1 -- 172.21.15.37:0/3514027263 wait complete. 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.751+0000 7f259f3f0740 20 client.26161 trim_cache size 0 max 16384 2024-03-21T09:14:11.749 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:14:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f25800a3110 msgr2=0x7f25800a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f25800a3110 0x7f25800a55d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f258c009580 tx=0x7f258c009a40 comp rx=0 tx=0).stop 2024-03-21T09:14:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 msgr2=0x7f2594197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 0x7f2594197da0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2590004af0 tx=0x7f2590032ea0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 shutdown_connections 2024-03-21T09:14:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2594107500 0x7f259419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2594101e20 0x7f25941982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f25800a3110 0x7f25800a55d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 --2- 172.21.15.37:0/1468283505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2594101460 0x7f2594197da0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 >> 172.21.15.37:0/1468283505 conn(0x7f25940fcff0 msgr2=0x7f25940fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 shutdown_connections 2024-03-21T09:14:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:11.752+0000 7f259b200640 1 -- 172.21.15.37:0/1468283505 wait complete. 2024-03-21T09:14:11.768 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:14:11.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:14:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.004+0000 7ff5a7800640 1 -- 172.21.15.37:0/4126324107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0103840 msgr2=0x7ff5a0103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.004+0000 7ff5a7800640 1 --2- 172.21.15.37:0/4126324107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0103840 0x7ff5a0103cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009930 tx=0x7ff58c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.004+0000 7ff5a7800640 1 -- 172.21.15.37:0/4126324107 shutdown_connections 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.004+0000 7ff5a7800640 1 --2- 172.21.15.37:0/4126324107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5a0104200 0x7ff5a0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.005+0000 7ff5a7800640 1 --2- 172.21.15.37:0/4126324107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0103840 0x7ff5a0103cc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.005+0000 7ff5a7800640 1 --2- 172.21.15.37:0/4126324107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0109850 0x7ff5a0109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.005+0000 7ff5a7800640 1 -- 172.21.15.37:0/4126324107 >> 172.21.15.37:0/4126324107 conn(0x7ff5a00ff2f0 msgr2=0x7ff5a0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.005+0000 7ff5a7800640 1 -- 172.21.15.37:0/4126324107 shutdown_connections 2024-03-21T09:14:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.005+0000 7ff5a7800640 1 -- 172.21.15.37:0/4126324107 wait complete. 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.006+0000 7ff5a7800640 1 Processor -- start 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.006+0000 7ff5a7800640 1 -- start start 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.006+0000 7ff5a7800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 0x7ff5a019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.006+0000 7ff5a7800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5a0104200 0x7ff5a019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a7800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0109850 0x7ff5a019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a7800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a0113cb0 con 0x7ff5a0104200 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a7800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a0113b30 con 0x7ff5a0103840 2024-03-21T09:14:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a7800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a0113e30 con 0x7ff5a0109850 2024-03-21T09:14:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 0x7ff5a019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 0x7ff5a019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36376/0 (socket says 172.21.15.37:36376) 2024-03-21T09:14:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 -- 172.21.15.37:0/132350504 learned_addr learned my addr 172.21.15.37:0/132350504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a5800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0109850 0x7ff5a019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff59fe00640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5a0104200 0x7ff5a019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 -- 172.21.15.37:0/132350504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0109850 msgr2=0x7ff5a019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0109850 0x7ff5a019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 -- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5a0104200 msgr2=0x7ff5a019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5a0104200 0x7ff5a019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.007+0000 7ff5a4e00640 1 -- 172.21.15.37:0/132350504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a019f1c0 con 0x7ff5a0103840 2024-03-21T09:14:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.008+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 0x7ff5a019a1d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff59400ebc0 tx=0x7ff59400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.009+0000 7ff59d600640 1 -- 172.21.15.37:0/132350504 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594010630 con 0x7ff5a0103840 2024-03-21T09:14:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.009+0000 7ff59d600640 1 -- 172.21.15.37:0/132350504 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff594010cb0 con 0x7ff5a0103840 2024-03-21T09:14:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.009+0000 7ff59d600640 1 -- 172.21.15.37:0/132350504 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594018bd0 con 0x7ff5a0103840 2024-03-21T09:14:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.009+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a019f4b0 con 0x7ff5a0103840 2024-03-21T09:14:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.009+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a01a6d40 con 0x7ff5a0103840 2024-03-21T09:14:12.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.013+0000 7ff5aa443740 1 Processor -- start 2024-03-21T09:14:12.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.013+0000 7ff5aa443740 1 -- start start 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ed5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed6080 0x555714ede630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714edec00 0x555714ee1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5aa443740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555714e392c0 con 0x555714ed56d0 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5aa443740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555714de1c90 con 0x555714ed6080 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5aa443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555714e35360 con 0x555714edec00 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5a5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ed5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714edec00 0x555714ee1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5a5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ed5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46796/0 (socket says 172.21.15.37:46796) 2024-03-21T09:14:12.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.014+0000 7ff5a5800640 1 -- 172.21.15.37:0/3567566353 learned_addr learned my addr 172.21.15.37:0/3567566353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff59d600640 1 -- 172.21.15.37:0/132350504 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff5940173d0 con 0x7ff5a0103840 2024-03-21T09:14:12.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed6080 0x555714ede630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff59fe00640 1 -- 172.21.15.37:0/3567566353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed6080 msgr2=0x555714ede630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff59fe00640 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed6080 0x555714ede630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff59fe00640 1 -- 172.21.15.37:0/3567566353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 msgr2=0x555714ed5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff59fe00640 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ed5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff59fe00640 1 -- 172.21.15.37:0/3567566353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555714e06080 con 0x555714edec00 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff5a5800640 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ed5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.015+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed6080 0x555714ede630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.016+0000 7ff59fe00640 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714edec00 0x555714ee1050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff58c036040 tx=0x7ff58c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5409bdecc7028a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.016+0000 7ff592000640 1 -- 172.21.15.37:0/3567566353 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c044020 con 0x555714edec00 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.016+0000 7ff592000640 1 -- 172.21.15.37:0/3567566353 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff58c02c880 con 0x555714edec00 2024-03-21T09:14:12.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.016+0000 7ff592000640 1 -- 172.21.15.37:0/3567566353 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0046c0 con 0x555714edec00 2024-03-21T09:14:12.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.017+0000 7ff5aa443740 1 -- 172.21.15.37:0/3567566353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714edec00 msgr2=0x555714ee1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.017+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714edec00 0x555714ee1050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff58c036040 tx=0x7ff58c005d50 comp rx=0 tx=0).stop 2024-03-21T09:14:12.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.017+0000 7ff59d600640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff5880a3110 0x7ff5880a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.017+0000 7ff59d600640 1 -- 172.21.15.37:0/132350504 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff5940d01a0 con 0x7ff5a0103840 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.018+0000 7ff5aa443740 1 -- 172.21.15.37:0/3567566353 shutdown_connections 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.018+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714edec00 0x555714ee1050 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.018+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed6080 0x555714ede630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.018+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3567566353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ed5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.018+0000 7ff5aa443740 1 -- 172.21.15.37:0/3567566353 >> 172.21.15.37:0/3567566353 conn(0x555714ecfda0 msgr2=0x555714ed0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.018+0000 7ff5a5800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff5880a3110 0x7ff5880a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.019+0000 7ff5a5800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff5880a3110 0x7ff5880a55d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff598009ea0 tx=0x7ff598005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.019+0000 7ff5aa443740 1 -- 172.21.15.37:0/3567566353 shutdown_connections 2024-03-21T09:14:12.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.019+0000 7ff5aa443740 1 -- 172.21.15.37:0/3567566353 wait complete. 2024-03-21T09:14:12.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.020+0000 7ff5aa443740 1 -- start start 2024-03-21T09:14:12.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.020+0000 7ff5aa443740 1 Processor -- start 2024-03-21T09:14:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ede460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714edb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714edec00 0x555714edbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555714ee1b60 con 0x555714ed56d0 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555714df50d0 con 0x555714edec00 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555714ee1ce0 con 0x555714ed6080 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714edb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5a4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ede460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff5aa443740 10 client.0 _ll_register_callbacks cb 0x555714df9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.021+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714edb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52720/0 (socket says 172.21.15.37:52720) 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 -- 172.21.15.37:0/2313038995 learned_addr learned my addr 172.21.15.37:0/2313038995 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff5a5800640 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714edec00 0x555714edbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 -- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714edec00 msgr2=0x555714edbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714edec00 0x555714edbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 -- 172.21.15.37:0/2313038995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 msgr2=0x555714ede460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ede460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 -- 172.21.15.37:0/2313038995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555714b4d3f0 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714edb7f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff58c02c170 tx=0x7ff58c02cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5a8a5650f9b40c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.022+0000 7ff586a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff586a00640 1 -- 172.21.15.37:0/2313038995 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c04f070 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff586a00640 1 -- 172.21.15.37:0/2313038995 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff58c040680 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff58c02c880 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff586a00640 1 -- 172.21.15.37:0/2313038995 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff58c05a050 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff5aa443740 10 client.26173 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555714c177d0 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.023+0000 7ff586a00640 1 -- 172.21.15.37:0/2313038995 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff58c02c880 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff586a00640 1 -- 172.21.15.37:0/2313038995 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff58c04bb60 con 0x555714ed6080 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5aa443740 10 client.26173 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5aa443740 10 client.26173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:14:12.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5aa443740 20 client.26173 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714eee7d0 0x555714ef0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5aa443740 4 client.26173 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555714e46800 con 0x555714eee7d0 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.024+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714eee7d0 0x555714ef0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.025+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714eee7d0 0x555714ef0bc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51b86e10ffa66ee server_cookie=d6307554b60fd5c4 in_seq=0 out_seq=0 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.025+0000 7ff586a00640 10 client.26173 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.025+0000 7ff586a00640 1 -- 172.21.15.37:0/2313038995 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555714e46800 con 0x555714eee7d0 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.025+0000 7ff586a00640 10 client.26173 handle_command_reply: tid=0 2024-03-21T09:14:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.025+0000 7ff5aa443740 1 client.26173 shutdown 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 msgr2=0x555714edb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714edb7f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff58c02c170 tx=0x7ff58c02cc60 comp rx=0 tx=0).stop 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 shutdown_connections 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714eee7d0 0x555714ef0bc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714edec00 0x555714edbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714edb7f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2313038995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714ed56d0 0x555714ede460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff586a00640 0 client.26173 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 >> 172.21.15.37:0/2313038995 conn(0x555714ecfda0 msgr2=0x555714ed1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 shutdown_connections 2024-03-21T09:14:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 1 -- 172.21.15.37:0/2313038995 wait complete. 2024-03-21T09:14:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.026+0000 7ff5aa443740 20 client.26173 trim_cache size 0 max 16384 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.044+0000 7ff5aa443740 1 Processor -- start 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 -- start start 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ed11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714eddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ede460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01b3160 con 0x555714edec00 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01b0c60 con 0x555714ed56d0 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5aa443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01aed00 con 0x555714ed6080 2024-03-21T09:14:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714eddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff5a4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ed11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714eddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52730/0 (socket says 172.21.15.37:52730) 2024-03-21T09:14:12.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.045+0000 7ff59fe00640 1 -- 172.21.15.37:0/3243983713 learned_addr learned my addr 172.21.15.37:0/3243983713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a5800640 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ede460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a4e00640 1 -- 172.21.15.37:0/3243983713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 msgr2=0x555714eddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714eddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a4e00640 1 -- 172.21.15.37:0/3243983713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 msgr2=0x555714ede460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ede460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a4e00640 1 -- 172.21.15.37:0/3243983713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555714ed9d40 con 0x555714ed56d0 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff59fe00640 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714eddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ed11c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5940cd4f0 tx=0x7ff5940d25f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71f3aa26a0bea18 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff591600640 1 -- 172.21.15.37:0/3243983713 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594010e60 con 0x555714ed56d0 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff591600640 1 -- 172.21.15.37:0/3243983713 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5940a4ba0 con 0x555714ed56d0 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.046+0000 7ff591600640 1 -- 172.21.15.37:0/3243983713 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5940bc0b0 con 0x555714ed56d0 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 -- 172.21.15.37:0/3243983713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 msgr2=0x555714ed11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ed11c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5940cd4f0 tx=0x7ff5940d25f0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 -- 172.21.15.37:0/3243983713 shutdown_connections 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ede460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714eddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 --2- 172.21.15.37:0/3243983713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ed11c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 -- 172.21.15.37:0/3243983713 >> 172.21.15.37:0/3243983713 conn(0x555714ecfda0 msgr2=0x555714eda820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 -- 172.21.15.37:0/3243983713 shutdown_connections 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.047+0000 7ff5aa443740 1 -- 172.21.15.37:0/3243983713 wait complete. 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.048+0000 7ff5aa443740 1 -- start start 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.048+0000 7ff5aa443740 1 Processor -- start 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ef3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714ef38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ef1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01af300 con 0x555714edec00 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01b0600 con 0x555714ed56d0 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01af940 con 0x555714ed6080 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5aa443740 10 client.0 _ll_register_callbacks cb 0x7ff598009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5a4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ef3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5a4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ef3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36422/0 (socket says 172.21.15.37:36422) 2024-03-21T09:14:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5a4e00640 1 -- 172.21.15.37:0/2363846871 learned_addr learned my addr 172.21.15.37:0/2363846871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.049+0000 7ff5a5800640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ef1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714ef38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5a4e00640 1 -- 172.21.15.37:0/2363846871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 msgr2=0x555714ef38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714ef38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5a4e00640 1 -- 172.21.15.37:0/2363846871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 msgr2=0x555714ef1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ef1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5a4e00640 1 -- 172.21.15.37:0/2363846871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555714ef2030 con 0x555714ed56d0 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714ef38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5a4e00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ef3370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff59409be90 tx=0x7ff594018670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de64f3e87201ef51 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff586a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff586a00640 1 -- 172.21.15.37:0/2363846871 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594010e60 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff586a00640 1 -- 172.21.15.37:0/2363846871 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff5940cd7e0 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.050+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5940a4ba0 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff586a00640 1 -- 172.21.15.37:0/2363846871 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5940b5020 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 10 client.26153 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555714f530b0 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff586a00640 1 -- 172.21.15.37:0/2363846871 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5940a4ba0 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff586a00640 1 -- 172.21.15.37:0/2363846871 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff5940bc0b0 con 0x555714ed56d0 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 10 client.26153 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 10 client.26153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:14:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 20 client.26153 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714f536a0 0x555714f55ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 4 client.26153 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.051+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555714e40b50 con 0x555714f536a0 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.052+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714f536a0 0x555714f55ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.052+0000 7ff59fe00640 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714f536a0 0x555714f55ac0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b92aab515c03052f server_cookie=f8d5dda4de415213 in_seq=0 out_seq=0 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.052+0000 7ff586a00640 10 client.26153 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.052+0000 7ff586a00640 1 -- 172.21.15.37:0/2363846871 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555714e40b50 con 0x555714f536a0 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.052+0000 7ff586a00640 10 client.26153 handle_command_reply: tid=0 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.052+0000 7ff5aa443740 1 client.26153 shutdown 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 msgr2=0x555714ef3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ef3370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff59409be90 tx=0x7ff594018670 comp rx=0 tx=0).stop 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 shutdown_connections 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x555714f536a0 0x555714f55ac0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555714edec00 0x555714ef1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555714ed6080 0x555714ef38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 --2- 172.21.15.37:0/2363846871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555714ed56d0 0x555714ef3370 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff586a00640 0 client.26153 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 >> 172.21.15.37:0/2363846871 conn(0x555714ecfda0 msgr2=0x555714ece620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 shutdown_connections 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 1 -- 172.21.15.37:0/2363846871 wait complete. 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.053+0000 7ff5aa443740 20 client.26153 trim_cache size 0 max 16384 2024-03-21T09:14:12.051 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:14:12.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.054+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff5880a3110 msgr2=0x7ff5880a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.054+0000 7ff5a7800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff5880a3110 0x7ff5880a55d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff598009ea0 tx=0x7ff598005f90 comp rx=0 tx=0).stop 2024-03-21T09:14:12.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.054+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 msgr2=0x7ff5a019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.054+0000 7ff5a7800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 0x7ff5a019a1d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff59400ebc0 tx=0x7ff59400c6a0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 shutdown_connections 2024-03-21T09:14:12.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a0109850 0x7ff5a019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5a0104200 0x7ff5a019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff5880a3110 0x7ff5880a55d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 --2- 172.21.15.37:0/132350504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a0103840 0x7ff5a019a1d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 >> 172.21.15.37:0/132350504 conn(0x7ff5a00ff2f0 msgr2=0x7ff5a00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 shutdown_connections 2024-03-21T09:14:12.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.055+0000 7ff5a7800640 1 -- 172.21.15.37:0/132350504 wait complete. 2024-03-21T09:14:12.068 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:14:12.068 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:14:12.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.305+0000 7f6692e00640 1 -- 172.21.15.37:0/1451403351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c1028c0 msgr2=0x7f668c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.305+0000 7f6692e00640 1 --2- 172.21.15.37:0/1451403351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c1028c0 0x7f668c102ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f667c009900 tx=0x7f667c02bcd0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 -- 172.21.15.37:0/1451403351 shutdown_connections 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 --2- 172.21.15.37:0/1451403351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c10bc90 0x7f668c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 --2- 172.21.15.37:0/1451403351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c1031e0 0x7f668c10b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 --2- 172.21.15.37:0/1451403351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c1028c0 0x7f668c102ca0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 -- 172.21.15.37:0/1451403351 >> 172.21.15.37:0/1451403351 conn(0x7f668c077750 msgr2=0x7f668c077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 -- 172.21.15.37:0/1451403351 shutdown_connections 2024-03-21T09:14:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.306+0000 7f6692e00640 1 -- 172.21.15.37:0/1451403351 wait complete. 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 Processor -- start 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 -- start start 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c1028c0 0x7f668c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 0x7f668c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c10bc90 0x7f668c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c100d00 con 0x7f668c1028c0 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c100b80 con 0x7f668c1031e0 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.307+0000 7f6692e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c100e80 con 0x7f668c10bc90 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 0x7f668c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f6690e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c10bc90 0x7f668c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 0x7f668c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36436/0 (socket says 172.21.15.37:36436) 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 -- 172.21.15.37:0/860045595 learned_addr learned my addr 172.21.15.37:0/860045595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c1028c0 0x7f668c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 -- 172.21.15.37:0/860045595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c10bc90 msgr2=0x7f668c19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c10bc90 0x7f668c19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 -- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c1028c0 msgr2=0x7f668c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c1028c0 0x7f668c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 -- 172.21.15.37:0/860045595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c19f130 con 0x7f668c1031e0 2024-03-21T09:14:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.308+0000 7f668b400640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 0x7f668c19a9f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f667800d950 tx=0x7f667800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.309+0000 7f6688c00640 1 -- 172.21.15.37:0/860045595 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678017070 con 0x7f668c1031e0 2024-03-21T09:14:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.309+0000 7f6688c00640 1 -- 172.21.15.37:0/860045595 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6678009c30 con 0x7f668c1031e0 2024-03-21T09:14:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.309+0000 7f6688c00640 1 -- 172.21.15.37:0/860045595 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800f6b0 con 0x7f668c1031e0 2024-03-21T09:14:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.309+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c19f420 con 0x7f668c1031e0 2024-03-21T09:14:12.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.309+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c1a6cb0 con 0x7f668c1031e0 2024-03-21T09:14:12.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.312+0000 7f6688c00640 1 -- 172.21.15.37:0/860045595 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6678010460 con 0x7f668c1031e0 2024-03-21T09:14:12.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.313+0000 7f6695c07740 1 Processor -- start 2024-03-21T09:14:12.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.313+0000 7f6695c07740 1 -- start start 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6695c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59227ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6695c07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a59230630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6695c07740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a59233050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6695c07740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a5918b2c0 con 0x558a59230c00 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6695c07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a59133c90 con 0x558a59228080 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6695c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a59187360 con 0x558a592276d0 2024-03-21T09:14:12.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6690e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59227ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6690e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59227ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52762/0 (socket says 172.21.15.37:52762) 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6690e00640 1 -- 172.21.15.37:0/3845768006 learned_addr learned my addr 172.21.15.37:0/3845768006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f668be00640 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a59230630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.314+0000 7f6688c00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66700a3110 0x7f66700a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 -- 172.21.15.37:0/3845768006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 msgr2=0x558a59230630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a59230630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 -- 172.21.15.37:0/3845768006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 msgr2=0x558a59233050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6688c00640 1 -- 172.21.15.37:0/860045595 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f66780cf740 con 0x7f668c1031e0 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f668b400640 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a59233050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a59233050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 -- 172.21.15.37:0/3845768006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a59158080 con 0x558a592276d0 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f668b400640 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a59233050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:12.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66700a3110 0x7f66700a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6690e00640 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59227ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6684011590 tx=0x7f6684011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0407b0dd1a7eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.315+0000 7f6681600640 1 -- 172.21.15.37:0/3845768006 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684017050 con 0x558a592276d0 2024-03-21T09:14:12.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.316+0000 7f6681600640 1 -- 172.21.15.37:0/3845768006 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f668401bc10 con 0x558a592276d0 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.316+0000 7f6681600640 1 -- 172.21.15.37:0/3845768006 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400e3b0 con 0x558a592276d0 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.316+0000 7f6695c07740 1 -- 172.21.15.37:0/3845768006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 msgr2=0x558a59227ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.316+0000 7f6695c07740 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59227ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6684011590 tx=0x7f6684011a50 comp rx=0 tx=0).stop 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.316+0000 7f6690e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66700a3110 0x7f66700a55d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6684022000 tx=0x7f6684023000 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 -- 172.21.15.37:0/3845768006 shutdown_connections 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a59233050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a59230630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 --2- 172.21.15.37:0/3845768006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59227ab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 -- 172.21.15.37:0/3845768006 >> 172.21.15.37:0/3845768006 conn(0x558a59221da0 msgr2=0x558a59222180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 -- 172.21.15.37:0/3845768006 shutdown_connections 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 -- 172.21.15.37:0/3845768006 wait complete. 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 -- start start 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.317+0000 7f6695c07740 1 Processor -- start 2024-03-21T09:14:12.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a592304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59228080 0x558a5922d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 0x558a5922dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a59233b70 con 0x558a59228080 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a591470d0 con 0x558a59230c00 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a59233cf0 con 0x558a592276d0 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6695c07740 10 client.0 _ll_register_callbacks cb 0x558a5914be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6690e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 0x558a5922dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f668be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a592304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6690e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 0x558a5922dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36464/0 (socket says 172.21.15.37:36464) 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f6690e00640 1 -- 172.21.15.37:0/464152614 learned_addr learned my addr 172.21.15.37:0/464152614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.318+0000 7f668b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59228080 0x558a5922d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6690e00640 1 -- 172.21.15.37:0/464152614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 msgr2=0x558a592304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6690e00640 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a592304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6690e00640 1 -- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59228080 msgr2=0x558a5922d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6690e00640 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59228080 0x558a5922d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6690e00640 1 -- 172.21.15.37:0/464152614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a58e9f3f0 con 0x558a59230c00 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6690e00640 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 0x558a5922dd70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f668400f470 tx=0x7f6684009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f2ef9380d315cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6676000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6676000640 1 -- 172.21.15.37:0/464152614 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668401bdc0 con 0x558a59230c00 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6676000640 1 -- 172.21.15.37:0/464152614 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f66840049e0 con 0x558a59230c00 2024-03-21T09:14:12.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.319+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f668401bc10 con 0x558a59230c00 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.320+0000 7f6676000640 1 -- 172.21.15.37:0/464152614 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6684017020 con 0x558a59230c00 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.320+0000 7f6695c07740 10 client.26165 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.320+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558a58f697d0 con 0x558a59230c00 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.320+0000 7f6676000640 1 -- 172.21.15.37:0/464152614 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f668401bc10 con 0x558a59230c00 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6676000640 1 -- 172.21.15.37:0/464152614 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f6684036030 con 0x558a59230c00 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6695c07740 10 client.26165 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6695c07740 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6695c07740 20 client.26165 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6695c07740 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a59240990 0x558a59242d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6695c07740 4 client.26165 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a59198800 con 0x558a59240990 2024-03-21T09:14:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.321+0000 7f668be00640 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a59240990 0x558a59242d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.322+0000 7f668be00640 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a59240990 0x558a59242d80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570b358d65a64349 server_cookie=bfd96decb81cdf7d in_seq=0 out_seq=0 2024-03-21T09:14:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.322+0000 7f6676000640 10 client.26165 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.323+0000 7f6676000640 1 -- 172.21.15.37:0/464152614 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558a59198800 con 0x558a59240990 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.323+0000 7f6676000640 10 client.26165 handle_command_reply: tid=0 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.323+0000 7f6695c07740 1 client.26165 shutdown 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 msgr2=0x558a5922dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 0x558a5922dd70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f668400f470 tx=0x7f6684009520 comp rx=0 tx=0).stop 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 shutdown_connections 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a59240990 0x558a59242d80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59230c00 0x558a5922dd70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59228080 0x558a5922d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 --2- 172.21.15.37:0/464152614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a592304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6676000640 0 client.26165 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.324+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 >> 172.21.15.37:0/464152614 conn(0x558a59221da0 msgr2=0x558a59223af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.325+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 shutdown_connections 2024-03-21T09:14:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.326+0000 7f6695c07740 1 -- 172.21.15.37:0/464152614 wait complete. 2024-03-21T09:14:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.326+0000 7f6695c07740 20 client.26165 trim_cache size 0 max 16384 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.346+0000 7f6695c07740 1 Processor -- start 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.346+0000 7f6695c07740 1 -- start start 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6695c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59223820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6695c07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a5922ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6695c07740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a592304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6695c07740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1b31a0 con 0x558a59230c00 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6695c07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1b0ca0 con 0x558a59228080 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6695c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1aed40 con 0x558a592276d0 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f668be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59223820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f668b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a5922ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f668be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59223820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52786/0 (socket says 172.21.15.37:52786) 2024-03-21T09:14:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f668be00640 1 -- 172.21.15.37:0/1393049008 learned_addr learned my addr 172.21.15.37:0/1393049008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.347+0000 7f6690e00640 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a592304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f668be00640 1 -- 172.21.15.37:0/1393049008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 msgr2=0x558a5922ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f668be00640 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a5922ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f668be00640 1 -- 172.21.15.37:0/1393049008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 msgr2=0x558a592304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f668be00640 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a592304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f668be00640 1 -- 172.21.15.37:0/1393049008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a592430f0 con 0x558a592276d0 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f6690e00640 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a592304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f668be00640 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59223820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f667c005ec0 tx=0x7f667c02e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ca282060f88fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.348+0000 7f6680c00640 1 -- 172.21.15.37:0/1393049008 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c048250 con 0x558a592276d0 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6680c00640 1 -- 172.21.15.37:0/1393049008 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f667c02ce00 con 0x558a592276d0 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6680c00640 1 -- 172.21.15.37:0/1393049008 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0049f0 con 0x558a592276d0 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 -- 172.21.15.37:0/1393049008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 msgr2=0x558a59223820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59223820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f667c005ec0 tx=0x7f667c02e040 comp rx=0 tx=0).stop 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 -- 172.21.15.37:0/1393049008 shutdown_connections 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a592304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a59228080 0x558a5922ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 --2- 172.21.15.37:0/1393049008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a592276d0 0x558a59223820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.349+0000 7f6695c07740 1 -- 172.21.15.37:0/1393049008 >> 172.21.15.37:0/1393049008 conn(0x558a59221da0 msgr2=0x558a59223110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.350+0000 7f6695c07740 1 -- 172.21.15.37:0/1393049008 shutdown_connections 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.350+0000 7f6695c07740 1 -- 172.21.15.37:0/1393049008 wait complete. 2024-03-21T09:14:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.350+0000 7f6695c07740 1 -- start start 2024-03-21T09:14:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.350+0000 7f6695c07740 1 Processor -- start 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 0x558a59236730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a59228080 0x558a59236c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a5929b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1af340 con 0x558a59230c00 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1b0640 con 0x558a592276d0 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1af980 con 0x558a59228080 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6695c07740 10 client.0 _ll_register_callbacks cb 0x7f6684011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 0x558a59236730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a59228080 0x558a59236c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f6690e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a5929b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 0x558a59236730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36486/0 (socket says 172.21.15.37:36486) 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a59228080 0x558a59236c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52788/0 (socket says 172.21.15.37:52788) 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 -- 172.21.15.37:0/3816116425 learned_addr learned my addr 172.21.15.37:0/3816116425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 -- 172.21.15.37:0/3816116425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a59228080 msgr2=0x558a59236c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a59228080 0x558a59236c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 -- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 msgr2=0x558a5929b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a5929b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.351+0000 7f668be00640 1 -- 172.21.15.37:0/3816116425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a5929b6b0 con 0x558a592276d0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f668be00640 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 0x558a59236730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f667c036040 tx=0x7f667c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24feae2584bc3096 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6676000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6676000640 1 -- 172.21.15.37:0/3816116425 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c054020 con 0x558a592276d0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f667c02ce00 con 0x558a592276d0 2024-03-21T09:14:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6676000640 1 -- 172.21.15.37:0/3816116425 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f667c03a040 con 0x558a592276d0 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6676000640 1 -- 172.21.15.37:0/3816116425 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f667c004470 con 0x558a592276d0 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6695c07740 10 client.26171 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558a5929c850 con 0x558a592276d0 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.352+0000 7f6676000640 1 -- 172.21.15.37:0/3816116425 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f667c02ce00 con 0x558a592276d0 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6676000640 1 -- 172.21.15.37:0/3816116425 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f667c005390 con 0x558a592276d0 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6695c07740 10 client.26171 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6695c07740 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6695c07740 20 client.26171 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6695c07740 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a5929cf50 0x558a5929f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6695c07740 4 client.26171 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a59192b50 con 0x558a5929cf50 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f668b400640 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a5929cf50 0x558a5929f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f668b400640 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a5929cf50 0x558a5929f370 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a35540cbd7a115 server_cookie=fd697ba3bac67f7b in_seq=0 out_seq=0 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.353+0000 7f6676000640 10 client.26171 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6676000640 1 -- 172.21.15.37:0/3816116425 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a59192b50 con 0x558a5929cf50 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6676000640 10 client.26171 handle_command_reply: tid=0 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 client.26171 shutdown 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 msgr2=0x558a59236730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 0x558a59236730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f667c036040 tx=0x7f667c0046c0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 shutdown_connections 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a59230c00 0x558a5929b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a59228080 0x558a59236c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558a5929cf50 0x558a5929f370 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 --2- 172.21.15.37:0/3816116425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a592276d0 0x558a59236730 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 >> 172.21.15.37:0/3816116425 conn(0x558a59221da0 msgr2=0x558a59228cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6676000640 0 client.26171 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 shutdown_connections 2024-03-21T09:14:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 1 -- 172.21.15.37:0/3816116425 wait complete. 2024-03-21T09:14:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.354+0000 7f6695c07740 20 client.26171 trim_cache size 0 max 16384 2024-03-21T09:14:12.352 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.355+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66700a3110 msgr2=0x7f66700a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.355+0000 7f6692e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66700a3110 0x7f66700a55d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6684022000 tx=0x7f6684023000 comp rx=0 tx=0).stop 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.355+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 msgr2=0x7f668c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.355+0000 7f6692e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 0x7f668c19a9f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f667800d950 tx=0x7f667800de10 comp rx=0 tx=0).stop 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 shutdown_connections 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668c10bc90 0x7f668c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66700a3110 0x7f66700a55d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f668c1031e0 0x7f668c19a9f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 --2- 172.21.15.37:0/860045595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c1028c0 0x7f668c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 >> 172.21.15.37:0/860045595 conn(0x7f668c077750 msgr2=0x7f668c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 shutdown_connections 2024-03-21T09:14:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:12.356+0000 7f6692e00640 1 -- 172.21.15.37:0/860045595 wait complete. 2024-03-21T09:14:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:12 smithi037 ceph-mon[31830]: pgmap v63: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 738 KiB/s wr, 256 op/s 2024-03-21T09:14:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:12 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3848942539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:14:12.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:12 smithi053 ceph-mon[36169]: pgmap v63: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 738 KiB/s wr, 256 op/s 2024-03-21T09:14:12.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:12 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3848942539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:14:12.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:12 smithi071 ceph-mon[36597]: pgmap v63: 129 pgs: 129 active+clean; 62 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 738 KiB/s wr, 256 op/s 2024-03-21T09:14:12.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:12 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3848942539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:14:12.785 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:12.785 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-21T09:14:12.785 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:12.785 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:12.786 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:12.787 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-21T09:14:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:12.789 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:12.791 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.792+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1894622297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 msgr2=0x7f9fdc075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.792+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1894622297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 0x7f9fdc075fb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009930 tx=0x7f9fd002bc30 comp rx=0 tx=0).stop 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.792+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1894622297 shutdown_connections 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.792+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1894622297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc112510 0x7f9fdc114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.793+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1894622297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.793+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1894622297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 0x7f9fdc075fb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.793+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1894622297 >> 172.21.15.37:0/1894622297 conn(0x7f9fdc0ff320 msgr2=0x7f9fdc101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.793+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1894622297 shutdown_connections 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.793+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1894622297 wait complete. 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.794+0000 7f9fe3e00640 1 Processor -- start 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.794+0000 7f9fe3e00640 1 -- start start 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 0x7f9fdc19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc112510 0x7f9fdc1a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1180c0 con 0x7f9fdc112510 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc117f40 con 0x7f9fdc0764f0 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc118240 con 0x7f9fdc075bd0 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc112510 0x7f9fdc1a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc19eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36492/0 (socket says 172.21.15.37:36492) 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1254354706 learned_addr learned my addr 172.21.15.37:0/1254354706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.795+0000 7f9fe1400640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 0x7f9fdc19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1254354706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 msgr2=0x7f9fdc19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 0x7f9fdc19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc112510 msgr2=0x7f9fdc1a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc112510 0x7f9fdc1a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1254354706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc1a3590 con 0x7f9fdc0764f0 2024-03-21T09:14:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc19eae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800e9f0 tx=0x7f9fc800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fda000640 1 -- 172.21.15.37:0/1254354706 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800cd80 con 0x7f9fdc0764f0 2024-03-21T09:14:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fda000640 1 -- 172.21.15.37:0/1254354706 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8010910 con 0x7f9fdc0764f0 2024-03-21T09:14:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fda000640 1 -- 172.21.15.37:0/1254354706 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8018990 con 0x7f9fdc0764f0 2024-03-21T09:14:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.796+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc1a3880 con 0x7f9fdc0764f0 2024-03-21T09:14:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.797+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc1ab110 con 0x7f9fdc0764f0 2024-03-21T09:14:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.801+0000 7f9fda000640 1 -- 172.21.15.37:0/1254354706 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9fc8018b30 con 0x7f9fdc0764f0 2024-03-21T09:14:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 Processor -- start 2024-03-21T09:14:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 -- start start 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f1183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f121180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f123ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7f0da3b0 con 0x559f7f117fd0 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7f07c830 con 0x559f7f1216c0 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.802+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7f0255d0 con 0x559f7f1188f0 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f123ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f1183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f1183b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46916/0 (socket says 172.21.15.37:46916) 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f123ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36504/0 (socket says 172.21.15.37:36504) 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe1e00640 1 -- 172.21.15.37:0/4138586943 learned_addr learned my addr 172.21.15.37:0/4138586943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe1400640 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fda000640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9fc40a3100 0x7f9fc40a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 -- 172.21.15.37:0/4138586943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 msgr2=0x559f7f121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 -- 172.21.15.37:0/4138586943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 msgr2=0x559f7f1183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f1183b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.803+0000 7f9fe0a00640 1 -- 172.21.15.37:0/4138586943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f7f03d2c0 con 0x559f7f1216c0 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.804+0000 7f9fda000640 1 -- 172.21.15.37:0/1254354706 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9fc80cfe30 con 0x7f9fdc0764f0 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.804+0000 7f9fe1e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9fc40a3100 0x7f9fc40a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.804+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f123ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80cd1e0 tx=0x7f9fc8024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463372049e389168 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.804+0000 7f9fce600640 1 -- 172.21.15.37:0/4138586943 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc809b890 con 0x559f7f1216c0 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fce600640 1 -- 172.21.15.37:0/4138586943 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc809bf30 con 0x559f7f1216c0 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fce600640 1 -- 172.21.15.37:0/4138586943 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8013190 con 0x559f7f1216c0 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/4138586943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 msgr2=0x559f7f123ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f123ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80cd1e0 tx=0x7f9fc8024000 comp rx=0 tx=0).stop 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fe1e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9fc40a3100 0x7f9fc40a55c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009fd0 tx=0x7f9fd4004a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/4138586943 shutdown_connections 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f123ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.805+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.806+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/4138586943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f1183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.806+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/4138586943 >> 172.21.15.37:0/4138586943 conn(0x559f7f1128f0 msgr2=0x559f7f112cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.806+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/4138586943 shutdown_connections 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.806+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/4138586943 wait complete. 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.806+0000 7f9fe6bd9740 1 -- start start 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.806+0000 7f9fe6bd9740 1 Processor -- start 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 0x559f7f11e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f1188f0 0x559f7f11ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f11f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7f038a20 con 0x559f7f1188f0 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7f0788d0 con 0x559f7f1216c0 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f7f124230 con 0x559f7f117fd0 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe6bd9740 10 client.0 _ll_register_callbacks cb 0x559f7f0751c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 0x559f7f11e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 0x559f7f11e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52832/0 (socket says 172.21.15.37:52832) 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe1400640 1 -- 172.21.15.37:0/2595095173 learned_addr learned my addr 172.21.15.37:0/2595095173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe1e00640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f11f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.807+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f1188f0 0x559f7f11ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fe1400640 1 -- 172.21.15.37:0/2595095173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 msgr2=0x559f7f11f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fe1400640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f11f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fe1400640 1 -- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f1188f0 msgr2=0x559f7f11ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fe1400640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f1188f0 0x559f7f11ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fe1400640 1 -- 172.21.15.37:0/2595095173 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f7ed91210 con 0x559f7f117fd0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fe1400640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 0x559f7f11e6c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006010 tx=0x7f9fd0005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee151ac89ed21dce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.808+0000 7f9fc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fc2a00640 1 -- 172.21.15.37:0/2595095173 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd002cb10 con 0x559f7f117fd0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fc2a00640 1 -- 172.21.15.37:0/2595095173 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9fd002ccb0 con 0x559f7f117fd0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fc809bf30 con 0x559f7f117fd0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fc2a00640 1 -- 172.21.15.37:0/2595095173 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd0054050 con 0x559f7f117fd0 2024-03-21T09:14:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fe6bd9740 10 client.26206 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559f7ee5d620 con 0x559f7f117fd0 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fc2a00640 1 -- 172.21.15.37:0/2595095173 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc809bf30 con 0x559f7f117fd0 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.809+0000 7f9fc2a00640 1 -- 172.21.15.37:0/2595095173 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9fd00035f0 con 0x559f7f117fd0 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe6bd9740 10 client.26206 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe6bd9740 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe6bd9740 20 client.26206 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f131610 0x559f7f133a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe6bd9740 4 client.26206 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f7f089d70 con 0x559f7f131610 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f131610 0x559f7f133a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f131610 0x559f7f133a90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d0a1676ee906b6 server_cookie=6324f46d11c41c73 in_seq=0 out_seq=0 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.810+0000 7f9fc2a00640 10 client.26206 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.811+0000 7f9fc2a00640 1 -- 172.21.15.37:0/2595095173 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f7f089d70 con 0x559f7f131610 2024-03-21T09:14:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.811+0000 7f9fc2a00640 10 client.26206 handle_command_reply: tid=0 2024-03-21T09:14:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 client.26206 shutdown 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 msgr2=0x559f7f11e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 0x559f7f11e6c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006010 tx=0x7f9fd0005e70 comp rx=0 tx=0).stop 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 shutdown_connections 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f131610 0x559f7f133a90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f1188f0 0x559f7f11ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/2595095173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f117fd0 0x559f7f11e6c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 >> 172.21.15.37:0/2595095173 conn(0x559f7f1128f0 msgr2=0x559f7f114640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fc2a00640 0 client.26206 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 shutdown_connections 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/2595095173 wait complete. 2024-03-21T09:14:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.812+0000 7f9fe6bd9740 20 client.26206 trim_cache size 0 max 16384 2024-03-21T09:14:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.828+0000 7f9fe6bd9740 1 Processor -- start 2024-03-21T09:14:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.828+0000 7f9fe6bd9740 1 -- start start 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f120df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f119940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f119e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1b7570 con 0x559f7f117fd0 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1b5070 con 0x559f7f1216c0 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1b3110 con 0x559f7f1188f0 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f119940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f119940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52836/0 (socket says 172.21.15.37:52836) 2024-03-21T09:14:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f119e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe0a00640 1 -- 172.21.15.37:0/117305278 learned_addr learned my addr 172.21.15.37:0/117305278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.829+0000 7f9fe1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe1e00640 1 -- 172.21.15.37:0/117305278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 msgr2=0x559f7f119940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe1e00640 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f119940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe1e00640 1 -- 172.21.15.37:0/117305278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 msgr2=0x559f7f120df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe1e00640 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f120df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe1e00640 1 -- 172.21.15.37:0/117305278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f7f123ed0 con 0x559f7f1216c0 2024-03-21T09:14:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f119940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fe1e00640 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f119e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4010e50 tx=0x7f9fd4013030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24dff01c409a6537 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fcdc00640 1 -- 172.21.15.37:0/117305278 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd402e070 con 0x559f7f1216c0 2024-03-21T09:14:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fcdc00640 1 -- 172.21.15.37:0/117305278 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fd401ee20 con 0x559f7f1216c0 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.830+0000 7f9fcdc00640 1 -- 172.21.15.37:0/117305278 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd401c420 con 0x559f7f1216c0 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/117305278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 msgr2=0x559f7f119e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f119e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4010e50 tx=0x7f9fd4013030 comp rx=0 tx=0).stop 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/117305278 shutdown_connections 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1216c0 0x559f7f119e80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1188f0 0x559f7f119940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/117305278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/117305278 >> 172.21.15.37:0/117305278 conn(0x559f7f1128f0 msgr2=0x559f7f112cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/117305278 shutdown_connections 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.831+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/117305278 wait complete. 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.832+0000 7f9fe6bd9740 1 -- start start 2024-03-21T09:14:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.832+0000 7f9fe6bd9740 1 Processor -- start 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f124850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 0x559f7f124d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1216c0 0x559f7f1252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1b3710 con 0x559f7f117fd0 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1b4a10 con 0x559f7f1188f0 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1b3d50 con 0x559f7f1216c0 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe6bd9740 10 client.0 _ll_register_callbacks cb 0x7f9fd0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 0x559f7f124d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1216c0 0x559f7f1252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 0x559f7f124d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36524/0 (socket says 172.21.15.37:36524) 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1862206120 learned_addr learned my addr 172.21.15.37:0/1862206120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe1400640 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f124850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1862206120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1216c0 msgr2=0x559f7f1252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1216c0 0x559f7f1252d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 msgr2=0x559f7f124850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f124850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.833+0000 7f9fe0a00640 1 -- 172.21.15.37:0/1862206120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f7f18bf60 con 0x559f7f1188f0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fe0a00640 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 0x559f7f124d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9fc809bec0 tx=0x7f9fc8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9c64ede0de6ba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fc2a00640 1 -- 172.21.15.37:0/1862206120 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80a4890 con 0x559f7f1188f0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fc2a00640 1 -- 172.21.15.37:0/1862206120 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9fc809a230 con 0x559f7f1188f0 2024-03-21T09:14:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fd401ee20 con 0x559f7f1188f0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fc2a00640 1 -- 172.21.15.37:0/1862206120 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc809bf30 con 0x559f7f1188f0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fe6bd9740 10 client.26189 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559f7f18d1d0 con 0x559f7f1188f0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.834+0000 7f9fc2a00640 1 -- 172.21.15.37:0/1862206120 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fd401ee20 con 0x559f7f1188f0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fc2a00640 1 -- 172.21.15.37:0/1862206120 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9fc8092ce0 con 0x559f7f1188f0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe6bd9740 10 client.26189 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe6bd9740 10 client.26189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe6bd9740 20 client.26189 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f18d510 0x559f7f18f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe6bd9740 4 client.26189 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f7f0840c0 con 0x559f7f18d510 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.835+0000 7f9fe1400640 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f18d510 0x559f7f18f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.836+0000 7f9fe1400640 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f18d510 0x559f7f18f900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b657e59fb09442 server_cookie=752c988b30f53b43 in_seq=0 out_seq=0 2024-03-21T09:14:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.836+0000 7f9fc2a00640 10 client.26189 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fc2a00640 1 -- 172.21.15.37:0/1862206120 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f7f0840c0 con 0x559f7f18d510 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fc2a00640 10 client.26189 handle_command_reply: tid=0 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 client.26189 shutdown 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 msgr2=0x559f7f124d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 0x559f7f124d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9fc809bec0 tx=0x7f9fc8009310 comp rx=0 tx=0).stop 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 shutdown_connections 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f7f1216c0 0x559f7f1252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559f7f18d510 0x559f7f18f900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f7f1188f0 0x559f7f124d90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 --2- 172.21.15.37:0/1862206120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f7f117fd0 0x559f7f124850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fc2a00640 0 client.26189 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 >> 172.21.15.37:0/1862206120 conn(0x559f7f1128f0 msgr2=0x559f7f119940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 shutdown_connections 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 1 -- 172.21.15.37:0/1862206120 wait complete. 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.847+0000 7f9fe6bd9740 20 client.26189 trim_cache size 0 max 16384 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:14:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9fc40a3100 msgr2=0x7f9fc40a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9fc40a3100 0x7f9fc40a55c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009fd0 tx=0x7f9fd4004a40 comp rx=0 tx=0).stop 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 msgr2=0x7f9fdc19eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc19eae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800e9f0 tx=0x7f9fc800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 shutdown_connections 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc112510 0x7f9fdc1a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9fc40a3100 0x7f9fc40a55c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fdc0764f0 0x7f9fdc19eae0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 --2- 172.21.15.37:0/1254354706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fdc075bd0 0x7f9fdc19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.848+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 >> 172.21.15.37:0/1254354706 conn(0x7f9fdc0ff320 msgr2=0x7f9fdc113b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.849+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 shutdown_connections 2024-03-21T09:14:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:12.849+0000 7f9fe3e00640 1 -- 172.21.15.37:0/1254354706 wait complete. 2024-03-21T09:14:13.374 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:14:13.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:14:13.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:14:13.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.615+0000 7f9ed2600640 1 -- 172.21.15.37:0/2806877320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1031e0 msgr2=0x7f9ecc10b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.615+0000 7f9ed2600640 1 --2- 172.21.15.37:0/2806877320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1031e0 0x7f9ecc10b6c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009900 tx=0x7f9eb802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 -- 172.21.15.37:0/2806877320 shutdown_connections 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 --2- 172.21.15.37:0/2806877320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 0x7f9ecc10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 --2- 172.21.15.37:0/2806877320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1031e0 0x7f9ecc10b6c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 --2- 172.21.15.37:0/2806877320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 -- 172.21.15.37:0/2806877320 >> 172.21.15.37:0/2806877320 conn(0x7f9ecc077750 msgr2=0x7f9ecc077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 -- 172.21.15.37:0/2806877320 shutdown_connections 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.616+0000 7f9ed2600640 1 -- 172.21.15.37:0/2806877320 wait complete. 2024-03-21T09:14:13.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.617+0000 7f9ed2600640 1 Processor -- start 2024-03-21T09:14:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 -- start start 2024-03-21T09:14:13.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ecc1031e0 0x7f9ecc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 0x7f9ecc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc111a70 con 0x7f9ecc1031e0 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1118f0 con 0x7f9ecc1028c0 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ed2600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc111bf0 con 0x7f9ecc10bc90 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36542/0 (socket says 172.21.15.37:36542) 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ecb400640 1 -- 172.21.15.37:0/3786130904 learned_addr learned my addr 172.21.15.37:0/3786130904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.618+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 0x7f9ecc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ecc1031e0 0x7f9ecc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecb400640 1 -- 172.21.15.37:0/3786130904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 msgr2=0x7f9ecc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 0x7f9ecc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecb400640 1 -- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ecc1031e0 msgr2=0x7f9ecc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ecc1031e0 0x7f9ecc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecb400640 1 -- 172.21.15.37:0/3786130904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc19f200 con 0x7f9ecc1028c0 2024-03-21T09:14:13.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 0x7f9ecc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:13.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc19a210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00e9f0 tx=0x7f9ebc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.619+0000 7f9ec3e00640 1 -- 172.21.15.37:0/3786130904 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00cd80 con 0x7f9ecc1028c0 2024-03-21T09:14:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.620+0000 7f9ec3e00640 1 -- 172.21.15.37:0/3786130904 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc010910 con 0x7f9ecc1028c0 2024-03-21T09:14:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.620+0000 7f9ec3e00640 1 -- 172.21.15.37:0/3786130904 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc018990 con 0x7f9ecc1028c0 2024-03-21T09:14:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.620+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc19f4f0 con 0x7f9ecc1028c0 2024-03-21T09:14:13.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.620+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc103660 con 0x7f9ecc1028c0 2024-03-21T09:14:13.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.623+0000 7f9ec3e00640 1 -- 172.21.15.37:0/3786130904 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9ebc010ac0 con 0x7f9ecc1028c0 2024-03-21T09:14:13.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.626+0000 7f9ed5430740 1 Processor -- start 2024-03-21T09:14:13.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.626+0000 7f9ed5430740 1 -- start start 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c164460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ed5430740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178c0c8eb0 con 0x56178c164080 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ed5430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178c071a60 con 0x56178c1649a0 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ed5430740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178c0c4f50 con 0x56178c16d580 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ec3e00640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9eb00a3110 0x7f9eb00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36552/0 (socket says 172.21.15.37:36552) 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ecb400640 1 -- 172.21.15.37:0/3611356190 learned_addr learned my addr 172.21.15.37:0/3611356190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c164460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ec3e00640 1 -- 172.21.15.37:0/3786130904 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9ebc0cfee0 con 0x7f9ecc1028c0 2024-03-21T09:14:13.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.627+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9eb00a3110 0x7f9eb00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecb400640 1 -- 172.21.15.37:0/3611356190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 msgr2=0x56178c16f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecb400640 1 -- 172.21.15.37:0/3611356190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 msgr2=0x56178c164460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c164460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecb400640 1 -- 172.21.15.37:0/3611356190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56178bea57e0 con 0x56178c1649a0 2024-03-21T09:14:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c164460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16d040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0cd2d0 tx=0x7f9ebc06ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6a9e57e3dca591 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ec0c00640 1 -- 172.21.15.37:0/3611356190 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc09bf10 con 0x56178c1649a0 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ec0c00640 1 -- 172.21.15.37:0/3611356190 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc0a0060 con 0x56178c1649a0 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.628+0000 7f9ec0c00640 1 -- 172.21.15.37:0/3611356190 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc066b40 con 0x56178c1649a0 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.629+0000 7f9ed5430740 1 -- 172.21.15.37:0/3611356190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 msgr2=0x56178c16d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.629+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9eb00a3110 0x7f9eb00a55d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4005c40 tx=0x7f9ec4009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.629+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16d040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0cd2d0 tx=0x7f9ebc06ecb0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 -- 172.21.15.37:0/3611356190 shutdown_connections 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16d040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3611356190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c164460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 -- 172.21.15.37:0/3611356190 >> 172.21.15.37:0/3611356190 conn(0x56178c15e7b0 msgr2=0x56178c15eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 -- 172.21.15.37:0/3611356190 shutdown_connections 2024-03-21T09:14:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 -- 172.21.15.37:0/3611356190 wait complete. 2024-03-21T09:14:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.630+0000 7f9ed5430740 1 -- start start 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.631+0000 7f9ed5430740 1 Processor -- start 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.631+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 0x56178c16cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.631+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 0x56178c16a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.631+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c16d580 0x56178c16aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ed5430740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178c170370 con 0x56178c16d580 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ed5430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178c084eb0 con 0x56178c164080 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ed5430740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178c1704f0 con 0x56178c1649a0 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ed5430740 10 client.0 _ll_register_callbacks cb 0x56178c079150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 0x56178c16a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 0x56178c16cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 0x56178c16cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36558/0 (socket says 172.21.15.37:36558) 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 -- 172.21.15.37:0/2612930068 learned_addr learned my addr 172.21.15.37:0/2612930068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 0x56178c16a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52882/0 (socket says 172.21.15.37:52882) 2024-03-21T09:14:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c16d580 0x56178c16aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 -- 172.21.15.37:0/2612930068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 msgr2=0x56178c16a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 0x56178c16a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 -- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c16d580 msgr2=0x56178c16aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c16d580 0x56178c16aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecb400640 1 -- 172.21.15.37:0/2612930068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56178bf05f70 con 0x56178c164080 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.632+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 0x56178c16a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9ecb400640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 0x56178c16cfd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0d0210 tx=0x7f9ebc0cd1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ad808956eee3c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9eb5600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9eb5600640 1 -- 172.21.15.37:0/2612930068 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0a4920 con 0x56178c164080 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9eb5600640 1 -- 172.21.15.37:0/2612930068 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9ebc098580 con 0x56178c164080 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ebc0a0060 con 0x56178c164080 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9eb5600640 1 -- 172.21.15.37:0/2612930068 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ebc0b5020 con 0x56178c164080 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9ed5430740 10 client.26201 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56178bd9d170 con 0x56178c164080 2024-03-21T09:14:13.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.633+0000 7f9eb5600640 1 -- 172.21.15.37:0/2612930068 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc0a0060 con 0x56178c164080 2024-03-21T09:14:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9eb5600640 1 -- 172.21.15.37:0/2612930068 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9ebc09b360 con 0x56178c164080 2024-03-21T09:14:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ed5430740 10 client.26201 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ed5430740 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ed5430740 20 client.26201 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ed5430740 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c17d540 0x56178c17f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ed5430740 4 client.26201 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56178c0d63f0 con 0x56178c17d540 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.634+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c17d540 0x56178c17f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.635+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c17d540 0x56178c17f930 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff80ea06d322767 server_cookie=af584ec05fd60632 in_seq=0 out_seq=0 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.635+0000 7f9eb5600640 10 client.26201 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.635+0000 7f9eb5600640 1 -- 172.21.15.37:0/2612930068 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56178c0d63f0 con 0x56178c17d540 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.635+0000 7f9eb5600640 10 client.26201 handle_command_reply: tid=0 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.635+0000 7f9ed5430740 1 client.26201 shutdown 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 msgr2=0x56178c16cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 0x56178c16cfd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0d0210 tx=0x7f9ebc0cd1e0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 shutdown_connections 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c16d580 0x56178c16aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c1649a0 0x56178c16a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c17d540 0x56178c17f930 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 --2- 172.21.15.37:0/2612930068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c164080 0x56178c16cfd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 >> 172.21.15.37:0/2612930068 conn(0x56178c15e7b0 msgr2=0x56178c160500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9eb5600640 0 client.26201 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 shutdown_connections 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 1 -- 172.21.15.37:0/2612930068 wait complete. 2024-03-21T09:14:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.636+0000 7f9ed5430740 20 client.26201 trim_cache size 0 max 16384 2024-03-21T09:14:13.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.663+0000 7f9ed5430740 1 Processor -- start 2024-03-21T09:14:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.663+0000 7f9ed5430740 1 -- start start 2024-03-21T09:14:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c178e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ed5430740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1b7590 con 0x56178c164080 2024-03-21T09:14:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ed5430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1b5090 con 0x56178c1649a0 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ed5430740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1b3130 con 0x56178c16d580 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36562/0 (socket says 172.21.15.37:36562) 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 -- 172.21.15.37:0/1466619753 learned_addr learned my addr 172.21.15.37:0/1466619753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecb400640 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c178e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 -- 172.21.15.37:0/1466619753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 msgr2=0x56178c16cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 -- 172.21.15.37:0/1466619753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 msgr2=0x56178c178e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.664+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c178e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9ecaa00640 1 -- 172.21.15.37:0/1466619753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56178c16fce0 con 0x56178c1649a0 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9ecb400640 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c178e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16ca90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009230 tx=0x7f9eb802c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe195adf6cd879e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9eb7e00640 1 -- 172.21.15.37:0/1466619753 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb802c920 con 0x56178c1649a0 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9eb7e00640 1 -- 172.21.15.37:0/1466619753 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9eb8005040 con 0x56178c1649a0 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.665+0000 7f9eb7e00640 1 -- 172.21.15.37:0/1466619753 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8004c80 con 0x56178c1649a0 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.666+0000 7f9ed5430740 1 -- 172.21.15.37:0/1466619753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 msgr2=0x56178c16ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.666+0000 7f9ed5430740 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16ca90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009230 tx=0x7f9eb802c790 comp rx=0 tx=0).stop 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 -- 172.21.15.37:0/1466619753 shutdown_connections 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c16cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c16ca90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 --2- 172.21.15.37:0/1466619753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c178e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 -- 172.21.15.37:0/1466619753 >> 172.21.15.37:0/1466619753 conn(0x56178c15e7b0 msgr2=0x56178c178740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 -- 172.21.15.37:0/1466619753 shutdown_connections 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 -- 172.21.15.37:0/1466619753 wait complete. 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 -- start start 2024-03-21T09:14:13.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.667+0000 7f9ed5430740 1 Processor -- start 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c1826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c180a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c180f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1b3730 con 0x56178c164080 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1b4a30 con 0x56178c1649a0 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1b3d70 con 0x56178c16d580 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ed5430740 10 client.0 _ll_register_callbacks cb 0x7f9ebc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ecbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c180f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c180a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ecbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c180f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:52902/0 (socket says 172.21.15.37:52902) 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ecaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c180a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36566/0 (socket says 172.21.15.37:36566) 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ecbe00640 1 -- 172.21.15.37:0/3728781273 learned_addr learned my addr 172.21.15.37:0/3728781273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.668+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c1826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecbe00640 1 -- 172.21.15.37:0/3728781273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 msgr2=0x56178c180a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c180a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecbe00640 1 -- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 msgr2=0x56178c1826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c1826a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecbe00640 1 -- 172.21.15.37:0/3728781273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56178c1817b0 con 0x56178c16d580 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c1826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecaa00640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c180a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ecbe00640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c180f80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9ec401c690 tx=0x7f9ec4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15711f8795d5cb50 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9eb5600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9eb5600640 1 -- 172.21.15.37:0/3728781273 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec402f060 con 0x56178c16d580 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9eb5600640 1 -- 172.21.15.37:0/3728781273 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9ec402a680 con 0x56178c16d580 2024-03-21T09:14:13.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.669+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9eb8005040 con 0x56178c16d580 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9eb5600640 1 -- 172.21.15.37:0/3728781273 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ec4036050 con 0x56178c16d580 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 10 client.26230 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56178c1d9ad0 con 0x56178c16d580 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9eb5600640 1 -- 172.21.15.37:0/3728781273 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9eb8005040 con 0x56178c16d580 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9eb5600640 1 -- 172.21.15.37:0/3728781273 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9ec400ea30 con 0x56178c16d580 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 10 client.26230 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 10 client.26230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 20 client.26230 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c1da290 0x56178c1dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 4 client.26230 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.670+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56178c0d0740 con 0x56178c1da290 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.671+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c1da290 0x56178c1dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.671+0000 7f9ecb400640 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c1da290 0x56178c1dc6b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a6b1be7d25ea6d server_cookie=39bc22abf620c83d in_seq=0 out_seq=0 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.671+0000 7f9eb5600640 10 client.26230 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.673+0000 7f9eb5600640 1 -- 172.21.15.37:0/3728781273 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56178c0d0740 con 0x56178c1da290 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.673+0000 7f9eb5600640 10 client.26230 handle_command_reply: tid=0 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.673+0000 7f9ed5430740 1 client.26230 shutdown 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 msgr2=0x56178c180f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c180f80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9ec401c690 tx=0x7f9ec4004a40 comp rx=0 tx=0).stop 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 shutdown_connections 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56178c16d580 0x56178c180f80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56178c1da290 0x56178c1dc6b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56178c1649a0 0x56178c180a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 --2- 172.21.15.37:0/3728781273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56178c164080 0x56178c1826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 >> 172.21.15.37:0/3728781273 conn(0x56178c15e7b0 msgr2=0x56178c178fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.674+0000 7f9eb5600640 0 client.26230 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.675+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 shutdown_connections 2024-03-21T09:14:13.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.675+0000 7f9ed5430740 1 -- 172.21.15.37:0/3728781273 wait complete. 2024-03-21T09:14:13.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.675+0000 7f9ed5430740 20 client.26230 trim_cache size 0 max 16384 2024-03-21T09:14:13.672 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:14:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:14:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "7c31e7a4-7d7a-4d34-bcee-327c5e9f5437", 2024-03-21T09:14:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:14:13.673 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:14:13.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9eb00a3110 msgr2=0x7f9eb00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9eb00a3110 0x7f9eb00a55d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4005c40 tx=0x7f9ec4009500 comp rx=0 tx=0).stop 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 msgr2=0x7f9ecc19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc19a210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00e9f0 tx=0x7f9ebc00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 shutdown_connections 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ecc10bc90 0x7f9ecc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ecc1031e0 0x7f9ecc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9eb00a3110 0x7f9eb00a55d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 --2- 172.21.15.37:0/3786130904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ecc1028c0 0x7f9ecc19a210 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.676+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 >> 172.21.15.37:0/3786130904 conn(0x7f9ecc077750 msgr2=0x7f9ecc107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.677+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 shutdown_connections 2024-03-21T09:14:13.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.677+0000 7f9ed2600640 1 -- 172.21.15.37:0/3786130904 wait complete. 2024-03-21T09:14:13.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:14:13.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 -- 172.21.15.37:0/4084145221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 msgr2=0x7f9be8073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 --2- 172.21.15.37:0/4084145221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be8073e50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009900 tx=0x7f9bd802bcc0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 -- 172.21.15.37:0/4084145221 shutdown_connections 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 --2- 172.21.15.37:0/4084145221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be810dec0 0x7f9be81102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 --2- 172.21.15.37:0/4084145221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 0x7f9be810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 --2- 172.21.15.37:0/4084145221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be8073e50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 -- 172.21.15.37:0/4084145221 >> 172.21.15.37:0/4084145221 conn(0x7f9be80fd110 msgr2=0x7f9be80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 -- 172.21.15.37:0/4084145221 shutdown_connections 2024-03-21T09:14:13.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.926+0000 7f9bede00640 1 -- 172.21.15.37:0/4084145221 wait complete. 2024-03-21T09:14:13.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.928+0000 7f9bede00640 1 Processor -- start 2024-03-21T09:14:13.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.928+0000 7f9bede00640 1 -- start start 2024-03-21T09:14:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9bede00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9bede00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 0x7f9be819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9bede00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be810dec0 0x7f9be819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9bede00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8113c90 con 0x7f9be810dec0 2024-03-21T09:14:13.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9bede00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8113b10 con 0x7f9be8074390 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9bede00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8113e10 con 0x7f9be8073a70 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9be6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9be6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35224/0 (socket says 172.21.15.37:35224) 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9be6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 0x7f9be819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9be6a00640 1 -- 172.21.15.37:0/2873750215 learned_addr learned my addr 172.21.15.37:0/2873750215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.929+0000 7f9be7400640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be810dec0 0x7f9be819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6a00640 1 -- 172.21.15.37:0/2873750215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 msgr2=0x7f9be819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6a00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 0x7f9be819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6a00640 1 -- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be810dec0 msgr2=0x7f9be819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6a00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be810dec0 0x7f9be819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6a00640 1 -- 172.21.15.37:0/2873750215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be819f240 con 0x7f9be8073a70 2024-03-21T09:14:13.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6000640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 0x7f9be819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:13.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9be6a00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be819a250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8000c00 tx=0x7f9bd80043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.930+0000 7f9beca00640 1 -- 172.21.15.37:0/2873750215 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8042020 con 0x7f9be8073a70 2024-03-21T09:14:13.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.931+0000 7f9beca00640 1 -- 172.21.15.37:0/2873750215 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd8005000 con 0x7f9be8073a70 2024-03-21T09:14:13.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.931+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be819f4d0 con 0x7f9be8073a70 2024-03-21T09:14:13.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.931+0000 7f9beca00640 1 -- 172.21.15.37:0/2873750215 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8003b20 con 0x7f9be8073a70 2024-03-21T09:14:13.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.931+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be81a6dc0 con 0x7f9be8073a70 2024-03-21T09:14:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.935+0000 7f9bf0b78740 1 Processor -- start 2024-03-21T09:14:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.935+0000 7f9beca00640 1 -- 172.21.15.37:0/2873750215 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9bd8045020 con 0x7f9be8073a70 2024-03-21T09:14:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 -- start start 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d56950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d62680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6cbb250 con 0x5606a6d60290 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6c63a30 con 0x5606a6d57c00 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6cb72f0 con 0x5606a6d56f20 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9be7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d56950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9be7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d56950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57606/0 (socket says 172.21.15.37:57606) 2024-03-21T09:14:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9be7400640 1 -- 172.21.15.37:0/4290595300 learned_addr learned my addr 172.21.15.37:0/4290595300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.936+0000 7f9be6a00640 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be7400640 1 -- 172.21.15.37:0/4290595300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 msgr2=0x5606a6d573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be6000640 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d62680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be7400640 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be7400640 1 -- 172.21.15.37:0/4290595300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 msgr2=0x5606a6d62680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be7400640 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d62680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be7400640 1 -- 172.21.15.37:0/4290595300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a6c4e690 con 0x5606a6d57c00 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9be7400640 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d56950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009f30 tx=0x7f9be00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f6a27a18712789 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9beca00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9bcc0a3110 0x7f9bcc0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9beca00640 1 -- 172.21.15.37:0/2873750215 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9bd80f1770 con 0x7f9be8073a70 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.937+0000 7f9bd3e00640 1 -- 172.21.15.37:0/4290595300 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be001e070 con 0x5606a6d57c00 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bd3e00640 1 -- 172.21.15.37:0/4290595300 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be0003c30 con 0x5606a6d57c00 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bd3e00640 1 -- 172.21.15.37:0/4290595300 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000f8b0 con 0x5606a6d57c00 2024-03-21T09:14:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9be6a00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9bcc0a3110 0x7f9bcc0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 -- 172.21.15.37:0/4290595300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 msgr2=0x5606a6d56950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9be6a00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9bcc0a3110 0x7f9bcc0a55d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5606a6d58150 tx=0x7f9bd80f7530 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d56950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009f30 tx=0x7f9be00118d0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 -- 172.21.15.37:0/4290595300 shutdown_connections 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d62680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/4290595300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d56950 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.938+0000 7f9bf0b78740 1 -- 172.21.15.37:0/4290595300 >> 172.21.15.37:0/4290595300 conn(0x5606a6d1e200 msgr2=0x5606a6d51970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.939+0000 7f9bf0b78740 1 -- 172.21.15.37:0/4290595300 shutdown_connections 2024-03-21T09:14:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.939+0000 7f9bf0b78740 1 -- 172.21.15.37:0/4290595300 wait complete. 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 -- start start 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 Processor -- start 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d56f20 0x5606a6d5f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 0x5606a6d5cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d5d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6d63100 con 0x5606a6d56f20 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6c76e80 con 0x5606a6d60290 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.940+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6d63280 con 0x5606a6d57c00 2024-03-21T09:14:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9bf0b78740 10 client.0 _ll_register_callbacks cb 0x5606a6c89130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 0x5606a6d5cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 0x5606a6d5cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35238/0 (socket says 172.21.15.37:35238) 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 -- 172.21.15.37:0/1714076956 learned_addr learned my addr 172.21.15.37:0/1714076956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be7400640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d56f20 0x5606a6d5f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be7400640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 -- 172.21.15.37:0/1714076956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 msgr2=0x5606a6d5d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d5d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 -- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d56f20 msgr2=0x5606a6d5f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d56f20 0x5606a6d5f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.941+0000 7f9be6000640 1 -- 172.21.15.37:0/1714076956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606a6a997d0 con 0x5606a6d57c00 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9be6000640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 0x5606a6d5cd40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4011690 tx=0x7f9bd4011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471ef4f6664e424e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bd0c00640 1 -- 172.21.15.37:0/1714076956 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4018070 con 0x5606a6d57c00 2024-03-21T09:14:13.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bd0c00640 1 -- 172.21.15.37:0/1714076956 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9bd400fb40 con 0x5606a6d57c00 2024-03-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9be0003c30 con 0x5606a6d57c00 2024-03-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bd0c00640 1 -- 172.21.15.37:0/1714076956 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd4027050 con 0x5606a6d57c00 2024-03-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bf0b78740 10 client.26242 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5606a69cf3f0 con 0x5606a6d57c00 2024-03-21T09:14:13.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.942+0000 7f9bd0c00640 1 -- 172.21.15.37:0/1714076956 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be0003c30 con 0x5606a6d57c00 2024-03-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bd0c00640 1 -- 172.21.15.37:0/1714076956 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9bd401c400 con 0x5606a6d57c00 2024-03-21T09:14:13.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bf0b78740 10 client.26242 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bf0b78740 10 client.26242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bf0b78740 20 client.26242 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6d70280 0x5606a6d72670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bf0b78740 4 client.26242 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606a6cc8790 con 0x5606a6d70280 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.943+0000 7f9be7400640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6d70280 0x5606a6d72670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.944+0000 7f9be7400640 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6d70280 0x5606a6d72670 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0448af1e71a155 server_cookie=6d10939402511ce6 in_seq=0 out_seq=0 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.944+0000 7f9bd0c00640 10 client.26242 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bd0c00640 1 -- 172.21.15.37:0/1714076956 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5606a6cc8790 con 0x5606a6d70280 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bd0c00640 10 client.26242 handle_command_reply: tid=0 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 client.26242 shutdown 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 msgr2=0x5606a6d5cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 0x5606a6d5cd40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4011690 tx=0x7f9bd4011b50 comp rx=0 tx=0).stop 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 shutdown_connections 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6d70280 0x5606a6d72670 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d57c00 0x5606a6d5cd40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/1714076956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d56f20 0x5606a6d5f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bd0c00640 0 client.26242 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 >> 172.21.15.37:0/1714076956 conn(0x5606a6d1e200 msgr2=0x5606a6d536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 shutdown_connections 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.945+0000 7f9bf0b78740 1 -- 172.21.15.37:0/1714076956 wait complete. 2024-03-21T09:14:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.946+0000 7f9bf0b78740 20 client.26242 trim_cache size 0 max 16384 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.968+0000 7f9bf0b78740 1 Processor -- start 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.968+0000 7f9bf0b78740 1 -- start start 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d5f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81b3150 con 0x5606a6d60290 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81b0c50 con 0x5606a6d57c00 2024-03-21T09:14:13.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81aecf0 con 0x5606a6d56f20 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9be6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d5f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9be6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d5f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:57626/0 (socket says 172.21.15.37:57626) 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9be6000640 1 -- 172.21.15.37:0/2349886449 learned_addr learned my addr 172.21.15.37:0/2349886449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9be6000640 1 -- 172.21.15.37:0/2349886449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 msgr2=0x5606a6d5f470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9be6000640 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5f470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.969+0000 7f9be6000640 1 -- 172.21.15.37:0/2349886449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 msgr2=0x5606a6d61680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9be7400640 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9be6000640 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d61680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9be6000640 1 -- 172.21.15.37:0/2349886449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a698f0f0 con 0x5606a6d57c00 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9be7400640 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:13.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9be6000640 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d5f9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9bd401c820 tx=0x7f9bd40058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283093bc79a4d683 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9bd3400640 1 -- 172.21.15.37:0/2349886449 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4031020 con 0x5606a6d57c00 2024-03-21T09:14:13.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9bd3400640 1 -- 172.21.15.37:0/2349886449 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd400be60 con 0x5606a6d57c00 2024-03-21T09:14:13.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.970+0000 7f9bd3400640 1 -- 172.21.15.37:0/2349886449 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4018040 con 0x5606a6d57c00 2024-03-21T09:14:13.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2349886449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 msgr2=0x5606a6d5f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d5f9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9bd401c820 tx=0x7f9bd40058f0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2349886449 shutdown_connections 2024-03-21T09:14:13.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d60290 0x5606a6d61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d57c00 0x5606a6d5f9b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2349886449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2349886449 >> 172.21.15.37:0/2349886449 conn(0x5606a6d1e200 msgr2=0x5606a6d514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2349886449 shutdown_connections 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.971+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2349886449 wait complete. 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 -- start start 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 Processor -- start 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d57c00 0x5606a6d5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d60c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81af2f0 con 0x5606a6d57c00 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81b05f0 con 0x5606a6d60290 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.972+0000 7f9bf0b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81af930 con 0x5606a6d56f20 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9bf0b78740 10 client.0 _ll_register_callbacks cb 0x7f9bd400ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:13.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35260/0 (socket says 172.21.15.37:35260) 2024-03-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 -- 172.21.15.37:0/2472981076 learned_addr learned my addr 172.21.15.37:0/2472981076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d60c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be6000640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d57c00 0x5606a6d5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 -- 172.21.15.37:0/2472981076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 msgr2=0x5606a6d60c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d60c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 -- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d57c00 msgr2=0x5606a6d5a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d57c00 0x5606a6d5a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.973+0000 7f9be7400640 1 -- 172.21.15.37:0/2472981076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606a6d61200 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9be7400640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5a290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9be001c040 tx=0x7f9be0004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ee5e9e8752cc70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bd0c00640 1 -- 172.21.15.37:0/2472981076 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be001e070 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd400be60 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bd0c00640 1 -- 172.21.15.37:0/2472981076 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9be0003970 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bd0c00640 1 -- 172.21.15.37:0/2472981076 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be0003b90 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bf0b78740 10 client.26245 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.974+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5606a6dcc2c0 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bd0c00640 1 -- 172.21.15.37:0/2472981076 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd400be60 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bd0c00640 1 -- 172.21.15.37:0/2472981076 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9be00041b0 con 0x5606a6d56f20 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bf0b78740 10 client.26245 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bf0b78740 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bf0b78740 20 client.26245 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:13.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6dcc7a0 0x5606a6dceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bf0b78740 4 client.26245 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5606a6cc2ae0 con 0x5606a6dcc7a0 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.975+0000 7f9be6000640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6dcc7a0 0x5606a6dceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9be6000640 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6dcc7a0 0x5606a6dceb90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2304ccdd946b394 server_cookie=1934a83d3b5075c9 in_seq=0 out_seq=0 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bd0c00640 10 client.26245 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bd0c00640 1 -- 172.21.15.37:0/2472981076 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5606a6cc2ae0 con 0x5606a6dcc7a0 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bd0c00640 10 client.26245 handle_command_reply: tid=0 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bf0b78740 1 client.26245 shutdown 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 msgr2=0x5606a6d5a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5a290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9be001c040 tx=0x7f9be0004930 comp rx=0 tx=0).stop 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.976+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 shutdown_connections 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5606a6dcc7a0 0x5606a6dceb90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606a6d60290 0x5606a6d60c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606a6d57c00 0x5606a6d5a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bd0c00640 0 client.26245 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 --2- 172.21.15.37:0/2472981076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606a6d56f20 0x5606a6d5a290 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 >> 172.21.15.37:0/2472981076 conn(0x5606a6d1e200 msgr2=0x5606a6d5b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 shutdown_connections 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 1 -- 172.21.15.37:0/2472981076 wait complete. 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.977+0000 7f9bf0b78740 20 client.26245 trim_cache size 0 max 16384 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:14:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "7c31e7a4-7d7a-4d34-bcee-327c5e9f5437": { 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "7c31e7a4-7d7a-4d34-bcee-327c5e9f5437", 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:14:13.974 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9bcc0a3110 msgr2=0x7f9bcc0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9bcc0a3110 0x7f9bcc0a55d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5606a6d58150 tx=0x7f9bd80f7530 comp rx=0 tx=0).stop 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 msgr2=0x7f9be819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be819a250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8000c00 tx=0x7f9bd80043c0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 shutdown_connections 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be810dec0 0x7f9be819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9bcc0a3110 0x7f9bcc0a55d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be8074390 0x7f9be819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 --2- 172.21.15.37:0/2873750215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be8073a70 0x7f9be819a250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:13.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 >> 172.21.15.37:0/2873750215 conn(0x7f9be80fd110 msgr2=0x7f9be80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 shutdown_connections 2024-03-21T09:14:13.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:13.978+0000 7f9bede00640 1 -- 172.21.15.37:0/2873750215 wait complete. 2024-03-21T09:14:13.989 INFO:tasks.cephfs.filesystem:scrub status for tag:7c31e7a4-7d7a-4d34-bcee-327c5e9f5437 - {'path': '/', 'tag': '7c31e7a4-7d7a-4d34-bcee-327c5e9f5437', 'options': 'recursive,force'} 2024-03-21T09:14:14.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:14 smithi037 ceph-mon[31830]: pgmap v64: 129 pgs: 129 active+clean; 67 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 992 KiB/s wr, 334 op/s 2024-03-21T09:14:14.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:14 smithi053 ceph-mon[36169]: pgmap v64: 129 pgs: 129 active+clean; 67 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 992 KiB/s wr, 334 op/s 2024-03-21T09:14:14.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:14 smithi071 ceph-mon[36597]: pgmap v64: 129 pgs: 129 active+clean; 67 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 992 KiB/s wr, 334 op/s 2024-03-21T09:14:14.844 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:14.845 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:14.847 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:14.848 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-21T09:14:14.850 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:14.850 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:14.850 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:14.850 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-21T09:14:15.342 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:15.342 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:15.342 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:15.343 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:15.344 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:15.345 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:15.347 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:15.348 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:15.348 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-21T09:14:15.348 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:15.348 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:15 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:14:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:15 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:15 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:14:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:15 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:14:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:15 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:15 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:15 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:14:15.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:15 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:15 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:14:15.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:15 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:14:15.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:15 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:15 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:15 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:14:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:15 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:15 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:14:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:15 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:14:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:15 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:15.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:15 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:16.004 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:16.004 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-21T09:14:16.005 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:16.007 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-21T09:14:16.008 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:16.009 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:16.010 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:16.011 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-21T09:14:16.011 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:16.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:16 smithi053 ceph-mon[36169]: pgmap v65: 129 pgs: 129 active+clean; 67 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 958 KiB/s wr, 229 op/s 2024-03-21T09:14:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:16 smithi071 ceph-mon[36597]: pgmap v65: 129 pgs: 129 active+clean; 67 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 958 KiB/s wr, 229 op/s 2024-03-21T09:14:16.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:16 smithi037 ceph-mon[31830]: pgmap v65: 129 pgs: 129 active+clean; 67 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 958 KiB/s wr, 229 op/s 2024-03-21T09:14:18.193 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:18.194 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:18.195 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:18.196 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-21T09:14:18.197 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-21T09:14:18.198 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-21T09:14:18.199 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:18.474 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:18 smithi037 ceph-mon[31830]: pgmap v66: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 1.2 MiB/s wr, 319 op/s 2024-03-21T09:14:18.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:18 smithi071 ceph-mon[36597]: pgmap v66: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 1.2 MiB/s wr, 319 op/s 2024-03-21T09:14:18.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:18 smithi053 ceph-mon[36169]: pgmap v66: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 1.2 MiB/s wr, 319 op/s 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-21T09:14:19.888 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:19.889 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-21T09:14:19.890 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:19.891 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:19.892 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:19.893 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:19.894 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:20.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:20.399 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:20.400 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-21T09:14:20.401 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:20.402 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:20 smithi071 ceph-mon[36597]: pgmap v67: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 527 KiB/s wr, 167 op/s 2024-03-21T09:14:20.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:20 smithi037 ceph-mon[31830]: pgmap v67: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 527 KiB/s wr, 167 op/s 2024-03-21T09:14:20.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:20 smithi053 ceph-mon[36169]: pgmap v67: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 527 KiB/s wr, 167 op/s 2024-03-21T09:14:21.561 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-21T09:14:21.561 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:21.561 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:21.561 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-21T09:14:21.562 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:21.563 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:21.564 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:21.565 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:21.567 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:21.567 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-21T09:14:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:22 smithi071 ceph-mon[36597]: pgmap v68: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 527 KiB/s wr, 167 op/s 2024-03-21T09:14:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:22 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:14:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:22 smithi037 ceph-mon[31830]: pgmap v68: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 527 KiB/s wr, 167 op/s 2024-03-21T09:14:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:22 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:14:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:22 smithi053 ceph-mon[36169]: pgmap v68: 129 pgs: 129 active+clean; 81 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 527 KiB/s wr, 167 op/s 2024-03-21T09:14:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:22 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:14:23.538 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:23.539 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:23.540 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:23.541 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:23.542 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:23.544 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:23.544 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-21T09:14:23.544 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:23.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:23 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:14:23.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:23 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:14:23.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:23 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:14:23.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:23 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:14:23.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:23 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:14:23.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:23 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:14:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:24 smithi071 ceph-mon[36597]: pgmap v69: 129 pgs: 129 active+clean; 367 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 773 KiB/s wr, 230 op/s 2024-03-21T09:14:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:24 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:14:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:24 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:14:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:24 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:24.611 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:24.612 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-21T09:14:24.613 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:24.615 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-21T09:14:24.616 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:24 smithi037 ceph-mon[31830]: pgmap v69: 129 pgs: 129 active+clean; 367 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 773 KiB/s wr, 230 op/s 2024-03-21T09:14:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:24 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:14:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:24 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:14:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:24 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:14:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:24 smithi053 ceph-mon[36169]: pgmap v69: 129 pgs: 129 active+clean; 367 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 773 KiB/s wr, 230 op/s 2024-03-21T09:14:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:24 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:14:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:24 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:14:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:24 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:25.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:25.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:25.141 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:25.142 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-21T09:14:25.143 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:25.144 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-21T09:14:25.145 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:25.146 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:25.146 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:25.146 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-21T09:14:25.146 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:25.146 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 -- 172.21.15.37:0/4223136790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 msgr2=0x7f7a14069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 --2- 172.21.15.37:0/4223136790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14069810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009900 tx=0x7f79fc02bcd0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 -- 172.21.15.37:0/4223136790 shutdown_connections 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 --2- 172.21.15.37:0/4223136790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a1410dc90 0x7f7a14110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 --2- 172.21.15.37:0/4223136790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14069810 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 --2- 172.21.15.37:0/4223136790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a14102430 0x7f7a14068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 -- 172.21.15.37:0/4223136790 >> 172.21.15.37:0/4223136790 conn(0x7f7a14075f00 msgr2=0x7f7a14076310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 -- 172.21.15.37:0/4223136790 shutdown_connections 2024-03-21T09:14:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.012+0000 7f7a18a00640 1 -- 172.21.15.37:0/4223136790 wait complete. 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.013+0000 7f7a18a00640 1 Processor -- start 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 -- start start 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a14102430 0x7f7a1419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a1410dc90 0x7f7a1419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14113a60 con 0x7f7a14102430 2024-03-21T09:14:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141138e0 con 0x7f7a1410dc90 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a18a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14113be0 con 0x7f7a140693d0 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51432/0 (socket says 172.21.15.37:51432) 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a13400640 1 -- 172.21.15.37:0/3992659783 learned_addr learned my addr 172.21.15.37:0/3992659783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.014+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a14102430 0x7f7a1419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13e00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a1410dc90 0x7f7a1419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13400640 1 -- 172.21.15.37:0/3992659783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a1410dc90 msgr2=0x7f7a1419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13400640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a1410dc90 0x7f7a1419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13400640 1 -- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a14102430 msgr2=0x7f7a1419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13400640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a14102430 0x7f7a1419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13400640 1 -- 172.21.15.37:0/3992659783 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1419eef0 con 0x7f7a140693d0 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13e00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a1410dc90 0x7f7a1419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a14102430 0x7f7a1419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a13400640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14199f00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400ebc0 tx=0x7f7a0400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a0be00640 1 -- 172.21.15.37:0/3992659783 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04010630 con 0x7f7a140693d0 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a0be00640 1 -- 172.21.15.37:0/3992659783 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a04010cb0 con 0x7f7a140693d0 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a0be00640 1 -- 172.21.15.37:0/3992659783 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04018bd0 con 0x7f7a140693d0 2024-03-21T09:14:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.015+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1419f1e0 con 0x7f7a140693d0 2024-03-21T09:14:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.016+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a141a6a70 con 0x7f7a140693d0 2024-03-21T09:14:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.019+0000 7f7a1cc4b740 1 Processor -- start 2024-03-21T09:14:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.019+0000 7f7a1cc4b740 1 -- start start 2024-03-21T09:14:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0c2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b0c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266b07b3b0 con 0x56266b0b91e0 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266b01d830 con 0x56266b0b9b00 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266afc65d0 con 0x56266b0c28d0 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b0c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b0c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51434/0 (socket says 172.21.15.37:51434) 2024-03-21T09:14:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a12a00640 1 -- 172.21.15.37:0/3831891594 learned_addr learned my addr 172.21.15.37:0/3831891594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.020+0000 7f7a13e00640 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a12a00640 1 -- 172.21.15.37:0/3831891594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 msgr2=0x56266b0c2390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0c2390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a12a00640 1 -- 172.21.15.37:0/3831891594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 msgr2=0x56266b0b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a12a00640 1 -- 172.21.15.37:0/3831891594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266afde2c0 con 0x56266b0c28d0 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a0be00640 1 -- 172.21.15.37:0/3992659783 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7a040107d0 con 0x7f7a140693d0 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b0c4cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f79fc02c210 tx=0x7f79fc005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8f8f2741c46e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a03e00640 1 -- 172.21.15.37:0/3831891594 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc042070 con 0x56266b0c28d0 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a03e00640 1 -- 172.21.15.37:0/3831891594 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79fc02cb20 con 0x56266b0c28d0 2024-03-21T09:14:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.021+0000 7f7a03e00640 1 -- 172.21.15.37:0/3831891594 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0046c0 con 0x56266b0c28d0 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3831891594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 msgr2=0x56266b0c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b0c4cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f79fc02c210 tx=0x7f79fc005dd0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3831891594 shutdown_connections 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b0c4cc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3831891594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3831891594 >> 172.21.15.37:0/3831891594 conn(0x56266b0b3b00 msgr2=0x56266b0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a0be00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f79f80a30f0 0x7f79f80a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.022+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3831891594 shutdown_connections 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.023+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3831891594 wait complete. 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.023+0000 7f7a0be00640 1 -- 172.21.15.37:0/3992659783 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7a040d0280 con 0x7f7a140693d0 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.023+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f79f80a30f0 0x7f79f80a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.023+0000 7f7a1cc4b740 1 -- start start 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 Processor -- start 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f79f80a30f0 0x7f79f80a55b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79fc000c00 tx=0x7f79fc02cf90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b91e0 0x56266b0bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 0x56266b0bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266afd9a20 con 0x56266b0c28d0 2024-03-21T09:14:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266b0198d0 con 0x56266b0b91e0 2024-03-21T09:14:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266b0c54a0 con 0x56266b0b9b00 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.024+0000 7f7a1cc4b740 10 client.0 _ll_register_callbacks cb 0x56266b0161c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 0x56266b0bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 0x56266b0bfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51438/0 (socket says 172.21.15.37:51438) 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 -- 172.21.15.37:0/3732012928 learned_addr learned my addr 172.21.15.37:0/3732012928 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a13400640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b91e0 0x56266b0bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a13e00640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 -- 172.21.15.37:0/3732012928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b91e0 msgr2=0x56266b0bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b91e0 0x56266b0bf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 -- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 msgr2=0x56266b0c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.025+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a12a00640 1 -- 172.21.15.37:0/3732012928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56266ad32210 con 0x56266b0b9b00 2024-03-21T09:14:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a13400640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b91e0 0x56266b0bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a13e00640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a12a00640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 0x56266b0bfef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x56266b0b4c90 tx=0x7f79fc005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a728991618601d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a00c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a00c00640 1 -- 172.21.15.37:0/3732012928 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc057070 con 0x56266b0b9b00 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79fc02cb20 con 0x56266b0b9b00 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.026+0000 7f7a00c00640 1 -- 172.21.15.37:0/3732012928 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f79fc003d40 con 0x56266b0b9b00 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a00c00640 1 -- 172.21.15.37:0/3732012928 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79fc062050 con 0x56266b0b9b00 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 10 client.26257 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56266adfe620 con 0x56266b0b9b00 2024-03-21T09:14:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a00c00640 1 -- 172.21.15.37:0/3732012928 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79fc02cb20 con 0x56266b0b9b00 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a00c00640 1 -- 172.21.15.37:0/3732012928 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f79fc04e050 con 0x56266b0b9b00 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 10 client.26257 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 20 client.26257 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b0d27c0 0x56266b0d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 4 client.26257 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.027+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56266b02ad70 con 0x56266b0d27c0 2024-03-21T09:14:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.029+0000 7f7a13400640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b0d27c0 0x56266b0d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.033+0000 7f7a13400640 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b0d27c0 0x56266b0d4bb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0caa8fb6f2730e server_cookie=d609ee07f3b0525a in_seq=0 out_seq=0 2024-03-21T09:14:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.033+0000 7f7a00c00640 10 client.26257 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.034+0000 7f7a00c00640 1 -- 172.21.15.37:0/3732012928 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56266b02ad70 con 0x56266b0d27c0 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.034+0000 7f7a00c00640 10 client.26257 handle_command_reply: tid=0 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.034+0000 7f7a1cc4b740 1 client.26257 shutdown 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.034+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 msgr2=0x56266b0bfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.034+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 0x56266b0bfef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x56266b0b4c90 tx=0x7f79fc005dd0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 shutdown_connections 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b9b00 0x56266b0bfef0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b0d27c0 0x56266b0d4bb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/3732012928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b91e0 0x56266b0bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a00c00640 0 client.26257 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 >> 172.21.15.37:0/3732012928 conn(0x56266b0b3b00 msgr2=0x56266b0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 shutdown_connections 2024-03-21T09:14:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/3732012928 wait complete. 2024-03-21T09:14:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.035+0000 7f7a1cc4b740 20 client.26257 trim_cache size 0 max 16384 2024-03-21T09:14:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 Processor -- start 2024-03-21T09:14:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 -- start start 2024-03-21T09:14:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 0x56266b0c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141b2e40 con 0x56266b0c28d0 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141b0940 con 0x56266b0b9b00 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.059+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141ae9e0 con 0x56266b0b91e0 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a12a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 0x56266b0c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 0x56266b0c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51446/0 (socket says 172.21.15.37:51446) 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 -- 172.21.15.37:0/1209869770 learned_addr learned my addr 172.21.15.37:0/1209869770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a12a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:56582/0 (socket says 172.21.15.37:56582) 2024-03-21T09:14:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13e00640 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 -- 172.21.15.37:0/1209869770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 msgr2=0x56266b0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 -- 172.21.15.37:0/1209869770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 msgr2=0x56266b0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 -- 172.21.15.37:0/1209869770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266b0aed10 con 0x56266b0b91e0 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a12a00640 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.060+0000 7f7a13400640 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 0x56266b0c20e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7a040cdc30 tx=0x7f7a040cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423a24a4658fcdef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.061+0000 7f7a03400640 1 -- 172.21.15.37:0/1209869770 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04010e60 con 0x56266b0b91e0 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.061+0000 7f7a03400640 1 -- 172.21.15.37:0/1209869770 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a040a0420 con 0x56266b0b91e0 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.061+0000 7f7a03400640 1 -- 172.21.15.37:0/1209869770 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04098940 con 0x56266b0b91e0 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1209869770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 msgr2=0x56266b0c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 0x56266b0c20e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7a040cdc30 tx=0x7f7a040cf040 comp rx=0 tx=0).stop 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1209869770 shutdown_connections 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0c28d0 0x56266b0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1209869770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0b91e0 0x56266b0c20e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1209869770 >> 172.21.15.37:0/1209869770 conn(0x56266b0b3b00 msgr2=0x56266b0bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1209869770 shutdown_connections 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1209869770 wait complete. 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.062+0000 7f7a1cc4b740 1 -- start start 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 Processor -- start 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b126d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141aefe0 con 0x56266b0b91e0 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141b02e0 con 0x56266b0b9b00 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141af620 con 0x56266b0c28d0 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a1cc4b740 10 client.0 _ll_register_callbacks cb 0x7f7a0c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.063+0000 7f7a13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b126d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b126d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51458/0 (socket says 172.21.15.37:51458) 2024-03-21T09:14:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 -- 172.21.15.37:0/1710269376 learned_addr learned my addr 172.21.15.37:0/1710269376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13400640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a12a00640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 -- 172.21.15.37:0/1710269376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 msgr2=0x56266b0bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 -- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 msgr2=0x56266b0bc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0bc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a13e00640 1 -- 172.21.15.37:0/1710269376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56266b127330 con 0x56266b0c28d0 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.064+0000 7f7a12a00640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a13e00640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b126d70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c012700 tx=0x7f7a0c012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91eff0a477821e4a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a00c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a00c00640 1 -- 172.21.15.37:0/1710269376 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c022060 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a040a0420 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a00c00640 1 -- 172.21.15.37:0/1710269376 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7a0c004440 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a00c00640 1 -- 172.21.15.37:0/1710269376 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a0c029050 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.065+0000 7f7a1cc4b740 10 client.26263 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56266b12e740 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a00c00640 1 -- 172.21.15.37:0/1710269376 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a040a0420 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a00c00640 1 -- 172.21.15.37:0/1710269376 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7a0c01c030 con 0x56266b0c28d0 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 10 client.26263 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 20 client.26263 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b12eeb0 0x56266b1312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 4 client.26263 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:14:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.066+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56266b0250c0 con 0x56266b12eeb0 2024-03-21T09:14:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.067+0000 7f7a13400640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b12eeb0 0x56266b1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.068+0000 7f7a00c00640 10 client.26263 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.068+0000 7f7a13400640 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b12eeb0 0x56266b1312d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d306419a33642d76 server_cookie=c7611ac452c765a8 in_seq=0 out_seq=0 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a00c00640 1 -- 172.21.15.37:0/1710269376 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56266b0250c0 con 0x56266b12eeb0 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a00c00640 10 client.26263 handle_command_reply: tid=0 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 client.26263 shutdown 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 msgr2=0x56266b126d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b126d70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c012700 tx=0x7f7a0c012bc0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 shutdown_connections 2024-03-21T09:14:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266b0c28d0 0x56266b126d70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56266b12eeb0 0x56266b1312d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56266b0b9b00 0x56266b0bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 --2- 172.21.15.37:0/1710269376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56266b0b91e0 0x56266b0bc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 >> 172.21.15.37:0/1710269376 conn(0x56266b0b3b00 msgr2=0x56266b0bf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a00c00640 0 client.26263 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 shutdown_connections 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.080+0000 7f7a1cc4b740 1 -- 172.21.15.37:0/1710269376 wait complete. 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.081+0000 7f7a1cc4b740 20 client.26263 trim_cache size 0 max 16384 2024-03-21T09:14:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:14:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:14:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:14:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:14:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.081+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f79f80a30f0 msgr2=0x7f79f80a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.081+0000 7f7a18a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f79f80a30f0 0x7f79f80a55b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79fc000c00 tx=0x7f79fc02cf90 comp rx=0 tx=0).stop 2024-03-21T09:14:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.081+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 msgr2=0x7f7a14199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.081+0000 7f7a18a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14199f00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400ebc0 tx=0x7f7a0400c6a0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 shutdown_connections 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f79f80a30f0 0x7f79f80a55b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a1410dc90 0x7f7a1419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a14102430 0x7f7a1419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 --2- 172.21.15.37:0/3992659783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a140693d0 0x7f7a14199f00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 >> 172.21.15.37:0/3992659783 conn(0x7f7a14075f00 msgr2=0x7f7a1410eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 shutdown_connections 2024-03-21T09:14:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:26.082+0000 7f7a18a00640 1 -- 172.21.15.37:0/3992659783 wait complete. 2024-03-21T09:14:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:26 smithi071 ceph-mon[36597]: pgmap v70: 129 pgs: 129 active+clean; 367 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 519 KiB/s wr, 152 op/s 2024-03-21T09:14:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:26 smithi037 ceph-mon[31830]: pgmap v70: 129 pgs: 129 active+clean; 367 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 519 KiB/s wr, 152 op/s 2024-03-21T09:14:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:26 smithi053 ceph-mon[36169]: pgmap v70: 129 pgs: 129 active+clean; 367 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 519 KiB/s wr, 152 op/s 2024-03-21T09:14:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-21T09:14:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:26.769 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:26.771 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:26.772 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:26.773 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-21T09:14:26.774 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:26.774 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:26.774 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:28 smithi071 ceph-mon[36597]: pgmap v71: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 789 KiB/s wr, 241 op/s 2024-03-21T09:14:28.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:28 smithi037 ceph-mon[31830]: pgmap v71: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 789 KiB/s wr, 241 op/s 2024-03-21T09:14:28.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:28 smithi053 ceph-mon[36169]: pgmap v71: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 789 KiB/s wr, 241 op/s 2024-03-21T09:14:29.018 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-21T09:14:29.018 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:29.018 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:29.018 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:29.018 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-21T09:14:29.018 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:29.019 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:29.020 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:29.021 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-21T09:14:29.022 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:29.023 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:29.024 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-21T09:14:29.631 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:29.633 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-21T09:14:29.635 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-21T09:14:30.134 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:30.135 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:30.136 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:30.138 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:30.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-21T09:14:30.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:30.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:30.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:30.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-21T09:14:30.140 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:30 smithi071 ceph-mon[36597]: pgmap v72: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 517 KiB/s wr, 151 op/s 2024-03-21T09:14:30.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:30 smithi037 ceph-mon[31830]: pgmap v72: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 517 KiB/s wr, 151 op/s 2024-03-21T09:14:30.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:30 smithi053 ceph-mon[36169]: pgmap v72: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 517 KiB/s wr, 151 op/s 2024-03-21T09:14:32.446 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:32.447 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:32.448 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-21T09:14:32.449 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:32.450 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:32.451 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:32.452 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:32.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:32 smithi071 ceph-mon[36597]: pgmap v73: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 517 KiB/s wr, 151 op/s 2024-03-21T09:14:32.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:32 smithi037 ceph-mon[31830]: pgmap v73: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 517 KiB/s wr, 151 op/s 2024-03-21T09:14:32.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:32 smithi053 ceph-mon[36169]: pgmap v73: 129 pgs: 129 active+clean; 309 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 517 KiB/s wr, 151 op/s 2024-03-21T09:14:34.392 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:34.393 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:34.394 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-21T09:14:34.395 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:34.396 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:34.397 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:34.398 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:34 smithi071 ceph-mon[36597]: pgmap v74: 129 pgs: 129 active+clean; 235 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 772 KiB/s wr, 234 op/s 2024-03-21T09:14:34.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:34 smithi037 ceph-mon[31830]: pgmap v74: 129 pgs: 129 active+clean; 235 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 772 KiB/s wr, 234 op/s 2024-03-21T09:14:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:34 smithi053 ceph-mon[36169]: pgmap v74: 129 pgs: 129 active+clean; 235 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 74 MiB/s rd, 772 KiB/s wr, 234 op/s 2024-03-21T09:14:34.931 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-21T09:14:34.931 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:34.932 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:34.933 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-21T09:14:34.934 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:34.935 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:34.937 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-21T09:14:34.937 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:34.937 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:34.937 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:34.937 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:35.869 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:35.870 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-21T09:14:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-21T09:14:35.872 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:35.874 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-21T09:14:36.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:36 smithi071 ceph-mon[36597]: pgmap v75: 129 pgs: 129 active+clean; 235 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 526 KiB/s wr, 171 op/s 2024-03-21T09:14:36.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:36 smithi037 ceph-mon[31830]: pgmap v75: 129 pgs: 129 active+clean; 235 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 526 KiB/s wr, 171 op/s 2024-03-21T09:14:36.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:36 smithi053 ceph-mon[36169]: pgmap v75: 129 pgs: 129 active+clean; 235 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 526 KiB/s wr, 171 op/s 2024-03-21T09:14:38.257 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:38.257 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:38.257 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:38.258 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:38.259 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-21T09:14:38.260 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-21T09:14:38.261 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-21T09:14:38.263 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:38.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:38 smithi053 ceph-mon[36169]: pgmap v76: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 776 KiB/s wr, 250 op/s 2024-03-21T09:14:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:38 smithi071 ceph-mon[36597]: pgmap v76: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 776 KiB/s wr, 250 op/s 2024-03-21T09:14:38.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:38 smithi037 ceph-mon[31830]: pgmap v76: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 776 KiB/s wr, 250 op/s 2024-03-21T09:14:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.264+0000 7f702d200640 1 -- 172.21.15.37:0/1548424077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028075bd0 msgr2=0x7f7028075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.264+0000 7f702d200640 1 --2- 172.21.15.37:0/1548424077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028075bd0 0x7f7028075fb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7018009930 tx=0x7f701802bc20 comp rx=0 tx=0).stop 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 -- 172.21.15.37:0/1548424077 shutdown_connections 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 --2- 172.21.15.37:0/1548424077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028112520 0x7f7028114910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 --2- 172.21.15.37:0/1548424077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 0x7f7028111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 --2- 172.21.15.37:0/1548424077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028075bd0 0x7f7028075fb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 -- 172.21.15.37:0/1548424077 >> 172.21.15.37:0/1548424077 conn(0x7f70280ff390 msgr2=0x7f70281017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 -- 172.21.15.37:0/1548424077 shutdown_connections 2024-03-21T09:14:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.265+0000 7f702d200640 1 -- 172.21.15.37:0/1548424077 wait complete. 2024-03-21T09:14:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.266+0000 7f702d200640 1 Processor -- start 2024-03-21T09:14:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 -- start start 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 0x7f702819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 0x7f702819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028112520 0x7f70281a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281180c0 con 0x7f70280764f0 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028117f40 con 0x7f7028112520 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f702d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028118240 con 0x7f7028075bd0 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f7025e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 0x7f702819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.267+0000 7f7025e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 0x7f702819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47052/0 (socket says 172.21.15.37:47052) 2024-03-21T09:14:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7026800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028112520 0x7f70281a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 -- 172.21.15.37:0/2978638574 learned_addr learned my addr 172.21.15.37:0/2978638574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f701d400640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 0x7f702819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 -- 172.21.15.37:0/2978638574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028112520 msgr2=0x7f70281a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028112520 0x7f70281a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 -- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 msgr2=0x7f702819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 0x7f702819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 -- 172.21.15.37:0/2978638574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70281a35c0 con 0x7f7028075bd0 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7026800640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028112520 0x7f70281a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f701d400640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 0x7f702819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.268+0000 7f7025e00640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 0x7f702819e620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7018005ab0 tx=0x7f7018005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.269+0000 7f701f400640 1 -- 172.21.15.37:0/2978638574 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018043020 con 0x7f7028075bd0 2024-03-21T09:14:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.269+0000 7f701f400640 1 -- 172.21.15.37:0/2978638574 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7018005000 con 0x7f7028075bd0 2024-03-21T09:14:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.269+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70281a3850 con 0x7f7028075bd0 2024-03-21T09:14:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.269+0000 7f701f400640 1 -- 172.21.15.37:0/2978638574 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018003ab0 con 0x7f7028075bd0 2024-03-21T09:14:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.269+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70281ab140 con 0x7f7028075bd0 2024-03-21T09:14:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 Processor -- start 2024-03-21T09:14:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 -- start start 2024-03-21T09:14:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6e66c0 0x56228c6e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c69f3b0 con 0x56228c6e66c0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c641830 con 0x56228c6dcfd0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f702ffdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c5ea5d0 con 0x56228c6dd8f0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f7026800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f701d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6e66c0 0x56228c6e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f7026800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44572/0 (socket says 172.21.15.37:44572) 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.273+0000 7f7026800640 1 -- 172.21.15.37:0/2829389360 learned_addr learned my addr 172.21.15.37:0/2829389360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7026800640 1 -- 172.21.15.37:0/2829389360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 msgr2=0x56228c6e6180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7026800640 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6e6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7026800640 1 -- 172.21.15.37:0/2829389360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6e66c0 msgr2=0x56228c6e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7026800640 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6e66c0 0x56228c6e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7026800640 1 -- 172.21.15.37:0/2829389360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228c6022c0 con 0x56228c6dcfd0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7026800640 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6dd3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7020011760 tx=0x7f7020011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4537f449242f45b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f701f400640 1 -- 172.21.15.37:0/2978638574 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7018046020 con 0x7f7028075bd0 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7017400640 1 -- 172.21.15.37:0/2829389360 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020004d20 con 0x56228c6dcfd0 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7017400640 1 -- 172.21.15.37:0/2829389360 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702000f9a0 con 0x56228c6dcfd0 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.274+0000 7f7017400640 1 -- 172.21.15.37:0/2829389360 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020004590 con 0x56228c6dcfd0 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.275+0000 7f702ffdc740 1 -- 172.21.15.37:0/2829389360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 msgr2=0x56228c6dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.275+0000 7f702ffdc740 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6dd3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7020011760 tx=0x7f7020011c20 comp rx=0 tx=0).stop 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 -- 172.21.15.37:0/2829389360 shutdown_connections 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6e66c0 0x56228c6e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 --2- 172.21.15.37:0/2829389360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6dd3b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 -- 172.21.15.37:0/2829389360 >> 172.21.15.37:0/2829389360 conn(0x56228c6d78f0 msgr2=0x56228c6d7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 -- 172.21.15.37:0/2829389360 shutdown_connections 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 -- 172.21.15.37:0/2829389360 wait complete. 2024-03-21T09:14:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.276+0000 7f702ffdc740 1 -- start start 2024-03-21T09:14:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 Processor -- start 2024-03-21T09:14:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f701f400640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f70080a3010 0x7f70080a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6e66c0 0x56228c6e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c5fda20 con 0x56228c6dd8f0 2024-03-21T09:14:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f701f400640 1 -- 172.21.15.37:0/2978638574 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f70180f19e0 con 0x7f7028075bd0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c63d8d0 con 0x56228c6dcfd0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c6e92f0 con 0x56228c6e66c0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f702ffdc740 10 client.0 _ll_register_callbacks cb 0x56228c63a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.277+0000 7f7026800640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f70080a3010 0x7f70080a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44574/0 (socket says 172.21.15.37:44574) 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 -- 172.21.15.37:0/1479104280 learned_addr learned my addr 172.21.15.37:0/1479104280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7026800640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6e66c0 0x56228c6e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f701d400640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 -- 172.21.15.37:0/1479104280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6e66c0 msgr2=0x56228c6e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6e66c0 0x56228c6e4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 -- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 msgr2=0x56228c6e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 -- 172.21.15.37:0/1479104280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228c356210 con 0x56228c6dcfd0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7026800640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6e66c0 0x56228c6e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f701d400640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7025e00640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6e37d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f70180ef390 tx=0x7f70180bd560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6869d72357b0cc76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f700fe00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f700fe00640 1 -- 172.21.15.37:0/1479104280 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70180c6550 con 0x56228c6dcfd0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.278+0000 7f7026800640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f70080a3010 0x7f70080a54d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56228c6d71a0 tx=0x7f70200040f0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f700fe00640 1 -- 172.21.15.37:0/1479104280 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f70180051b0 con 0x56228c6dcfd0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f702000f9a0 con 0x56228c6dcfd0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f700fe00640 1 -- 172.21.15.37:0/1479104280 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7018003c50 con 0x56228c6dcfd0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 10 client.26249 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56228c422620 con 0x56228c6dcfd0 2024-03-21T09:14:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f700fe00640 1 -- 172.21.15.37:0/1479104280 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702000f9a0 con 0x56228c6dcfd0 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f700fe00640 1 -- 172.21.15.37:0/1479104280 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f70180bcf90 con 0x56228c6dcfd0 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 10 client.26249 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 20 client.26249 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c6f6980 0x56228c6f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 4 client.26249 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.279+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56228c64ed70 con 0x56228c6f6980 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.280+0000 7f701d400640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c6f6980 0x56228c6f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.280+0000 7f701d400640 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c6f6980 0x56228c6f8dc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bce5982e175b01b server_cookie=1ba4ae88c2eaec9c in_seq=0 out_seq=0 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.280+0000 7f700fe00640 10 client.26249 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.281+0000 7f700fe00640 1 -- 172.21.15.37:0/1479104280 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56228c64ed70 con 0x56228c6f6980 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.281+0000 7f700fe00640 10 client.26249 handle_command_reply: tid=0 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.281+0000 7f702ffdc740 1 client.26249 shutdown 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 msgr2=0x56228c6e37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6e37d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f70180ef390 tx=0x7f70180bd560 comp rx=0 tx=0).stop 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 shutdown_connections 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6e66c0 0x56228c6e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c6f6980 0x56228c6f8dc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 --2- 172.21.15.37:0/1479104280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6dcfd0 0x56228c6e37d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 >> 172.21.15.37:0/1479104280 conn(0x56228c6d78f0 msgr2=0x56228c6d9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f700fe00640 0 client.26249 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 shutdown_connections 2024-03-21T09:14:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 1 -- 172.21.15.37:0/1479104280 wait complete. 2024-03-21T09:14:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.282+0000 7f702ffdc740 20 client.26249 trim_cache size 0 max 16384 2024-03-21T09:14:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.298+0000 7f702ffdc740 1 Processor -- start 2024-03-21T09:14:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.298+0000 7f702ffdc740 1 -- start start 2024-03-21T09:14:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.298+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dcfd0 0x56228c6e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.298+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6d9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f702ffdc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281b75d0 con 0x56228c6dcfd0 2024-03-21T09:14:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f702ffdc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281b50d0 con 0x56228c6e66c0 2024-03-21T09:14:39.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f702ffdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281b3170 con 0x56228c6dd8f0 2024-03-21T09:14:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f701d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6d9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7026800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7025e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dcfd0 0x56228c6e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f701d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6d9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47074/0 (socket says 172.21.15.37:47074) 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f701d400640 1 -- 172.21.15.37:0/3303349536 learned_addr learned my addr 172.21.15.37:0/3303349536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7026800640 1 -- 172.21.15.37:0/3303349536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 msgr2=0x56228c6d9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7026800640 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6d9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7026800640 1 -- 172.21.15.37:0/3303349536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dcfd0 msgr2=0x56228c6e5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7026800640 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dcfd0 0x56228c6e5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.299+0000 7f7026800640 1 -- 172.21.15.37:0/3303349536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228c6d8b20 con 0x56228c6e66c0 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.300+0000 7f7026800640 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6f1a00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7020010010 tx=0x7f7020009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf645daacef4f9a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.300+0000 7f7016a00640 1 -- 172.21.15.37:0/3303349536 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702001e040 con 0x56228c6e66c0 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.300+0000 7f7016a00640 1 -- 172.21.15.37:0/3303349536 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7020003970 con 0x56228c6e66c0 2024-03-21T09:14:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.300+0000 7f7016a00640 1 -- 172.21.15.37:0/3303349536 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020010a40 con 0x56228c6e66c0 2024-03-21T09:14:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 -- 172.21.15.37:0/3303349536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 msgr2=0x56228c6f1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6f1a00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7020010010 tx=0x7f7020009500 comp rx=0 tx=0).stop 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 -- 172.21.15.37:0/3303349536 shutdown_connections 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6f1a00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dd8f0 0x56228c6d9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 --2- 172.21.15.37:0/3303349536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dcfd0 0x56228c6e5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 -- 172.21.15.37:0/3303349536 >> 172.21.15.37:0/3303349536 conn(0x56228c6d78f0 msgr2=0x56228c6d8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 -- 172.21.15.37:0/3303349536 shutdown_connections 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.301+0000 7f702ffdc740 1 -- 172.21.15.37:0/3303349536 wait complete. 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.302+0000 7f702ffdc740 1 -- start start 2024-03-21T09:14:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.302+0000 7f702ffdc740 1 Processor -- start 2024-03-21T09:14:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.302+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 0x56228c6edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f702ffdc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f702ffdc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281b3770 con 0x56228c6dd8f0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f702ffdc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281b4a70 con 0x56228c6e66c0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f702ffdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281b3db0 con 0x56228c6dcfd0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 0x56228c6edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f702ffdc740 10 client.0 _ll_register_callbacks cb 0x7f7018009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 0x56228c6edfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47090/0 (socket says 172.21.15.37:47090) 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 -- 172.21.15.37:0/440830179 learned_addr learned my addr 172.21.15.37:0/440830179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f701d400640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7026800640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 -- 172.21.15.37:0/440830179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 msgr2=0x56228c6e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6e0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 -- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 msgr2=0x56228c6e0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7025e00640 1 -- 172.21.15.37:0/440830179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228c6e12d0 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f7026800640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.303+0000 7f701d400640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f7025e00640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 0x56228c6edfa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70180eef50 tx=0x7f70180bd5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab92d557bbc4577 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f700fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f700fe00640 1 -- 172.21.15.37:0/440830179 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70180c6550 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f700fe00640 1 -- 172.21.15.37:0/440830179 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f70180ba1b0 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7020003970 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f700fe00640 1 -- 172.21.15.37:0/440830179 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70180bd680 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f702ffdc740 10 client.26278 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.304+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56228c752f40 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f700fe00640 1 -- 172.21.15.37:0/440830179 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7020003970 con 0x56228c6dcfd0 2024-03-21T09:14:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f700fe00640 1 -- 172.21.15.37:0/440830179 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f70180a5030 con 0x56228c6dcfd0 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f702ffdc740 10 client.26278 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f702ffdc740 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f702ffdc740 20 client.26278 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f702ffdc740 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c7533f0 0x56228c755810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f702ffdc740 4 client.26278 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56228c6490c0 con 0x56228c7533f0 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.305+0000 7f701d400640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c7533f0 0x56228c755810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.306+0000 7f700fe00640 10 client.26278 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.306+0000 7f701d400640 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c7533f0 0x56228c755810 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595a30eefb1dd456 server_cookie=7d0a0236c4089167 in_seq=0 out_seq=0 2024-03-21T09:14:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.314+0000 7f700fe00640 1 -- 172.21.15.37:0/440830179 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56228c6490c0 con 0x56228c7533f0 2024-03-21T09:14:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.314+0000 7f700fe00640 10 client.26278 handle_command_reply: tid=0 2024-03-21T09:14:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.314+0000 7f702ffdc740 1 client.26278 shutdown 2024-03-21T09:14:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 msgr2=0x56228c6edfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 0x56228c6edfa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70180eef50 tx=0x7f70180bd5d0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 shutdown_connections 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56228c7533f0 0x56228c755810 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56228c6e66c0 0x56228c6e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56228c6dd8f0 0x56228c6e0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f700fe00640 0 client.26278 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 --2- 172.21.15.37:0/440830179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56228c6dcfd0 0x56228c6edfa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 >> 172.21.15.37:0/440830179 conn(0x56228c6d78f0 msgr2=0x56228c6e4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 shutdown_connections 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 1 -- 172.21.15.37:0/440830179 wait complete. 2024-03-21T09:14:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.315+0000 7f702ffdc740 20 client.26278 trim_cache size 0 max 16384 2024-03-21T09:14:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-21T09:14:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-21T09:14:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-21T09:14:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-21T09:14:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f70080a3010 msgr2=0x7f70080a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f70080a3010 0x7f70080a54d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56228c6d71a0 tx=0x7f70200040f0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 msgr2=0x7f702819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 0x7f702819e620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7018005ab0 tx=0x7f7018005e80 comp rx=0 tx=0).stop 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 shutdown_connections 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f70080a3010 0x7f70080a54d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7028112520 0x7f70281a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70280764f0 0x7f702819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 --2- 172.21.15.37:0/2978638574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7028075bd0 0x7f702819e620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 >> 172.21.15.37:0/2978638574 conn(0x7f70280ff390 msgr2=0x7f7028113be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 shutdown_connections 2024-03-21T09:14:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-21T09:14:39.316+0000 7f702d200640 1 -- 172.21.15.37:0/2978638574 wait complete. 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:39.571 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:39.574 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:39.575 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-21T09:14:39.576 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:39.576 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:39.576 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:39.576 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-21T09:14:39.576 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:39.576 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:40.112 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:40.113 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:40.116 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:40.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:40 smithi071 ceph-mon[36597]: pgmap v77: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 506 KiB/s wr, 161 op/s 2024-03-21T09:14:40.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:40 smithi037 ceph-mon[31830]: pgmap v77: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 506 KiB/s wr, 161 op/s 2024-03-21T09:14:40.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:40 smithi053 ceph-mon[36169]: pgmap v77: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 506 KiB/s wr, 161 op/s 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:41.639 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:41.642 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:41.643 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-21T09:14:41.644 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:41.645 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:41.645 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:41.645 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-21T09:14:42.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:42 smithi071 ceph-mon[36597]: pgmap v78: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 506 KiB/s wr, 161 op/s 2024-03-21T09:14:42.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:42 smithi037 ceph-mon[31830]: pgmap v78: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 506 KiB/s wr, 161 op/s 2024-03-21T09:14:42.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:42 smithi053 ceph-mon[36169]: pgmap v78: 129 pgs: 129 active+clean; 225 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 506 KiB/s wr, 161 op/s 2024-03-21T09:14:43.968 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:43.968 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:43.969 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:43.970 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:43.971 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:43.972 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:43.973 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-21T09:14:43.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:14:43.990 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:14:44.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.238+0000 7f4f58800640 1 -- 172.21.15.37:0/3719080234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501031e0 msgr2=0x7f4f5010b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.238+0000 7f4f58800640 1 --2- 172.21.15.37:0/3719080234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501031e0 0x7f4f5010b6c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009900 tx=0x7f4f4002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 -- 172.21.15.37:0/3719080234 shutdown_connections 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 --2- 172.21.15.37:0/3719080234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f5010bc90 0x7f4f5010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 --2- 172.21.15.37:0/3719080234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501031e0 0x7f4f5010b6c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 --2- 172.21.15.37:0/3719080234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501028c0 0x7f4f50102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 -- 172.21.15.37:0/3719080234 >> 172.21.15.37:0/3719080234 conn(0x7f4f50077750 msgr2=0x7f4f50077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 -- 172.21.15.37:0/3719080234 shutdown_connections 2024-03-21T09:14:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.239+0000 7f4f58800640 1 -- 172.21.15.37:0/3719080234 wait complete. 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.240+0000 7f4f58800640 1 Processor -- start 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.240+0000 7f4f58800640 1 -- start start 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f58800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 0x7f4f5019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f58800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501031e0 0x7f4f5019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f58800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f5010bc90 0x7f4f5019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f58800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f50111a70 con 0x7f4f5010bc90 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f58800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501118f0 con 0x7f4f501031e0 2024-03-21T09:14:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f58800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f50111bf0 con 0x7f4f501028c0 2024-03-21T09:14:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 0x7f4f5019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f55e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 0x7f4f5019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46706/0 (socket says 172.21.15.37:46706) 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f55e00640 1 -- 172.21.15.37:0/2462454515 learned_addr learned my addr 172.21.15.37:0/2462454515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.241+0000 7f4f55e00640 1 -- 172.21.15.37:0/2462454515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501031e0 msgr2=0x7f4f5019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.242+0000 7f4f55400640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501031e0 0x7f4f5019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.242+0000 7f4f55e00640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501031e0 0x7f4f5019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.242+0000 7f4f55e00640 1 -- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f5010bc90 msgr2=0x7f4f5019e9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.242+0000 7f4f55e00640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f5010bc90 0x7f4f5019e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.242+0000 7f4f55e00640 1 -- 172.21.15.37:0/2462454515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5019f110 con 0x7f4f501028c0 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.242+0000 7f4f55400640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501031e0 0x7f4f5019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:44.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.243+0000 7f4f55e00640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 0x7f4f5019a170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400e9f0 tx=0x7f4f4400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.244+0000 7f4f4aa00640 1 -- 172.21.15.37:0/2462454515 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400cd80 con 0x7f4f501028c0 2024-03-21T09:14:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.244+0000 7f4f4aa00640 1 -- 172.21.15.37:0/2462454515 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f44005500 con 0x7f4f501028c0 2024-03-21T09:14:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.244+0000 7f4f4aa00640 1 -- 172.21.15.37:0/2462454515 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44010690 con 0x7f4f501028c0 2024-03-21T09:14:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.244+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5019f400 con 0x7f4f501028c0 2024-03-21T09:14:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.244+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f501a6c90 con 0x7f4f501028c0 2024-03-21T09:14:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.248+0000 7f4f5b58c740 1 Processor -- start 2024-03-21T09:14:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.249+0000 7f4f5b58c740 1 -- start start 2024-03-21T09:14:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.249+0000 7f4f4aa00640 1 -- 172.21.15.37:0/2462454515 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4f440046c0 con 0x7f4f501028c0 2024-03-21T09:14:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.249+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 0x5570fc079950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.249+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc079f20 0x5570fc07a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.249+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc083290 0x5570fc085680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fbfde250 con 0x5570fc079f20 2024-03-21T09:14:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fbf86a30 con 0x5570fc07ac00 2024-03-21T09:14:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fbfda2f0 con 0x5570fc083290 2024-03-21T09:14:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 0x5570fc079950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f55400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc083290 0x5570fc085680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 0x5570fc079950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41392/0 (socket says 172.21.15.37:41392) 2024-03-21T09:14:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 -- 172.21.15.37:0/1740552139 learned_addr learned my addr 172.21.15.37:0/1740552139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 -- 172.21.15.37:0/1740552139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc083290 msgr2=0x5570fc085680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc083290 0x5570fc085680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 -- 172.21.15.37:0/1740552139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc079f20 msgr2=0x5570fc07a3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:14:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc079f20 0x5570fc07a3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f56800640 1 -- 172.21.15.37:0/1740552139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570fbf71690 con 0x5570fc07ac00 2024-03-21T09:14:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.250+0000 7f4f55400640 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc083290 0x5570fc085680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.251+0000 7f4f56800640 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 0x5570fc079950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009da0 tx=0x7f4f4c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd7a28e6c8647c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.251+0000 7f4f3f000640 1 -- 172.21.15.37:0/1740552139 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c004b20 con 0x5570fc07ac00 2024-03-21T09:14:44.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.251+0000 7f4f3f000640 1 -- 172.21.15.37:0/1740552139 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c004cc0 con 0x5570fc07ac00 2024-03-21T09:14:44.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.251+0000 7f4f3f000640 1 -- 172.21.15.37:0/1740552139 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c018930 con 0x5570fc07ac00 2024-03-21T09:14:44.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.251+0000 7f4f4aa00640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4f380a3190 0x7f4f380a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.252+0000 7f4f4aa00640 1 -- 172.21.15.37:0/2462454515 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4f440d00c0 con 0x7f4f501028c0 2024-03-21T09:14:44.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.252+0000 7f4f5b58c740 1 -- 172.21.15.37:0/1740552139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 msgr2=0x5570fc079950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.252+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 0x5570fc079950 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009da0 tx=0x7f4f4c00e720 comp rx=0 tx=0).stop 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.252+0000 7f4f55e00640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4f380a3190 0x7f4f380a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 -- 172.21.15.37:0/1740552139 shutdown_connections 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc083290 0x5570fc085680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc079f20 0x5570fc07a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/1740552139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc07ac00 0x5570fc079950 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 -- 172.21.15.37:0/1740552139 >> 172.21.15.37:0/1740552139 conn(0x5570fc041200 msgr2=0x5570fc074970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 -- 172.21.15.37:0/1740552139 shutdown_connections 2024-03-21T09:14:44.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f5b58c740 1 -- 172.21.15.37:0/1740552139 wait complete. 2024-03-21T09:14:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.253+0000 7f4f55e00640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4f380a3190 0x7f4f380a5650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5570fc07b150 tx=0x7f4f440056b0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 -- start start 2024-03-21T09:14:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 Processor -- start 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc082930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc080200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fc086150 con 0x5570fc07ac00 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fbf99e80 con 0x5570fc083290 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fc0862d0 con 0x5570fc079f20 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.254+0000 7f4f5b58c740 10 client.0 _ll_register_callbacks cb 0x5570fbfac130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc082930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc082930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46722/0 (socket says 172.21.15.37:46722) 2024-03-21T09:14:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 -- 172.21.15.37:0/971241201 learned_addr learned my addr 172.21.15.37:0/971241201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 -- 172.21.15.37:0/971241201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 msgr2=0x5570fc080200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc080200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 -- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 msgr2=0x5570fc07fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.255+0000 7f4f56800640 1 -- 172.21.15.37:0/971241201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570fbdbc7d0 con 0x5570fc079f20 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.256+0000 7f4f56800640 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc082930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009be0 tx=0x7f4f4c018510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9483e46e6fe0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.256+0000 7f4f33400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:44.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.256+0000 7f4f33400640 1 -- 172.21.15.37:0/971241201 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c005460 con 0x5570fc079f20 2024-03-21T09:14:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.256+0000 7f4f33400640 1 -- 172.21.15.37:0/971241201 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4f4c005600 con 0x5570fc079f20 2024-03-21T09:14:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.256+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f4c004cc0 con 0x5570fc079f20 2024-03-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.261+0000 7f4f33400640 1 -- 172.21.15.37:0/971241201 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f4c02e050 con 0x5570fc079f20 2024-03-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.261+0000 7f4f5b58c740 10 client.26290 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.261+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5570fbcf23f0 con 0x5570fc079f20 2024-03-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.261+0000 7f4f33400640 1 -- 172.21.15.37:0/971241201 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c004cc0 con 0x5570fc079f20 2024-03-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f33400640 1 -- 172.21.15.37:0/971241201 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4f4c018c40 con 0x5570fc079f20 2024-03-21T09:14:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f5b58c740 10 client.26290 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f5b58c740 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f5b58c740 20 client.26290 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc093190 0x5570fc095580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f5b58c740 4 client.26290 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.262+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570fbfeb790 con 0x5570fc093190 2024-03-21T09:14:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.263+0000 7f4f55400640 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc093190 0x5570fc095580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.266+0000 7f4f55400640 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc093190 0x5570fc095580 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c37fd7656e304b37 server_cookie=5c4b9df75fb2b550 in_seq=0 out_seq=0 2024-03-21T09:14:44.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.266+0000 7f4f33400640 10 client.26290 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:44.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.267+0000 7f4f33400640 1 -- 172.21.15.37:0/971241201 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5570fbfeb790 con 0x5570fc093190 2024-03-21T09:14:44.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.267+0000 7f4f33400640 10 client.26290 handle_command_reply: tid=0 2024-03-21T09:14:44.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.267+0000 7f4f5b58c740 1 client.26290 shutdown 2024-03-21T09:14:44.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.267+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 msgr2=0x5570fc082930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.267+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc082930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009be0 tx=0x7f4f4c018510 comp rx=0 tx=0).stop 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 shutdown_connections 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc093190 0x5570fc095580 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc080200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/971241201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc082930 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 >> 172.21.15.37:0/971241201 conn(0x5570fc041200 msgr2=0x5570fc07a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f33400640 0 client.26290 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 shutdown_connections 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 1 -- 172.21.15.37:0/971241201 wait complete. 2024-03-21T09:14:44.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.268+0000 7f4f5b58c740 20 client.26290 trim_cache size 0 max 16384 2024-03-21T09:14:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.286+0000 7f4f5b58c740 1 Processor -- start 2024-03-21T09:14:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.286+0000 7f4f5b58c740 1 -- start start 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.286+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc0823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.286+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc082930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc084680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501b3160 con 0x5570fc07ac00 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501b0c60 con 0x5570fc083290 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501aed00 con 0x5570fc079f20 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc0823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f55400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc082930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc0823f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46732/0 (socket says 172.21.15.37:46732) 2024-03-21T09:14:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f56800640 1 -- 172.21.15.37:0/3935627617 learned_addr learned my addr 172.21.15.37:0/3935627617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f56800640 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc084680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f55400640 1 -- 172.21.15.37:0/3935627617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 msgr2=0x5570fc0823f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f55400640 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc0823f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f55400640 1 -- 172.21.15.37:0/3935627617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 msgr2=0x5570fc084680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f55400640 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc084680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.287+0000 7f4f55400640 1 -- 172.21.15.37:0/3935627617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570fbcb20f0 con 0x5570fc07ac00 2024-03-21T09:14:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.288+0000 7f4f55400640 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc082930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f40005ab0 tx=0x7f4f40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f42be05309a134 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.288+0000 7f4f3e600640 1 -- 172.21.15.37:0/3935627617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4002c730 con 0x5570fc07ac00 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.288+0000 7f4f3e600640 1 -- 172.21.15.37:0/3935627617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f40043160 con 0x5570fc07ac00 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.288+0000 7f4f3e600640 1 -- 172.21.15.37:0/3935627617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40005290 con 0x5570fc07ac00 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 -- 172.21.15.37:0/3935627617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 msgr2=0x5570fc082930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc082930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f40005ab0 tx=0x7f4f40000bc0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 -- 172.21.15.37:0/3935627617 shutdown_connections 2024-03-21T09:14:44.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc084680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc082930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/3935627617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc0823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 -- 172.21.15.37:0/3935627617 >> 172.21.15.37:0/3935627617 conn(0x5570fc041200 msgr2=0x5570fc0744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 -- 172.21.15.37:0/3935627617 shutdown_connections 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.289+0000 7f4f5b58c740 1 -- 172.21.15.37:0/3935627617 wait complete. 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 -- start start 2024-03-21T09:14:44.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 Processor -- start 2024-03-21T09:14:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc07d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc07c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501af300 con 0x5570fc07ac00 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501b0600 con 0x5570fc083290 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501af940 con 0x5570fc079f20 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f5b58c740 10 client.0 _ll_register_callbacks cb 0x7f4f4c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.290+0000 7f4f56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc07d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc07d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46748/0 (socket says 172.21.15.37:46748) 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 -- 172.21.15.37:0/2157107212 learned_addr learned my addr 172.21.15.37:0/2157107212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc07c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f55400640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 -- 172.21.15.37:0/2157107212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 msgr2=0x5570fc07c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc07c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 -- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 msgr2=0x5570fc07bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:44.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 -- 172.21.15.37:0/2157107212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570fc07c9d0 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f55400640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f56800640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc07d5a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0044d0 tx=0x7f4f4c018510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af430917f713e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f33400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.291+0000 7f4f33400640 1 -- 172.21.15.37:0/2157107212 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0095c0 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f33400640 1 -- 172.21.15.37:0/2157107212 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4f4c003d80 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f40043160 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f33400640 1 -- 172.21.15.37:0/2157107212 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f4c047050 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 10 client.26296 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5570fc0ee450 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f33400640 1 -- 172.21.15.37:0/2157107212 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f40043160 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f33400640 1 -- 172.21.15.37:0/2157107212 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4f4c0186a0 con 0x5570fc079f20 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 10 client.26296 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:44.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 20 client.26296 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc0ee9c0 0x5570fc0f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 4 client.26296 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.292+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570fbfe5ae0 con 0x5570fc0ee9c0 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.293+0000 7f4f55400640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc0ee9c0 0x5570fc0f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.293+0000 7f4f55400640 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc0ee9c0 0x5570fc0f0de0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e44a6f9214e8c9 server_cookie=60411861224c5a04 in_seq=0 out_seq=0 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.293+0000 7f4f33400640 10 client.26296 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f33400640 1 -- 172.21.15.37:0/2157107212 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5570fbfe5ae0 con 0x5570fc0ee9c0 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f33400640 10 client.26296 handle_command_reply: tid=0 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 client.26296 shutdown 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 msgr2=0x5570fc07d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc07d5a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0044d0 tx=0x7f4f4c018510 comp rx=0 tx=0).stop 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 shutdown_connections 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5570fc0ee9c0 0x5570fc0f0de0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5570fc083290 0x5570fc07c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570fc07ac00 0x5570fc07bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 --2- 172.21.15.37:0/2157107212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570fc079f20 0x5570fc07d5a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 >> 172.21.15.37:0/2157107212 conn(0x5570fc041200 msgr2=0x5570fc07e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f33400640 0 client.26296 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 shutdown_connections 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 1 -- 172.21.15.37:0/2157107212 wait complete. 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.294+0000 7f4f5b58c740 20 client.26296 trim_cache size 0 max 16384 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:14:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:14:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:14:44.295 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:14:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4f380a3190 msgr2=0x7f4f380a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4f380a3190 0x7f4f380a5650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5570fc07b150 tx=0x7f4f440056b0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 msgr2=0x7f4f5019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 0x7f4f5019a170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400e9f0 tx=0x7f4f4400edf0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 shutdown_connections 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f5010bc90 0x7f4f5019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4f380a3190 0x7f4f380a5650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f501031e0 0x7f4f5019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 --2- 172.21.15.37:0/2462454515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f501028c0 0x7f4f5019a170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 >> 172.21.15.37:0/2462454515 conn(0x7f4f50077750 msgr2=0x7f4f50107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.295+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 shutdown_connections 2024-03-21T09:14:44.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.296+0000 7f4f58800640 1 -- 172.21.15.37:0/2462454515 wait complete. 2024-03-21T09:14:44.311 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:14:44.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:14:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:44 smithi037 ceph-mon[31830]: pgmap v79: 129 pgs: 129 active+clean; 219 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 756 KiB/s wr, 242 op/s 2024-03-21T09:14:44.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 -- 172.21.15.37:0/2439538926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8109850 msgr2=0x7f83a8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 --2- 172.21.15.37:0/2439538926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8109850 0x7f83a8109c30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f839c009900 tx=0x7f839c02bc40 comp rx=0 tx=0).stop 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 -- 172.21.15.37:0/2439538926 shutdown_connections 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 --2- 172.21.15.37:0/2439538926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8104200 0x7f83a8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 --2- 172.21.15.37:0/2439538926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 0x7f83a8103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 --2- 172.21.15.37:0/2439538926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8109850 0x7f83a8109c30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.550+0000 7f83b0200640 1 -- 172.21.15.37:0/2439538926 >> 172.21.15.37:0/2439538926 conn(0x7f83a80ff350 msgr2=0x7f83a8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.551+0000 7f83b0200640 1 -- 172.21.15.37:0/2439538926 shutdown_connections 2024-03-21T09:14:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.551+0000 7f83b0200640 1 -- 172.21.15.37:0/2439538926 wait complete. 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 Processor -- start 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 -- start start 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 0x7f83a819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8104200 0x7f83a819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8109850 0x7f83a819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a8113cb0 con 0x7f83a8103840 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a8113b30 con 0x7f83a8104200 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83b0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a8113e30 con 0x7f83a8109850 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83ace00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8104200 0x7f83a819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.552+0000 7f83ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 0x7f83a819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8109850 0x7f83a819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 0x7f83a819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35610/0 (socket says 172.21.15.37:35610) 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ad800640 1 -- 172.21.15.37:0/800658831 learned_addr learned my addr 172.21.15.37:0/800658831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 -- 172.21.15.37:0/800658831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8104200 msgr2=0x7f83a819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8104200 0x7f83a819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 -- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 msgr2=0x7f83a819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 0x7f83a819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 -- 172.21.15.37:0/800658831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a819f0e0 con 0x7f83a8109850 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ace00640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8104200 0x7f83a819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:44.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.553+0000 7f83ae200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8109850 0x7f83a819e9b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fd0 tx=0x7f83a000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.554+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000ce20 con 0x7f83a8109850 2024-03-21T09:14:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.554+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a819f3d0 con 0x7f83a8109850 2024-03-21T09:14:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.554+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83a0005020 con 0x7f83a8109850 2024-03-21T09:14:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.554+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0010720 con 0x7f83a8109850 2024-03-21T09:14:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.554+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a81a6c60 con 0x7f83a8109850 2024-03-21T09:14:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.555+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a8068970 con 0x7f83a8109850 2024-03-21T09:14:44.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.557+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f83a0005540 con 0x7f83a8109850 2024-03-21T09:14:44.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.559+0000 7f83a6000640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f83900a3090 0x7f83900a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:44.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.559+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f83a00d0150 con 0x7f83a8109850 2024-03-21T09:14:44.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.559+0000 7f83ad800640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f83900a3090 0x7f83900a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:44.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.560+0000 7f83ad800640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f83900a3090 0x7f83900a5550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f839c0096c0 tx=0x7f839c005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.567+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83a009c8a0 con 0x7f83a8109850 2024-03-21T09:14:44.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:44 smithi071 ceph-mon[36597]: pgmap v79: 129 pgs: 129 active+clean; 219 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 756 KiB/s wr, 242 op/s 2024-03-21T09:14:44.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:44 smithi053 ceph-mon[36169]: pgmap v79: 129 pgs: 129 active+clean; 219 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 756 KiB/s wr, 242 op/s 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:44.712 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:44.714 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-21T09:14:44.715 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:44.716 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-21T09:14:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:44.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.880+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83a802d070 con 0x7f83a8109850 2024-03-21T09:14:44.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.881+0000 7f83a6000640 1 -- 172.21.15.37:0/800658831 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11375 (secure 0 0 0) 0x7f83a009c1a0 con 0x7f83a8109850 2024-03-21T09:14:44.876 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:14:44.876 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:13:56.489186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:14:44.876 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-21T09:14:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.883+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f83900a3090 msgr2=0x7f83900a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.883+0000 7f83b0200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f83900a3090 0x7f83900a5550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f839c0096c0 tx=0x7f839c005a20 comp rx=0 tx=0).stop 2024-03-21T09:14:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.883+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8109850 msgr2=0x7f83a819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.883+0000 7f83b0200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8109850 0x7f83a819e9b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fd0 tx=0x7f83a000ef10 comp rx=0 tx=0).stop 2024-03-21T09:14:44.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 shutdown_connections 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a8109850 0x7f83a819e9b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f83900a3090 0x7f83900a5550 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83a8104200 0x7f83a819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 --2- 172.21.15.37:0/800658831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a8103840 0x7f83a819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 >> 172.21.15.37:0/800658831 conn(0x7f83a80ff350 msgr2=0x7f83a80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 shutdown_connections 2024-03-21T09:14:44.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:44.884+0000 7f83b0200640 1 -- 172.21.15.37:0/800658831 wait complete. 2024-03-21T09:14:44.902 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:14:44.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:14:45.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.119+0000 7f8d15e00640 1 -- 172.21.15.37:0/4206296024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d101008b0 msgr2=0x7f8d10100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.119+0000 7f8d15e00640 1 --2- 172.21.15.37:0/4206296024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d101008b0 0x7f8d10100c90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009930 tx=0x7f8d0002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 -- 172.21.15.37:0/4206296024 shutdown_connections 2024-03-21T09:14:45.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 --2- 172.21.15.37:0/4206296024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10102870 0x7f8d1010d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 --2- 172.21.15.37:0/4206296024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101011d0 0x7f8d10102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 --2- 172.21.15.37:0/4206296024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d101008b0 0x7f8d10100c90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 -- 172.21.15.37:0/4206296024 >> 172.21.15.37:0/4206296024 conn(0x7f8d10077780 msgr2=0x7f8d10077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 -- 172.21.15.37:0/4206296024 shutdown_connections 2024-03-21T09:14:45.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.120+0000 7f8d15e00640 1 -- 172.21.15.37:0/4206296024 wait complete. 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 Processor -- start 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 -- start start 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101008b0 0x7f8d101036e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 0x7f8d10103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10104160 0x7f8d101a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d10110ab0 con 0x7f8d101008b0 2024-03-21T09:14:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d10110930 con 0x7f8d10104160 2024-03-21T09:14:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.121+0000 7f8d15e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d10110c30 con 0x7f8d10102870 2024-03-21T09:14:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 0x7f8d10103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 0x7f8d10103c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46806/0 (socket says 172.21.15.37:46806) 2024-03-21T09:14:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 -- 172.21.15.37:0/2947468688 learned_addr learned my addr 172.21.15.37:0/2947468688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0f400640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10104160 0x7f8d101a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101008b0 0x7f8d101036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 -- 172.21.15.37:0/2947468688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10104160 msgr2=0x7f8d101a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10104160 0x7f8d101a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 -- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101008b0 msgr2=0x7f8d101036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101008b0 0x7f8d101036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0e000640 1 -- 172.21.15.37:0/2947468688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d101a68d0 con 0x7f8d10102870 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.122+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101008b0 0x7f8d101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.123+0000 7f8d0e000640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 0x7f8d10103c20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005fd0 tx=0x7f8cfc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.123+0000 7f8d14a00640 1 -- 172.21.15.37:0/2947468688 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc017070 con 0x7f8d10102870 2024-03-21T09:14:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.123+0000 7f8d14a00640 1 -- 172.21.15.37:0/2947468688 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cfc009c30 con 0x7f8d10102870 2024-03-21T09:14:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.123+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d101a6bc0 con 0x7f8d10102870 2024-03-21T09:14:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.123+0000 7f8d14a00640 1 -- 172.21.15.37:0/2947468688 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc00f6b0 con 0x7f8d10102870 2024-03-21T09:14:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.123+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d101a7160 con 0x7f8d10102870 2024-03-21T09:14:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.127+0000 7f8d14a00640 1 -- 172.21.15.37:0/2947468688 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8cfc010460 con 0x7f8d10102870 2024-03-21T09:14:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.128+0000 7f8d18a67740 1 Processor -- start 2024-03-21T09:14:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.128+0000 7f8d18a67740 1 -- start start 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.128+0000 7f8d14a00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8cf40a3110 0x7f8cf40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.128+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 0x55874de4bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.128+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4c080 0x55874de54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.128+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de57050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d18a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874ddaf2c0 con 0x55874de54c00 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d18a67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874dd57c90 con 0x55874de4b6d0 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d18a67740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874ddab360 con 0x55874de4c080 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d14a00640 1 -- 172.21.15.37:0/2947468688 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8cfc0cf830 con 0x7f8d10102870 2024-03-21T09:14:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 0x55874de4bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 0x55874de4bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41486/0 (socket says 172.21.15.37:41486) 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 -- 172.21.15.37:0/3117224733 learned_addr learned my addr 172.21.15.37:0/3117224733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0e000640 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4c080 0x55874de54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 -- 172.21.15.37:0/3117224733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4c080 msgr2=0x55874de54630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0f400640 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de57050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4c080 0x55874de54630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 -- 172.21.15.37:0/3117224733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 msgr2=0x55874de57050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de57050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0ea00640 1 -- 172.21.15.37:0/3117224733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55874dd7c080 con 0x55874de4b6d0 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0f400640 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de57050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.129+0000 7f8d0f400640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8cf40a3110 0x7f8cf40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.130+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 0x55874de4bab0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8d0002c2d0 tx=0x7f8d00005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369d012520f8ff03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.130+0000 7f8cfbe00640 1 -- 172.21.15.37:0/3117224733 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00004240 con 0x55874de4b6d0 2024-03-21T09:14:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.130+0000 7f8cfbe00640 1 -- 172.21.15.37:0/3117224733 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d0002cdf0 con 0x55874de4b6d0 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.130+0000 7f8cfbe00640 1 -- 172.21.15.37:0/3117224733 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d000046c0 con 0x55874de4b6d0 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 -- 172.21.15.37:0/3117224733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 msgr2=0x55874de4bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d0f400640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8cf40a3110 0x7f8cf40a55d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d08005fd0 tx=0x7f8d08004ab0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 0x55874de4bab0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8d0002c2d0 tx=0x7f8d00005e50 comp rx=0 tx=0).stop 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 -- 172.21.15.37:0/3117224733 shutdown_connections 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de57050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4c080 0x55874de54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3117224733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4b6d0 0x55874de4bab0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 -- 172.21.15.37:0/3117224733 >> 172.21.15.37:0/3117224733 conn(0x55874de45da0 msgr2=0x55874de46180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 -- 172.21.15.37:0/3117224733 shutdown_connections 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.131+0000 7f8d18a67740 1 -- 172.21.15.37:0/3117224733 wait complete. 2024-03-21T09:14:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.132+0000 7f8d18a67740 1 -- start start 2024-03-21T09:14:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.132+0000 7f8d18a67740 1 Processor -- start 2024-03-21T09:14:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.132+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de54590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.132+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de51920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.132+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.132+0000 7f8d18a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874de57b10 con 0x55874de54c00 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d18a67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874dd6b0d0 con 0x55874de4c080 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d18a67740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874de57c90 con 0x55874de4b6d0 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d18a67740 10 client.0 _ll_register_callbacks cb 0x55874dd6fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de51920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de51920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41502/0 (socket says 172.21.15.37:41502) 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 -- 172.21.15.37:0/3148158333 learned_addr learned my addr 172.21.15.37:0/3148158333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0f400640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 -- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 msgr2=0x55874de54590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de54590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 -- 172.21.15.37:0/3148158333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 msgr2=0x55874de51f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de51f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0e000640 1 -- 172.21.15.37:0/3148158333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55874dac33f0 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.133+0000 7f8d0f400640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8d0e000640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de51920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc010fd0 tx=0x7f8cfc09b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a574978a7d63fc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8cf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8cf8c00640 1 -- 172.21.15.37:0/3148158333 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0b4060 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d0002cdf0 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8cf8c00640 1 -- 172.21.15.37:0/3148158333 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8cfc097e90 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8cf8c00640 1 -- 172.21.15.37:0/3148158333 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cfc020040 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8d18a67740 10 client.26285 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55874db8d7d0 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8cf8c00640 1 -- 172.21.15.37:0/3148158333 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d0002cdf0 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.134+0000 7f8cf8c00640 1 -- 172.21.15.37:0/3148158333 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8cfc0cf3b0 con 0x55874de4c080 2024-03-21T09:14:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d18a67740 10 client.26285 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d18a67740 10 client.26285 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d18a67740 20 client.26285 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874de64a10 0x55874de66e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d18a67740 4 client.26285 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55874ddbc800 con 0x55874de64a10 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.135+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874de64a10 0x55874de66e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.136+0000 7f8cf8c00640 10 client.26285 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:45.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.136+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874de64a10 0x55874de66e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=987c86a04ce573da server_cookie=c8ba04f3365125a0 in_seq=0 out_seq=0 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.136+0000 7f8cf8c00640 1 -- 172.21.15.37:0/3148158333 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55874ddbc800 con 0x55874de64a10 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.136+0000 7f8cf8c00640 10 client.26285 handle_command_reply: tid=0 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 client.26285 shutdown 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 msgr2=0x55874de51920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de51920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc010fd0 tx=0x7f8cfc09b930 comp rx=0 tx=0).stop 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 shutdown_connections 2024-03-21T09:14:45.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874de64a10 0x55874de66e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de54c00 0x55874de51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de51920 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 --2- 172.21.15.37:0/3148158333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8cf8c00640 0 client.26285 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 >> 172.21.15.37:0/3148158333 conn(0x55874de45da0 msgr2=0x55874de47af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 shutdown_connections 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 1 -- 172.21.15.37:0/3148158333 wait complete. 2024-03-21T09:14:45.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.137+0000 7f8d18a67740 20 client.26285 trim_cache size 0 max 16384 2024-03-21T09:14:45.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.153+0000 7f8d18a67740 1 Processor -- start 2024-03-21T09:14:45.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.153+0000 7f8d18a67740 1 -- start start 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.153+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4b6d0 0x55874de54050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.153+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de54590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 0x55874de4eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d18a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b31e0 con 0x55874de4b6d0 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d18a67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b0ce0 con 0x55874de4c080 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d18a67740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101aed80 con 0x55874de54c00 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 0x55874de4eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de54590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41510/0 (socket says 172.21.15.37:41510) 2024-03-21T09:14:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 0x55874de4eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46842/0 (socket says 172.21.15.37:46842) 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0e000640 1 -- 172.21.15.37:0/1036378033 learned_addr learned my addr 172.21.15.37:0/1036378033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4b6d0 0x55874de54050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 -- 172.21.15.37:0/1036378033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 msgr2=0x55874de54590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de54590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 -- 172.21.15.37:0/1036378033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4b6d0 msgr2=0x55874de54050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4b6d0 0x55874de54050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0f400640 1 -- 172.21.15.37:0/1036378033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55874de508d0 con 0x55874de54c00 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0e000640 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.154+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4b6d0 0x55874de54050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.155+0000 7f8d0f400640 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 0x55874de4eba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d08010da0 tx=0x7f8d08015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3cb286b6ec018 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.155+0000 7f8cfb400640 1 -- 172.21.15.37:0/1036378033 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08004670 con 0x55874de54c00 2024-03-21T09:14:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.155+0000 7f8cfb400640 1 -- 172.21.15.37:0/1036378033 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d08004810 con 0x55874de54c00 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.155+0000 7f8cfb400640 1 -- 172.21.15.37:0/1036378033 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0800fe40 con 0x55874de54c00 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 -- 172.21.15.37:0/1036378033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 msgr2=0x55874de4eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 0x55874de4eba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d08010da0 tx=0x7f8d08015000 comp rx=0 tx=0).stop 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 -- 172.21.15.37:0/1036378033 shutdown_connections 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de54c00 0x55874de4eba0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de4c080 0x55874de54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 --2- 172.21.15.37:0/1036378033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4b6d0 0x55874de54050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 -- 172.21.15.37:0/1036378033 >> 172.21.15.37:0/1036378033 conn(0x55874de45da0 msgr2=0x55874de44eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 -- 172.21.15.37:0/1036378033 shutdown_connections 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 -- 172.21.15.37:0/1036378033 wait complete. 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.156+0000 7f8d18a67740 1 -- start start 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 Processor -- start 2024-03-21T09:14:45.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de5a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4c080 0x55874de5a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de54c00 0x55874de5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101af380 con 0x55874de4c080 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b0680 con 0x55874de54c00 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101af9c0 con 0x55874de4b6d0 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d18a67740 10 client.0 _ll_register_callbacks cb 0x7f8cfc09b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de5a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de5a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46844/0 (socket says 172.21.15.37:46844) 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d0ea00640 1 -- 172.21.15.37:0/2056787342 learned_addr learned my addr 172.21.15.37:0/2056787342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d0f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de54c00 0x55874de5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.157+0000 7f8d0e000640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4c080 0x55874de5a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0ea00640 1 -- 172.21.15.37:0/2056787342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de54c00 msgr2=0x55874de5aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de54c00 0x55874de5aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0ea00640 1 -- 172.21.15.37:0/2056787342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4c080 msgr2=0x55874de5a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4c080 0x55874de5a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0ea00640 1 -- 172.21.15.37:0/2056787342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55874debf2e0 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0f400640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de54c00 0x55874de5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d0ea00640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de5a450 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8d000057c0 tx=0x7f8d0002bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9290b8136c099205 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8cf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8cf8c00640 1 -- 172.21.15.37:0/2056787342 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00004240 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d08004810 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.158+0000 7f8cf8c00640 1 -- 172.21.15.37:0/2056787342 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8d000043e0 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8cf8c00640 1 -- 172.21.15.37:0/2056787342 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d00052030 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 10 client.26320 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55874dec0620 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8cf8c00640 1 -- 172.21.15.37:0/2056787342 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d08004810 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8cf8c00640 1 -- 172.21.15.37:0/2056787342 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8d000344d0 con 0x55874de4b6d0 2024-03-21T09:14:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 10 client.26320 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 10 client.26320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:14:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 20 client.26320 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874dec09e0 0x55874dec2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 4 client.26320 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:14:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.159+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55874ddb6b50 con 0x55874dec09e0 2024-03-21T09:14:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.160+0000 7f8d0e000640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874dec09e0 0x55874dec2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.160+0000 7f8cf8c00640 10 client.26320 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.160+0000 7f8d0e000640 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874dec09e0 0x55874dec2dd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4203799bdc3d03c2 server_cookie=85f862f0546fbe56 in_seq=0 out_seq=0 2024-03-21T09:14:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.161+0000 7f8cf8c00640 1 -- 172.21.15.37:0/2056787342 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55874ddb6b50 con 0x55874dec09e0 2024-03-21T09:14:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.161+0000 7f8cf8c00640 10 client.26320 handle_command_reply: tid=0 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.161+0000 7f8d18a67740 1 client.26320 shutdown 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.161+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 msgr2=0x55874de5a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.161+0000 7f8d18a67740 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de5a450 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8d000057c0 tx=0x7f8d0002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 shutdown_connections 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55874de54c00 0x55874de5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55874dec09e0 0x55874dec2dd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55874de4c080 0x55874de5a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 --2- 172.21.15.37:0/2056787342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55874de4b6d0 0x55874de5a450 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8cf8c00640 0 client.26320 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 >> 172.21.15.37:0/2056787342 conn(0x55874de45da0 msgr2=0x55874de4c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 shutdown_connections 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 1 -- 172.21.15.37:0/2056787342 wait complete. 2024-03-21T09:14:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.162+0000 7f8d18a67740 20 client.26320 trim_cache size 0 max 16384 2024-03-21T09:14:45.160 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:14:45.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8cf40a3110 msgr2=0x7f8cf40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8cf40a3110 0x7f8cf40a55d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d08005fd0 tx=0x7f8d08004ab0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 msgr2=0x7f8d10103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 0x7f8d10103c20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005fd0 tx=0x7f8cfc00dde0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 shutdown_connections 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8cf40a3110 0x7f8cf40a55d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d10104160 0x7f8d101a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d10102870 0x7f8d10103c20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 --2- 172.21.15.37:0/2947468688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d101008b0 0x7f8d101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 >> 172.21.15.37:0/2947468688 conn(0x7f8d10077780 msgr2=0x7f8d10105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 shutdown_connections 2024-03-21T09:14:45.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.163+0000 7f8d15e00640 1 -- 172.21.15.37:0/2947468688 wait complete. 2024-03-21T09:14:45.177 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:14:45.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:14:45.250 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:45.250 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:45.250 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-21T09:14:45.250 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:45.250 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:45.251 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:45.252 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:45.253 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:45.254 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:45.255 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-21T09:14:45.435 INFO:tasks.workunit.client.0.smithi037.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:45.436 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:45.436 INFO:tasks.workunit.client.0.smithi037.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:45.436 INFO:tasks.workunit.client.0.smithi037.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-21T09:14:45.436 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T09:14:45.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 -- 172.21.15.37:0/3302748396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 msgr2=0x7fa33411a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 --2- 172.21.15.37:0/3302748396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa320009900 tx=0x7fa32002bc50 comp rx=0 tx=0).stop 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 -- 172.21.15.37:0/3302748396 shutdown_connections 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 --2- 172.21.15.37:0/3302748396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 0x7fa33411d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 --2- 172.21.15.37:0/3302748396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a800 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 --2- 172.21.15.37:0/3302748396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 0x7fa33406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.468+0000 7fa33b200640 1 -- 172.21.15.37:0/3302748396 >> 172.21.15.37:0/3302748396 conn(0x7fa33406c7f0 msgr2=0x7fa33410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.469+0000 7fa33b200640 1 -- 172.21.15.37:0/3302748396 shutdown_connections 2024-03-21T09:14:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.469+0000 7fa33b200640 1 -- 172.21.15.37:0/3302748396 wait complete. 2024-03-21T09:14:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.470+0000 7fa33b200640 1 Processor -- start 2024-03-21T09:14:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.471+0000 7fa33b200640 1 -- start start 2024-03-21T09:14:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.471+0000 7fa33b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 0x7fa33411a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.471+0000 7fa33b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.471+0000 7fa33b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 0x7fa3341134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa333e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 0x7fa33411a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa333e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 0x7fa33411a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41538/0 (socket says 172.21.15.37:41538) 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa333400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa333400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46872/0 (socket says 172.21.15.37:46872) 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa333e00640 1 -- 172.21.15.37:0/3482989178 learned_addr learned my addr 172.21.15.37:0/3482989178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa334120850 con 0x7fa33411ad40 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341206d0 con 0x7fa33406d100 2024-03-21T09:14:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.472+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341209d0 con 0x7fa33406da20 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa339200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 0x7fa3341134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa333400640 1 -- 172.21.15.37:0/3482989178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 msgr2=0x7fa33411a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa333400640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 0x7fa33411a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa333400640 1 -- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 msgr2=0x7fa3341134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa333400640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 0x7fa3341134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa333400640 1 -- 172.21.15.37:0/3482989178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334113da0 con 0x7fa33406da20 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.473+0000 7fa339200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 0x7fa3341134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:14:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.474+0000 7fa333400640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa32002c260 tx=0x7fa32002ea00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.474+0000 7fa330c00640 1 -- 172.21.15.37:0/3482989178 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320041020 con 0x7fa33406da20 2024-03-21T09:14:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.475+0000 7fa330c00640 1 -- 172.21.15.37:0/3482989178 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa32002ee00 con 0x7fa33406da20 2024-03-21T09:14:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.475+0000 7fa330c00640 1 -- 172.21.15.37:0/3482989178 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320004c90 con 0x7fa33406da20 2024-03-21T09:14:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.475+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa334114030 con 0x7fa33406da20 2024-03-21T09:14:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.475+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa334117fa0 con 0x7fa33406da20 2024-03-21T09:14:45.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.479+0000 7fa330c00640 1 -- 172.21.15.37:0/3482989178 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa320044020 con 0x7fa33406da20 2024-03-21T09:14:45.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.481+0000 7fa330c00640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3180a31e0 0x7fa3180a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.481+0000 7fa33de5c740 1 Processor -- start 2024-03-21T09:14:45.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.482+0000 7fa33de5c740 1 -- start start 2024-03-21T09:14:45.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.482+0000 7fa330c00640 1 -- 172.21.15.37:0/3482989178 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa3200f1030 con 0x7fa33406da20 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.482+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.482+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c167630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.482+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c16a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.482+0000 7fa333e00640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3180a31e0 0x7fa3180a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa33de5c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0c22c0 con 0x56216c167c00 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa33de5c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c06ac90 con 0x56216c15e6d0 2024-03-21T09:14:45.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa33de5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0be360 con 0x56216c15f080 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa333e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa333e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c15eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41540/0 (socket says 172.21.15.37:41540) 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa333e00640 1 -- 172.21.15.37:0/2086761718 learned_addr learned my addr 172.21.15.37:0/2086761718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa333400640 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.483+0000 7fa339200640 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c16a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.484+0000 7fa333400640 1 -- 172.21.15.37:0/2086761718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 msgr2=0x56216c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.484+0000 7fa333400640 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.484+0000 7fa333400640 1 -- 172.21.15.37:0/2086761718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 msgr2=0x56216c16a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.484+0000 7fa333400640 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c16a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.484+0000 7fa333400640 1 -- 172.21.15.37:0/2086761718 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56216c08f080 con 0x56216c15f080 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.484+0000 7fa333400640 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c167630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ee4f0 tx=0x7fa320005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9618785beb2985 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.485+0000 7fa333e00640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3180a31e0 0x7fa3180a56a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa32c013e80 tx=0x7fa32c013590 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.485+0000 7fa329600640 1 -- 172.21.15.37:0/2086761718 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200bcaf0 con 0x56216c15f080 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.485+0000 7fa329600640 1 -- 172.21.15.37:0/2086761718 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3200bd890 con 0x56216c15f080 2024-03-21T09:14:45.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.485+0000 7fa329600640 1 -- 172.21.15.37:0/2086761718 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b9800 con 0x56216c15f080 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.486+0000 7fa33de5c740 1 -- 172.21.15.37:0/2086761718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 msgr2=0x56216c167630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.486+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c167630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ee4f0 tx=0x7fa320005fd0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.487+0000 7fa33de5c740 1 -- 172.21.15.37:0/2086761718 shutdown_connections 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.487+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c16a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.487+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c167630 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.487+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2086761718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.487+0000 7fa33de5c740 1 -- 172.21.15.37:0/2086761718 >> 172.21.15.37:0/2086761718 conn(0x56216c158da0 msgr2=0x56216c159180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.488+0000 7fa33de5c740 1 -- 172.21.15.37:0/2086761718 shutdown_connections 2024-03-21T09:14:45.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.488+0000 7fa33de5c740 1 -- 172.21.15.37:0/2086761718 wait complete. 2024-03-21T09:14:45.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.489+0000 7fa33de5c740 1 -- start start 2024-03-21T09:14:45.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.489+0000 7fa33de5c740 1 Processor -- start 2024-03-21T09:14:45.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.489+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c1674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c1648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c164e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa33de5c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0be360 con 0x56216c167c00 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa33de5c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c07e0d0 con 0x56216c15e6d0 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa33de5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c16a930 con 0x56216c15f080 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa33de5c740 10 client.0 _ll_register_callbacks cb 0x56216c082e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.490+0000 7fa333400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c1648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.491+0000 7fa333400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c1648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46884/0 (socket says 172.21.15.37:46884) 2024-03-21T09:14:45.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.491+0000 7fa333e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c1674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.491+0000 7fa339200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.491+0000 7fa333400640 1 -- 172.21.15.37:0/2065863225 learned_addr learned my addr 172.21.15.37:0/2065863225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa333e00640 1 -- 172.21.15.37:0/2065863225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 msgr2=0x56216c1648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa333e00640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c1648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa333e00640 1 -- 172.21.15.37:0/2065863225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 msgr2=0x56216c164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa333e00640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c164e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa333e00640 1 -- 172.21.15.37:0/2065863225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56216bdd63f0 con 0x56216c15e6d0 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa339200640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.492+0000 7fa333400640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c1648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.493+0000 7fa333e00640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c1674a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009810 tx=0x7fa32c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c21c7046ac3348 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.493+0000 7fa31e000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.493+0000 7fa31e000640 1 -- 172.21.15.37:0/2065863225 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c02a430 con 0x56216c15e6d0 2024-03-21T09:14:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.493+0000 7fa31e000640 1 -- 172.21.15.37:0/2065863225 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa32c01c040 con 0x56216c15e6d0 2024-03-21T09:14:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.493+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3200bd890 con 0x56216c15e6d0 2024-03-21T09:14:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.494+0000 7fa31e000640 1 -- 172.21.15.37:0/2065863225 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32c032050 con 0x56216c15e6d0 2024-03-21T09:14:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.494+0000 7fa33de5c740 10 client.26303 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.494+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56216bea07d0 con 0x56216c15e6d0 2024-03-21T09:14:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.495+0000 7fa31e000640 1 -- 172.21.15.37:0/2065863225 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3200bd890 con 0x56216c15e6d0 2024-03-21T09:14:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.495+0000 7fa31e000640 1 -- 172.21.15.37:0/2065863225 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa32c020050 con 0x56216c15e6d0 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.496+0000 7fa33de5c740 10 client.26303 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.496+0000 7fa33de5c740 10 client.26303 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.496+0000 7fa33de5c740 20 client.26303 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.496+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c177660 0x56216c179a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.496+0000 7fa33de5c740 4 client.26303 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.496+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56216c0cf800 con 0x56216c177660 2024-03-21T09:14:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.497+0000 7fa333400640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c177660 0x56216c179a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.498+0000 7fa333400640 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c177660 0x56216c179a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e464c4d695de65ea server_cookie=3f1e6b8e4fd5c6bc in_seq=0 out_seq=0 2024-03-21T09:14:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.498+0000 7fa31e000640 10 client.26303 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa31e000640 1 -- 172.21.15.37:0/2065863225 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56216c0cf800 con 0x56216c177660 2024-03-21T09:14:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa31e000640 10 client.26303 handle_command_reply: tid=0 2024-03-21T09:14:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa33de5c740 1 client.26303 shutdown 2024-03-21T09:14:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 msgr2=0x56216c1674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c1674a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009810 tx=0x7fa32c01d000 comp rx=0 tx=0).stop 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 shutdown_connections 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.500+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c177660 0x56216c179a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c1648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 1 --2- 172.21.15.37:0/2065863225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c15e6d0 0x56216c1674a0 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009810 tx=0x7fa32c01d000 comp rx=0 tx=0).stop 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 >> 172.21.15.37:0/2065863225 conn(0x56216c158da0 msgr2=0x56216c15aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa31e000640 0 client.26303 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 shutdown_connections 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 1 -- 172.21.15.37:0/2065863225 wait complete. 2024-03-21T09:14:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.501+0000 7fa33de5c740 20 client.26303 trim_cache size 0 max 16384 2024-03-21T09:14:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.530+0000 7fa33de5c740 1 Processor -- start 2024-03-21T09:14:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 -- start start 2024-03-21T09:14:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 0x56216c161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341c84d0 con 0x56216c167c00 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341c5fd0 con 0x56216c17f300 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.532+0000 7fa33de5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341c4070 con 0x56216c15f080 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 0x56216c161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 0x56216c161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41560/0 (socket says 172.21.15.37:41560) 2024-03-21T09:14:45.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 -- 172.21.15.37:0/3060887642 learned_addr learned my addr 172.21.15.37:0/3060887642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 -- 172.21.15.37:0/3060887642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 msgr2=0x56216c163170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:14:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c163170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 -- 172.21.15.37:0/3060887642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 msgr2=0x56216c166fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:14:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c166fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.533+0000 7fa339200640 1 -- 172.21.15.37:0/3060887642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56216c164840 con 0x56216c17f300 2024-03-21T09:14:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.534+0000 7fa339200640 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 0x56216c161b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa324009bd0 tx=0x7fa3240127c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda15fc4a4aaf1c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.534+0000 7fa328c00640 1 -- 172.21.15.37:0/3060887642 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324009cc0 con 0x56216c17f300 2024-03-21T09:14:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.534+0000 7fa328c00640 1 -- 172.21.15.37:0/3060887642 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa324005020 con 0x56216c17f300 2024-03-21T09:14:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.534+0000 7fa328c00640 1 -- 172.21.15.37:0/3060887642 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324004b00 con 0x56216c17f300 2024-03-21T09:14:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 -- 172.21.15.37:0/3060887642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 msgr2=0x56216c161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 0x56216c161b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa324009bd0 tx=0x7fa3240127c0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 -- 172.21.15.37:0/3060887642 shutdown_connections 2024-03-21T09:14:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c17f300 0x56216c161b00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c167c00 0x56216c166fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 --2- 172.21.15.37:0/3060887642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c15f080 0x56216c163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.536+0000 7fa33de5c740 1 -- 172.21.15.37:0/3060887642 >> 172.21.15.37:0/3060887642 conn(0x56216c158da0 msgr2=0x56216c157eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.537+0000 7fa33de5c740 1 -- 172.21.15.37:0/3060887642 shutdown_connections 2024-03-21T09:14:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.537+0000 7fa33de5c740 1 -- 172.21.15.37:0/3060887642 wait complete. 2024-03-21T09:14:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.537+0000 7fa33de5c740 1 -- start start 2024-03-21T09:14:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.538+0000 7fa33de5c740 1 Processor -- start 2024-03-21T09:14:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c15f080 0x56216c16f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 0x56216c16dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa33de5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c17f300 0x56216c16e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa333400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 0x56216c16dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa33de5c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341c4670 con 0x56216c15f080 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa333400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 0x56216c16dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41570/0 (socket says 172.21.15.37:41570) 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa333400640 1 -- 172.21.15.37:0/864520199 learned_addr learned my addr 172.21.15.37:0/864520199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa339200640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c17f300 0x56216c16e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.539+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341c5970 con 0x56216c167c00 2024-03-21T09:14:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.540+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341c4cb0 con 0x56216c17f300 2024-03-21T09:14:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.540+0000 7fa33de5c740 10 client.0 _ll_register_callbacks cb 0x7fa32c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.540+0000 7fa333e00640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c15f080 0x56216c16f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa339200640 1 -- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 msgr2=0x56216c16dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa339200640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 0x56216c16dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa339200640 1 -- 172.21.15.37:0/864520199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c15f080 msgr2=0x56216c16f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa339200640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c15f080 0x56216c16f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa339200640 1 -- 172.21.15.37:0/864520199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56216c1651b0 con 0x56216c17f300 2024-03-21T09:14:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa333400640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 0x56216c16dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.541+0000 7fa333e00640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c15f080 0x56216c16f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.542+0000 7fa339200640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c17f300 0x56216c16e2e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa324009f40 tx=0x7fa324009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e805216db40892 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.542+0000 7fa31e000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.542+0000 7fa31e000640 1 -- 172.21.15.37:0/864520199 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324004260 con 0x56216c17f300 2024-03-21T09:14:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.543+0000 7fa31e000640 1 -- 172.21.15.37:0/864520199 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa324004400 con 0x56216c17f300 2024-03-21T09:14:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.543+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa324005020 con 0x56216c17f300 2024-03-21T09:14:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.543+0000 7fa31e000640 1 -- 172.21.15.37:0/864520199 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa324004880 con 0x56216c17f300 2024-03-21T09:14:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.543+0000 7fa33de5c740 10 client.26335 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.543+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56216c1d3df0 con 0x56216c17f300 2024-03-21T09:14:45.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.544+0000 7fa31e000640 1 -- 172.21.15.37:0/864520199 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa324005020 con 0x56216c17f300 2024-03-21T09:14:45.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.544+0000 7fa31e000640 1 -- 172.21.15.37:0/864520199 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa32400f040 con 0x56216c17f300 2024-03-21T09:14:45.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa33de5c740 10 client.26335 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:45.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa33de5c740 10 client.26335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:14:45.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa33de5c740 20 client.26335 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa33de5c740 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c1d4340 0x56216c1d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa33de5c740 4 client.26335 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:14:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56216c0c9b50 con 0x56216c1d4340 2024-03-21T09:14:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.545+0000 7fa333e00640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c1d4340 0x56216c1d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.547+0000 7fa31e000640 10 client.26335 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.547+0000 7fa333e00640 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c1d4340 0x56216c1d6760 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bc246cfd065fac9 server_cookie=13cc5a60cb521900 in_seq=0 out_seq=0 2024-03-21T09:14:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.548+0000 7fa31e000640 1 -- 172.21.15.37:0/864520199 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56216c0c9b50 con 0x56216c1d4340 2024-03-21T09:14:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.548+0000 7fa31e000640 10 client.26335 handle_command_reply: tid=0 2024-03-21T09:14:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.548+0000 7fa33de5c740 1 client.26335 shutdown 2024-03-21T09:14:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.549+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c17f300 msgr2=0x56216c16e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.549+0000 7fa33de5c740 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c17f300 0x56216c16e2e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa324009f40 tx=0x7fa324009820 comp rx=0 tx=0).stop 2024-03-21T09:14:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 shutdown_connections 2024-03-21T09:14:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa33de5c740 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56216c1d4340 0x56216c1d6760 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa33de5c740 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c17f300 0x56216c16e2e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa33de5c740 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56216c167c00 0x56216c16dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa33de5c740 1 --2- 172.21.15.37:0/864520199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56216c15f080 0x56216c16f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 >> 172.21.15.37:0/864520199 conn(0x56216c158da0 msgr2=0x56216c15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.550+0000 7fa31e000640 0 client.26335 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:14:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.551+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 shutdown_connections 2024-03-21T09:14:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.551+0000 7fa33de5c740 1 -- 172.21.15.37:0/864520199 wait complete. 2024-03-21T09:14:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.551+0000 7fa33de5c740 20 client.26335 trim_cache size 0 max 16384 2024-03-21T09:14:45.546 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:14:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.556+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3180a31e0 msgr2=0x7fa3180a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.556+0000 7fa33b200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3180a31e0 0x7fa3180a56a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa32c013e80 tx=0x7fa32c013590 comp rx=0 tx=0).stop 2024-03-21T09:14:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 msgr2=0x7fa33411a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa32002c260 tx=0x7fa32002ea00 comp rx=0 tx=0).stop 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 shutdown_connections 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa33411ad40 0x7fa3341134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa33406da20 0x7fa33411a7e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3180a31e0 0x7fa3180a56a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 --2- 172.21.15.37:0/3482989178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa33406d100 0x7fa33411a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 >> 172.21.15.37:0/3482989178 conn(0x7fa33406c7f0 msgr2=0x7fa33411bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.557+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 shutdown_connections 2024-03-21T09:14:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.558+0000 7fa33b200640 1 -- 172.21.15.37:0/3482989178 wait complete. 2024-03-21T09:14:45.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:45 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/800658831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:14:45.583 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:14:45.583 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:14:45.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:45 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/800658831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:14:45.603 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-03-21T09:14:45.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T09:14:45.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:45 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/800658831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:14:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 -- 172.21.15.37:0/2716433676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8073a40 msgr2=0x7f51d8073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 --2- 172.21.15.37:0/2716433676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8073a40 0x7f51d8073e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009900 tx=0x7f51cc02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 -- 172.21.15.37:0/2716433676 shutdown_connections 2024-03-21T09:14:45.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 --2- 172.21.15.37:0/2716433676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d81124b0 0x7f51d81148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 --2- 172.21.15.37:0/2716433676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8074360 0x7f51d8111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 --2- 172.21.15.37:0/2716433676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8073a40 0x7f51d8073e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 -- 172.21.15.37:0/2716433676 >> 172.21.15.37:0/2716433676 conn(0x7f51d80fcfb0 msgr2=0x7f51d80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.851+0000 7f51de000640 1 -- 172.21.15.37:0/2716433676 shutdown_connections 2024-03-21T09:14:45.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.852+0000 7f51de000640 1 -- 172.21.15.37:0/2716433676 wait complete. 2024-03-21T09:14:45.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.853+0000 7f51de000640 1 Processor -- start 2024-03-21T09:14:45.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.853+0000 7f51de000640 1 -- start start 2024-03-21T09:14:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51de000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 0x7f51d819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51de000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8074360 0x7f51d819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d81124b0 0x7f51d81a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51de000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8117f00 con 0x7f51d8073a40 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51de000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8117d80 con 0x7f51d81124b0 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51de000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8118080 con 0x7f51d8074360 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 0x7f51d819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 0x7f51d819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35726/0 (socket says 172.21.15.37:35726) 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51dcc00640 1 -- 172.21.15.37:0/1473563001 learned_addr learned my addr 172.21.15.37:0/1473563001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51d7e00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8074360 0x7f51d819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.854+0000 7f51dd600640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d81124b0 0x7f51d81a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d7e00640 1 -- 172.21.15.37:0/1473563001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d81124b0 msgr2=0x7f51d81a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d7e00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d81124b0 0x7f51d81a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d7e00640 1 -- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 msgr2=0x7f51d819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d7e00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 0x7f51d819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d7e00640 1 -- 172.21.15.37:0/1473563001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d81a33f0 con 0x7f51d8074360 2024-03-21T09:14:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51dcc00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 0x7f51d819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51dd600640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d81124b0 0x7f51d81a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d7e00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8074360 0x7f51d819e940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f51c400e9f0 tx=0x7f51c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d5600640 1 -- 172.21.15.37:0/1473563001 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c400ccc0 con 0x7f51d8074360 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d5600640 1 -- 172.21.15.37:0/1473563001 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51c4005020 con 0x7f51d8074360 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51d5600640 1 -- 172.21.15.37:0/1473563001 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c40049e0 con 0x7f51d8074360 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d81a36e0 con 0x7f51d8074360 2024-03-21T09:14:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.855+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d81aaf70 con 0x7f51d8074360 2024-03-21T09:14:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51d5600640 1 -- 172.21.15.37:0/1473563001 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f51c4005560 con 0x7f51d8074360 2024-03-21T09:14:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 Processor -- start 2024-03-21T09:14:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 -- start start 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66bee1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66beeade0 0x55a66beed230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66be452c0 con 0x55a66bee2260 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bdedc90 con 0x55a66bee18b0 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.859+0000 7f51e23ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66be41360 con 0x55a66beeade0 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66bee1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66bee1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41604/0 (socket says 172.21.15.37:41604) 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 -- 172.21.15.37:0/2924132456 learned_addr learned my addr 172.21.15.37:0/2924132456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51d7e00640 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66beeade0 0x55a66beed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dcc00640 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 -- 172.21.15.37:0/2924132456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66beeade0 msgr2=0x55a66beed230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66beeade0 0x55a66beed230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 -- 172.21.15.37:0/2924132456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 msgr2=0x55a66beea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 -- 172.21.15.37:0/2924132456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a66be12080 con 0x55a66bee18b0 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dcc00640 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51d5600640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51c00a3110 0x7f51c00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.860+0000 7f51dd600640 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66bee1c90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51d0011760 tx=0x7f51d0011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4738932689a88c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.861+0000 7f51d5600640 1 -- 172.21.15.37:0/1473563001 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f51c40cfa90 con 0x7f51d8074360 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.861+0000 7f51dcc00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51c00a3110 0x7f51c00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.861+0000 7f51c9c00640 1 -- 172.21.15.37:0/2924132456 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0004d20 con 0x55a66bee18b0 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.861+0000 7f51c9c00640 1 -- 172.21.15.37:0/2924132456 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51d000f9a0 con 0x55a66bee18b0 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.861+0000 7f51c9c00640 1 -- 172.21.15.37:0/2924132456 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0004590 con 0x55a66bee18b0 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 -- 172.21.15.37:0/2924132456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 msgr2=0x55a66bee1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66bee1c90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51d0011760 tx=0x7f51d0011c20 comp rx=0 tx=0).stop 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51dcc00640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51c00a3110 0x7f51c00a55d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55a66bee2660 tx=0x7f51cc005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 -- 172.21.15.37:0/2924132456 shutdown_connections 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66beeade0 0x55a66beed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 --2- 172.21.15.37:0/2924132456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66bee1c90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 -- 172.21.15.37:0/2924132456 >> 172.21.15.37:0/2924132456 conn(0x55a66bedbf80 msgr2=0x55a66bedc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 -- 172.21.15.37:0/2924132456 shutdown_connections 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 -- 172.21.15.37:0/2924132456 wait complete. 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.862+0000 7f51e23ca740 1 -- start start 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 Processor -- start 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66bee7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bee7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66beedd50 con 0x55a66beeade0 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66be010d0 con 0x55a66bee18b0 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66beeded0 con 0x55a66bee2260 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51e23ca740 10 client.0 _ll_register_callbacks cb 0x55a66be05e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:45.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66bee7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66bee7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46936/0 (socket says 172.21.15.37:46936) 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.863+0000 7f51d7e00640 1 -- 172.21.15.37:0/574659264 learned_addr learned my addr 172.21.15.37:0/574659264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51dd600640 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bee7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51d7e00640 1 -- 172.21.15.37:0/574659264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 msgr2=0x55a66beea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51d7e00640 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beea6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51d7e00640 1 -- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 msgr2=0x55a66bee7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51d7e00640 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bee7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51d7e00640 1 -- 172.21.15.37:0/574659264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a66bb593f0 con 0x55a66bee2260 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51d7e00640 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66bee7a50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f51c401e000 tx=0x7f51c4097e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22172d49b6e05a03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51be000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51be000640 1 -- 172.21.15.37:0/574659264 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c401c070 con 0x55a66bee2260 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51d000f9a0 con 0x55a66bee2260 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51be000640 1 -- 172.21.15.37:0/574659264 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f51c409bf40 con 0x55a66bee2260 2024-03-21T09:14:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51be000640 1 -- 172.21.15.37:0/574659264 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c409b440 con 0x55a66bee2260 2024-03-21T09:14:45.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51e23ca740 10 client.26347 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:45.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.864+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a66bc237d0 con 0x55a66bee2260 2024-03-21T09:14:45.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51be000640 1 -- 172.21.15.37:0/574659264 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51d000f9a0 con 0x55a66bee2260 2024-03-21T09:14:45.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51be000640 1 -- 172.21.15.37:0/574659264 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f51c40a4060 con 0x55a66bee2260 2024-03-21T09:14:45.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51e23ca740 10 client.26347 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:45.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51e23ca740 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:45.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51e23ca740 20 client.26347 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:45.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51e23ca740 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66befac20 0x55a66befd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51e23ca740 4 client.26347 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.865+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a66be52800 con 0x55a66befac20 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.866+0000 7f51dcc00640 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66befac20 0x55a66befd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.866+0000 7f51dcc00640 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66befac20 0x55a66befd010 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def59dae22b35522 server_cookie=1b6bf7ad2cf3be6f in_seq=0 out_seq=0 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.866+0000 7f51be000640 10 client.26347 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.867+0000 7f51be000640 1 -- 172.21.15.37:0/574659264 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a66be52800 con 0x55a66befac20 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.867+0000 7f51be000640 10 client.26347 handle_command_reply: tid=0 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.867+0000 7f51e23ca740 1 client.26347 shutdown 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 msgr2=0x55a66bee7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66bee7a50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f51c401e000 tx=0x7f51c4097e40 comp rx=0 tx=0).stop 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 shutdown_connections 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bee7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66bee7a50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66befac20 0x55a66befd010 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 --2- 172.21.15.37:0/574659264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 >> 172.21.15.37:0/574659264 conn(0x55a66bedbf80 msgr2=0x55a66beddcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51be000640 0 client.26347 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 shutdown_connections 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 1 -- 172.21.15.37:0/574659264 wait complete. 2024-03-21T09:14:45.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.868+0000 7f51e23ca740 20 client.26347 trim_cache size 0 max 16384 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 Processor -- start 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 -- start start 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 0x55a66bedafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66beeade0 0x55a66beea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81b7320 con 0x55a66bee2260 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81b4e20 con 0x55a66beeade0 2024-03-21T09:14:45.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.882+0000 7f51e23ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81b2ec0 con 0x55a66bee18b0 2024-03-21T09:14:45.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 0x55a66bedafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 0x55a66bedafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46948/0 (socket says 172.21.15.37:46948) 2024-03-21T09:14:45.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 -- 172.21.15.37:0/3755103115 learned_addr learned my addr 172.21.15.37:0/3755103115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dd600640 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66beeade0 0x55a66beea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51d7e00640 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 -- 172.21.15.37:0/3755103115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66beeade0 msgr2=0x55a66beea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66beeade0 0x55a66beea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 -- 172.21.15.37:0/3755103115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 msgr2=0x55a66beea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 -- 172.21.15.37:0/3755103115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a66bedcb50 con 0x55a66bee18b0 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.883+0000 7f51dcc00640 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 0x55a66bedafe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f51cc02c2d0 tx=0x7f51cc040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02b425bcf79a89d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.884+0000 7f51c9200640 1 -- 172.21.15.37:0/3755103115 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc05a070 con 0x55a66bee18b0 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.884+0000 7f51c9200640 1 -- 172.21.15.37:0/3755103115 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51cc02c910 con 0x55a66bee18b0 2024-03-21T09:14:45.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.884+0000 7f51c9200640 1 -- 172.21.15.37:0/3755103115 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc005340 con 0x55a66bee18b0 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.884+0000 7f51e23ca740 1 -- 172.21.15.37:0/3755103115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 msgr2=0x55a66bedafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.884+0000 7f51e23ca740 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 0x55a66bedafe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f51cc02c2d0 tx=0x7f51cc040000 comp rx=0 tx=0).stop 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 -- 172.21.15.37:0/3755103115 shutdown_connections 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66beeade0 0x55a66beea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bee2260 0x55a66beea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 --2- 172.21.15.37:0/3755103115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee18b0 0x55a66bedafe0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 -- 172.21.15.37:0/3755103115 >> 172.21.15.37:0/3755103115 conn(0x55a66bedbf80 msgr2=0x55a66beda8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 -- 172.21.15.37:0/3755103115 shutdown_connections 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 -- 172.21.15.37:0/3755103115 wait complete. 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.885+0000 7f51e23ca740 1 -- start start 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 Processor -- start 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beeec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66beef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bf55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81b34c0 con 0x55a66beeade0 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81b47c0 con 0x55a66bee18b0 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81b3b00 con 0x55a66bee2260 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51e23ca740 10 client.0 _ll_register_callbacks cb 0x7f51c40d32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beeec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66beef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51dcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beeec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41632/0 (socket says 172.21.15.37:41632) 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.886+0000 7f51dcc00640 1 -- 172.21.15.37:0/100954330 learned_addr learned my addr 172.21.15.37:0/100954330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dd600640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bf55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dcc00640 1 -- 172.21.15.37:0/100954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 msgr2=0x55a66beef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dcc00640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66beef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dcc00640 1 -- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 msgr2=0x55a66bf55480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:45.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dcc00640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bf55480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dcc00640 1 -- 172.21.15.37:0/100954330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a66bf559c0 con 0x55a66bee18b0 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51d7e00640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66beef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51dcc00640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beeec40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000c00 tx=0x7f51cc0543b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102d12233d4f795c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51be000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51be000640 1 -- 172.21.15.37:0/100954330 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc063060 con 0x55a66bee18b0 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.887+0000 7f51be000640 1 -- 172.21.15.37:0/100954330 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f51cc02cb80 con 0x55a66bee18b0 2024-03-21T09:14:45.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51cc02c910 con 0x55a66bee18b0 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51be000640 1 -- 172.21.15.37:0/100954330 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51cc05a030 con 0x55a66bee18b0 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 10 client.26327 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a66bf56c70 con 0x55a66bee18b0 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51be000640 1 -- 172.21.15.37:0/100954330 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51cc02c910 con 0x55a66bee18b0 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51be000640 1 -- 172.21.15.37:0/100954330 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f51cc0035f0 con 0x55a66bee18b0 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 10 client.26327 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 20 client.26327 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66bf572f0 0x55a66bf59710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 4 client.26327 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:14:45.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.888+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a66be4cb50 con 0x55a66bf572f0 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.889+0000 7f51d7e00640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66bf572f0 0x55a66bf59710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.889+0000 7f51d7e00640 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66bf572f0 0x55a66bf59710 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9415f03a0848ca8 server_cookie=89cb2a914b84d9cc in_seq=0 out_seq=0 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.889+0000 7f51be000640 10 client.26327 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.889+0000 7f51be000640 1 -- 172.21.15.37:0/100954330 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a66be4cb50 con 0x55a66bf572f0 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.889+0000 7f51be000640 10 client.26327 handle_command_reply: tid=0 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 client.26327 shutdown 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 msgr2=0x55a66beeec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beeec40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000c00 tx=0x7f51cc0543b0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 shutdown_connections 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66beeade0 0x55a66bf55480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a66bee2260 0x55a66beef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a66bf572f0 0x55a66bf59710 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 --2- 172.21.15.37:0/100954330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a66bee18b0 0x55a66beeec40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51be000640 0 client.26327 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 >> 172.21.15.37:0/100954330 conn(0x55a66bedbf80 msgr2=0x55a66bee2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 shutdown_connections 2024-03-21T09:14:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 1 -- 172.21.15.37:0/100954330 wait complete. 2024-03-21T09:14:45.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.890+0000 7f51e23ca740 20 client.26327 trim_cache size 0 max 16384 2024-03-21T09:14:45.888 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51c00a3110 msgr2=0x7f51c00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51c00a3110 0x7f51c00a55d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55a66bee2660 tx=0x7f51cc005a20 comp rx=0 tx=0).stop 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8074360 msgr2=0x7f51d819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8074360 0x7f51d819e940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f51c400e9f0 tx=0x7f51c400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 shutdown_connections 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51c00a3110 0x7f51c00a55d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d81124b0 0x7f51d81a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d8074360 0x7f51d819e940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 --2- 172.21.15.37:0/1473563001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d8073a40 0x7f51d819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:45.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 >> 172.21.15.37:0/1473563001 conn(0x7f51d80fcfb0 msgr2=0x7f51d80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.891+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 shutdown_connections 2024-03-21T09:14:45.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:45.892+0000 7f51de000640 1 -- 172.21.15.37:0/1473563001 wait complete. 2024-03-21T09:14:46.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:46 smithi071 ceph-mon[36597]: pgmap v80: 129 pgs: 129 active+clean; 219 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 501 KiB/s wr, 159 op/s 2024-03-21T09:14:46.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:46 smithi037 ceph-mon[31830]: pgmap v80: 129 pgs: 129 active+clean; 219 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 501 KiB/s wr, 159 op/s 2024-03-21T09:14:46.667 DEBUG:teuthology.parallel:result is None 2024-03-21T09:14:46.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T09:14:46.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:46 smithi053 ceph-mon[36169]: pgmap v80: 129 pgs: 129 active+clean; 219 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 501 KiB/s wr, 159 op/s 2024-03-21T09:14:46.786 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T09:14:46.786 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-21T09:14:46.798 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:14:46.818 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-21T09:14:46.829 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-21T09:14:46.829 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T09:14:46.829 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:14:46.829 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-21T09:14:46.911 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:14:46.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:14:46.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:14:46.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-21T09:14:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.132+0000 7f92a2e00640 1 -- 172.21.15.37:0/4259953926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c107560 msgr2=0x7f929c107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.132+0000 7f92a2e00640 1 --2- 172.21.15.37:0/4259953926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c107560 0x7f929c107940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f928c009930 tx=0x7f928c02bc20 comp rx=0 tx=0).stop 2024-03-21T09:14:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 -- 172.21.15.37:0/4259953926 shutdown_connections 2024-03-21T09:14:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 --2- 172.21.15.37:0/4259953926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 --2- 172.21.15.37:0/4259953926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f929c1014a0 0x7f929c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 --2- 172.21.15.37:0/4259953926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c107560 0x7f929c107940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 -- 172.21.15.37:0/4259953926 >> 172.21.15.37:0/4259953926 conn(0x7f929c0fcfb0 msgr2=0x7f929c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 -- 172.21.15.37:0/4259953926 shutdown_connections 2024-03-21T09:14:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.133+0000 7f92a2e00640 1 -- 172.21.15.37:0/4259953926 wait complete. 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 Processor -- start 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 -- start start 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c1014a0 0x7f929c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f929c107560 0x7f929c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c111890 con 0x7f929c107560 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c111710 con 0x7f929c101e60 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c111a10 con 0x7f929c1014a0 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c1014a0 0x7f929c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41636/0 (socket says 172.21.15.37:41636) 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c1014a0 0x7f929c199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46976/0 (socket says 172.21.15.37:46976) 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a1000640 1 -- 172.21.15.37:0/1126163557 learned_addr learned my addr 172.21.15.37:0/1126163557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.134+0000 7f92a2400640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f929c107560 0x7f929c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a1000640 1 -- 172.21.15.37:0/1126163557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c1014a0 msgr2=0x7f929c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a1000640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c1014a0 0x7f929c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a1000640 1 -- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f929c107560 msgr2=0x7f929c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a1000640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f929c107560 0x7f929c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a1000640 1 -- 172.21.15.37:0/1126163557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c19ef10 con 0x7f929c101e60 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a1000640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c19a460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f928800e9f0 tx=0x7f928800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f929a000640 1 -- 172.21.15.37:0/1126163557 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928800cd80 con 0x7f929c101e60 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c19f200 con 0x7f929c101e60 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f929a000640 1 -- 172.21.15.37:0/1126163557 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9288010910 con 0x7f929c101e60 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.135+0000 7f929a000640 1 -- 172.21.15.37:0/1126163557 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92880188f0 con 0x7f929c101e60 2024-03-21T09:14:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.136+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c1a6a90 con 0x7f929c101e60 2024-03-21T09:14:47.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.138+0000 7f92a71b6740 1 Processor -- start 2024-03-21T09:14:47.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 -- start start 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821e1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c82145eb0 con 0x561c821ea790 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c820eea60 con 0x561c821e1bb0 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a71b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c82141f50 con 0x561c821e1290 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821e1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821ea250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41648/0 (socket says 172.21.15.37:41648) 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a1a00640 1 -- 172.21.15.37:0/933803398 learned_addr learned my addr 172.21.15.37:0/933803398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f92a2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821e1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46986/0 (socket says 172.21.15.37:46986) 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.139+0000 7f929a000640 1 -- 172.21.15.37:0/1126163557 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9288018ad0 con 0x7f929c101e60 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f92a1a00640 1 -- 172.21.15.37:0/933803398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 msgr2=0x561c821e1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f92a1a00640 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821e1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f92a1a00640 1 -- 172.21.15.37:0/933803398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 msgr2=0x561c821ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f92a1a00640 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f92a1a00640 1 -- 172.21.15.37:0/933803398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c81f227e0 con 0x561c821e1bb0 2024-03-21T09:14:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f92a1a00640 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821ea250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f928c005ab0 tx=0x7f928c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4cbe16bf76cf68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f9292a00640 1 -- 172.21.15.37:0/933803398 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c045070 con 0x561c821e1bb0 2024-03-21T09:14:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f9292a00640 1 -- 172.21.15.37:0/933803398 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f928c02c640 con 0x561c821e1bb0 2024-03-21T09:14:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.140+0000 7f9292a00640 1 -- 172.21.15.37:0/933803398 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c004b00 con 0x561c821e1bb0 2024-03-21T09:14:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 -- 172.21.15.37:0/933803398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 msgr2=0x561c821ea250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821ea250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f928c005ab0 tx=0x7f928c005e40 comp rx=0 tx=0).stop 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f929a000640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92800a3090 0x7f92800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 -- 172.21.15.37:0/933803398 shutdown_connections 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821ea250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 --2- 172.21.15.37:0/933803398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821e1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f929a000640 1 -- 172.21.15.37:0/1126163557 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f92880cff30 con 0x7f929c101e60 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 -- 172.21.15.37:0/933803398 >> 172.21.15.37:0/933803398 conn(0x561c821db9c0 msgr2=0x561c821dbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 -- 172.21.15.37:0/933803398 shutdown_connections 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a71b6740 1 -- 172.21.15.37:0/933803398 wait complete. 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.141+0000 7f92a2400640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92800a3090 0x7f92800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.142+0000 7f92a71b6740 1 -- start start 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.142+0000 7f92a71b6740 1 Processor -- start 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.142+0000 7f92a2400640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92800a3090 0x7f92800a5550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9294005fd0 tx=0x7f9294009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.142+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.142+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a71b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c821ed520 con 0x561c821ea790 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a71b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c82101eb0 con 0x561c821e1bb0 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a71b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c821ed6a0 con 0x561c821e1290 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a71b6740 10 client.0 _ll_register_callbacks cb 0x561c820f6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41652/0 (socket says 172.21.15.37:41652) 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 -- 172.21.15.37:0/3938769212 learned_addr learned my addr 172.21.15.37:0/3938769212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a2400640 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 -- 172.21.15.37:0/3938769212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 msgr2=0x561c821ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 -- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 msgr2=0x561c821e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821e7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 -- 172.21.15.37:0/3938769212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c81f82f70 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f92a1000640 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821e74e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92880cd8e0 tx=0x7f9288011f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c9db8235b10c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.143+0000 7f9287400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f9287400640 1 -- 172.21.15.37:0/3938769212 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92880a4990 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f9287400640 1 -- 172.21.15.37:0/3938769212 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f92880cd560 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f928c02c640 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f9287400640 1 -- 172.21.15.37:0/3938769212 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92880986a0 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f92a71b6740 10 client.26339 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561c81e1a170 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f9287400640 1 -- 172.21.15.37:0/3938769212 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f928c02c640 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f9287400640 1 -- 172.21.15.37:0/3938769212 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9288096cc0 con 0x561c821e1bb0 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f92a71b6740 10 client.26339 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.144+0000 7f92a71b6740 10 client.26339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f92a71b6740 20 client.26339 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c821fa780 0x561c821fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f92a71b6740 4 client.26339 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c821533f0 con 0x561c821fa780 2024-03-21T09:14:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f92a1a00640 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c821fa780 0x561c821fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f92a1a00640 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c821fa780 0x561c821fcb70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0bad9daf38170e6 server_cookie=d618785368a8eebc in_seq=0 out_seq=0 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.145+0000 7f9287400640 10 client.26339 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f9287400640 1 -- 172.21.15.37:0/3938769212 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561c821533f0 con 0x561c821fa780 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f9287400640 10 client.26339 handle_command_reply: tid=0 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 client.26339 shutdown 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 msgr2=0x561c821e74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821e74e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92880cd8e0 tx=0x7f9288011f60 comp rx=0 tx=0).stop 2024-03-21T09:14:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 shutdown_connections 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821ea790 0x561c821e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c821fa780 0x561c821fcb70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1bb0 0x561c821e74e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3938769212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1290 0x561c821ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f9287400640 0 client.26339 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.146+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 >> 172.21.15.37:0/3938769212 conn(0x561c821db9c0 msgr2=0x561c821dd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.147+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 shutdown_connections 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.147+0000 7f92a71b6740 1 -- 172.21.15.37:0/3938769212 wait complete. 2024-03-21T09:14:47.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.147+0000 7f92a71b6740 20 client.26339 trim_cache size 0 max 16384 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 Processor -- start 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 -- start start 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1290 0x561c821da900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 0x561c821e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821ea790 0x561c821ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1b2e60 con 0x561c821e1290 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1b0960 con 0x561c821ea790 2024-03-21T09:14:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.164+0000 7f92a71b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1aea00 con 0x561c821e1bb0 2024-03-21T09:14:47.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1290 0x561c821da900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 0x561c821e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a2400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821ea790 0x561c821ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 0x561c821e9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47004/0 (socket says 172.21.15.37:47004) 2024-03-21T09:14:47.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 -- 172.21.15.37:0/2471876742 learned_addr learned my addr 172.21.15.37:0/2471876742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 -- 172.21.15.37:0/2471876742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821ea790 msgr2=0x561c821ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821ea790 0x561c821ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 -- 172.21.15.37:0/2471876742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1290 msgr2=0x561c821da900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1290 0x561c821da900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 -- 172.21.15.37:0/2471876742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c821e30e0 con 0x561c821e1bb0 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a2400640 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821ea790 0x561c821ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1a00640 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1290 0x561c821da900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.165+0000 7f92a1000640 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 0x561c821e9c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f928809bfc0 tx=0x7f928809be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4eccf755aaa941 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f9292000640 1 -- 172.21.15.37:0/2471876742 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92880a4990 con 0x561c821e1bb0 2024-03-21T09:14:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f9292000640 1 -- 172.21.15.37:0/2471876742 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92880cd560 con 0x561c821e1bb0 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f9292000640 1 -- 172.21.15.37:0/2471876742 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288098840 con 0x561c821e1bb0 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 -- 172.21.15.37:0/2471876742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 msgr2=0x561c821e9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 0x561c821e9c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f928809bfc0 tx=0x7f928809be00 comp rx=0 tx=0).stop 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 -- 172.21.15.37:0/2471876742 shutdown_connections 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821ea790 0x561c821ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821e1bb0 0x561c821e9c10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 --2- 172.21.15.37:0/2471876742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1290 0x561c821da900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 -- 172.21.15.37:0/2471876742 >> 172.21.15.37:0/2471876742 conn(0x561c821db9c0 msgr2=0x561c821da1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 -- 172.21.15.37:0/2471876742 shutdown_connections 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.166+0000 7f92a71b6740 1 -- 172.21.15.37:0/2471876742 wait complete. 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 -- start start 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 Processor -- start 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 0x561c821ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1bb0 0x561c821efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821ea790 0x561c821f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1af000 con 0x561c821e1bb0 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1b0300 con 0x561c821e1290 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.167+0000 7f92a71b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1af640 con 0x561c821ea790 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a71b6740 10 client.0 _ll_register_callbacks cb 0x7f928800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 0x561c821ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1bb0 0x561c821efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 0x561c821ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41664/0 (socket says 172.21.15.37:41664) 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 -- 172.21.15.37:0/3241123640 learned_addr learned my addr 172.21.15.37:0/3241123640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a2400640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821ea790 0x561c821f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 -- 172.21.15.37:0/3241123640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821ea790 msgr2=0x561c821f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821ea790 0x561c821f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 -- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1bb0 msgr2=0x561c821efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1bb0 0x561c821efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 -- 172.21.15.37:0/3241123640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c821e55a0 con 0x561c821e1290 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1000640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1bb0 0x561c821efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f92a1a00640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 0x561c821ef8c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f928c0047d0 tx=0x7f928c004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99abd54f5a098db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.168+0000 7f9287400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:47.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f9287400640 1 -- 172.21.15.37:0/3241123640 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c045040 con 0x561c821e1290 2024-03-21T09:14:47.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f9287400640 1 -- 172.21.15.37:0/3241123640 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f928c02ca80 con 0x561c821e1290 2024-03-21T09:14:47.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92880cd560 con 0x561c821e1290 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f9287400640 1 -- 172.21.15.37:0/3241123640 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f928c05e050 con 0x561c821e1290 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f92a71b6740 10 client.26345 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561c82257120 con 0x561c821e1290 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f9287400640 1 -- 172.21.15.37:0/3241123640 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92880cd560 con 0x561c821e1290 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f9287400640 1 -- 172.21.15.37:0/3241123640 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f928c04a450 con 0x561c821e1290 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f92a71b6740 10 client.26345 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f92a71b6740 10 client.26345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.169+0000 7f92a71b6740 20 client.26345 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.170+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c822574b0 0x561c822598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.170+0000 7f92a71b6740 4 client.26345 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:14:47.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.170+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561c8214d740 con 0x561c822574b0 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.170+0000 7f92a1000640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c822574b0 0x561c822598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.170+0000 7f92a1000640 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c822574b0 0x561c822598a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51935228f7a21520 server_cookie=44a28b5d844e6e87 in_seq=0 out_seq=0 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.170+0000 7f9287400640 10 client.26345 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.171+0000 7f9287400640 1 -- 172.21.15.37:0/3241123640 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561c8214d740 con 0x561c822574b0 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.171+0000 7f9287400640 10 client.26345 handle_command_reply: tid=0 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.171+0000 7f92a71b6740 1 client.26345 shutdown 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 msgr2=0x561c821ef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 0x561c821ef8c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f928c0047d0 tx=0x7f928c004800 comp rx=0 tx=0).stop 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 shutdown_connections 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c821ea790 0x561c821f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c821e1bb0 0x561c821efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561c822574b0 0x561c822598a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 --2- 172.21.15.37:0/3241123640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c821e1290 0x561c821ef8c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 >> 172.21.15.37:0/3241123640 conn(0x561c821db9c0 msgr2=0x561c821e2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f9287400640 0 client.26345 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.172+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 shutdown_connections 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.173+0000 7f92a71b6740 1 -- 172.21.15.37:0/3241123640 wait complete. 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.173+0000 7f92a71b6740 20 client.26345 trim_cache size 0 max 16384 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "fe11a699-210e-4c48-9052-dece200b0334", 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:14:47.170 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:14:47.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.173+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92800a3090 msgr2=0x7f92800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.173+0000 7f92a2e00640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92800a3090 0x7f92800a5550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9294005fd0 tx=0x7f9294009500 comp rx=0 tx=0).stop 2024-03-21T09:14:47.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.173+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 msgr2=0x7f929c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.173+0000 7f92a2e00640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c19a460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f928800e9f0 tx=0x7f928800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 shutdown_connections 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f929c107560 0x7f929c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92800a3090 0x7f92800a5550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f929c101e60 0x7f929c19a460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 --2- 172.21.15.37:0/1126163557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f929c1014a0 0x7f929c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 >> 172.21.15.37:0/1126163557 conn(0x7f929c0fcfb0 msgr2=0x7f929c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 shutdown_connections 2024-03-21T09:14:47.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.174+0000 7f92a2e00640 1 -- 172.21.15.37:0/1126163557 wait complete. 2024-03-21T09:14:47.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:14:47.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:14:47.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.395+0000 7f8add800640 1 -- 172.21.15.37:0/3186628603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 msgr2=0x7f8ad810b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.395+0000 7f8add800640 1 --2- 172.21.15.37:0/3186628603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 0x7f8ad810b430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009900 tx=0x7f8ac402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 -- 172.21.15.37:0/3186628603 shutdown_connections 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 --2- 172.21.15.37:0/3186628603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 --2- 172.21.15.37:0/3186628603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 0x7f8ad810b430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 --2- 172.21.15.37:0/3186628603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8102630 0x7f8ad8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 -- 172.21.15.37:0/3186628603 >> 172.21.15.37:0/3186628603 conn(0x7f8ad8077130 msgr2=0x7f8ad80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 -- 172.21.15.37:0/3186628603 shutdown_connections 2024-03-21T09:14:47.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.396+0000 7f8add800640 1 -- 172.21.15.37:0/3186628603 wait complete. 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.397+0000 7f8add800640 1 Processor -- start 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.397+0000 7f8add800640 1 -- start start 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8add800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8102630 0x7f8ad8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8add800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 0x7f8ad819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8add800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8add800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad8111800 con 0x7f8ad8102630 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8add800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad8111680 con 0x7f8ad8102f50 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8add800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad8111980 con 0x7f8ad810ba50 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8adce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 0x7f8ad819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8adce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47036/0 (socket says 172.21.15.37:47036) 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8adce00640 1 -- 172.21.15.37:0/3714438923 learned_addr learned my addr 172.21.15.37:0/3714438923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8102630 0x7f8ad8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8adce00640 1 -- 172.21.15.37:0/3714438923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 msgr2=0x7f8ad819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.398+0000 7f8adce00640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 0x7f8ad819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8adce00640 1 -- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8102630 msgr2=0x7f8ad8199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8adce00640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8102630 0x7f8ad8199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8adce00640 1 -- 172.21.15.37:0/3714438923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad819ef50 con 0x7f8ad810ba50 2024-03-21T09:14:47.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8adce00640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad819e820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000efd0 tx=0x7f8ad000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8ad4c00640 1 -- 172.21.15.37:0/3714438923 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0003b70 con 0x7f8ad810ba50 2024-03-21T09:14:47.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8ad4c00640 1 -- 172.21.15.37:0/3714438923 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad0003d10 con 0x7f8ad810ba50 2024-03-21T09:14:47.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8ad4c00640 1 -- 172.21.15.37:0/3714438923 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00109f0 con 0x7f8ad810ba50 2024-03-21T09:14:47.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad819f240 con 0x7f8ad810ba50 2024-03-21T09:14:47.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.399+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a6ad0 con 0x7f8ad810ba50 2024-03-21T09:14:47.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.402+0000 7f8ae1b7d740 1 Processor -- start 2024-03-21T09:14:47.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.402+0000 7f8ad4c00640 1 -- 172.21.15.37:0/3714438923 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8ad0004020 con 0x7f8ad810ba50 2024-03-21T09:14:47.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 -- start start 2024-03-21T09:14:47.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa741b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa742570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa74d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633aa6a6250 con 0x5633aa74b460 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633aa64ea30 con 0x5633aa7420f0 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633aa6a22f0 con 0x5633aa742dd0 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa742570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa742570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41700/0 (socket says 172.21.15.37:41700) 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa741b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ad7e00640 1 -- 172.21.15.37:0/3985200879 learned_addr learned my addr 172.21.15.37:0/3985200879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.403+0000 7f8ad7400640 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa74d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8ad7e00640 1 -- 172.21.15.37:0/3985200879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 msgr2=0x5633aa741b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa741b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8ad7e00640 1 -- 172.21.15.37:0/3985200879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 msgr2=0x5633aa74d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa74d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8ad7e00640 1 -- 172.21.15.37:0/3985200879 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633aa639690 con 0x5633aa7420f0 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa742570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009cf0 tx=0x7f8ac8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2651f769058240e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8acd600640 1 -- 172.21.15.37:0/3985200879 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004230 con 0x5633aa7420f0 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8acd600640 1 -- 172.21.15.37:0/3985200879 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ac80043d0 con 0x5633aa7420f0 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.404+0000 7f8acd600640 1 -- 172.21.15.37:0/3985200879 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8019900 con 0x5633aa7420f0 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ad4c00640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8abc0a3090 0x7f8abc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ad4c00640 1 -- 172.21.15.37:0/3714438923 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8ad00cffc0 con 0x7f8ad810ba50 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/3985200879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 msgr2=0x5633aa742570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa742570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009cf0 tx=0x7f8ac8011810 comp rx=0 tx=0).stop 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ad7400640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8abc0a3090 0x7f8abc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/3985200879 shutdown_connections 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa74d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa742570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/3985200879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa741b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/3985200879 >> 172.21.15.37:0/3985200879 conn(0x5633aa709200 msgr2=0x5633aa73cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/3985200879 shutdown_connections 2024-03-21T09:14:47.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.405+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/3985200879 wait complete. 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.406+0000 7f8ad7400640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8abc0a3090 0x7f8abc0a5550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8ac402efd0 tx=0x7f8ac4005fd0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.406+0000 7f8ae1b7d740 1 -- start start 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.406+0000 7f8ae1b7d740 1 Processor -- start 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa747e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa748380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633aa74e2d0 con 0x5633aa74b460 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633aa661e80 con 0x5633aa7420f0 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633aa74e450 con 0x5633aa742dd0 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ae1b7d740 10 client.0 _ll_register_callbacks cb 0x5633aa674130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa747e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41702/0 (socket says 172.21.15.37:41702) 2024-03-21T09:14:47.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa747e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47050/0 (socket says 172.21.15.37:47050) 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 -- 172.21.15.37:0/1931634426 learned_addr learned my addr 172.21.15.37:0/1931634426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8adce00640 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa748380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 -- 172.21.15.37:0/1931634426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 msgr2=0x5633aa747e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa747e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 -- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 msgr2=0x5633aa748380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa748380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.407+0000 7f8ad7e00640 1 -- 172.21.15.37:0/1931634426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633aa4847d0 con 0x5633aa7420f0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74aab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80195c0 tx=0x7f8ac80038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8c626b4af1847fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ac2000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ac2000640 1 -- 172.21.15.37:0/1931634426 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8027070 con 0x5633aa7420f0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ac80043d0 con 0x5633aa7420f0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ac2000640 1 -- 172.21.15.37:0/1931634426 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8ac8005240 con 0x5633aa7420f0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ac2000640 1 -- 172.21.15.37:0/1931634426 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac8033050 con 0x5633aa7420f0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ae1b7d740 10 client.26357 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.408+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5633aa3ba3f0 con 0x5633aa7420f0 2024-03-21T09:14:47.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ac2000640 1 -- 172.21.15.37:0/1931634426 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ac80043d0 con 0x5633aa7420f0 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ac2000640 1 -- 172.21.15.37:0/1931634426 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8ac8004c30 con 0x5633aa7420f0 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ae1b7d740 10 client.26357 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ae1b7d740 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ae1b7d740 20 client.26357 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa75b250 0x5633aa75d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ae1b7d740 4 client.26357 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633aa6b3790 con 0x5633aa75b250 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.409+0000 7f8ad7400640 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa75b250 0x5633aa75d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.410+0000 7f8ad7400640 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa75b250 0x5633aa75d640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b56c14e4c0ab27 server_cookie=fc048da1adceab79 in_seq=0 out_seq=0 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.410+0000 7f8ac2000640 10 client.26357 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ac2000640 1 -- 172.21.15.37:0/1931634426 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5633aa6b3790 con 0x5633aa75b250 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ac2000640 10 client.26357 handle_command_reply: tid=0 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 client.26357 shutdown 2024-03-21T09:14:47.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 msgr2=0x5633aa74aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74aab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80195c0 tx=0x7f8ac80038a0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 shutdown_connections 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa74b460 0x5633aa748380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa747e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa75b250 0x5633aa75d640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1931634426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74aab0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ac2000640 0 client.26357 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 >> 172.21.15.37:0/1931634426 conn(0x5633aa709200 msgr2=0x5633aa73e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 shutdown_connections 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1931634426 wait complete. 2024-03-21T09:14:47.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.411+0000 7f8ae1b7d740 20 client.26357 trim_cache size 0 max 16384 2024-03-21T09:14:47.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 Processor -- start 2024-03-21T09:14:47.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 -- start start 2024-03-21T09:14:47.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa742dd0 0x5633aa74a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 0x5633aa74aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81b2f10 con 0x5633aa742dd0 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81b0a10 con 0x5633aa7420f0 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.428+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81aeab0 con 0x5633aa74b460 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8adce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 0x5633aa74aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41710/0 (socket says 172.21.15.37:41710) 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa742dd0 0x5633aa74a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 -- 172.21.15.37:0/616577215 learned_addr learned my addr 172.21.15.37:0/616577215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8adce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 0x5633aa74aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47058/0 (socket says 172.21.15.37:47058) 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 -- 172.21.15.37:0/616577215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 msgr2=0x5633aa74aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 0x5633aa74aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 -- 172.21.15.37:0/616577215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa742dd0 msgr2=0x5633aa74a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa742dd0 0x5633aa74a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 -- 172.21.15.37:0/616577215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633aa37a0f0 con 0x5633aa7420f0 2024-03-21T09:14:47.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8adce00640 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 0x5633aa74aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.429+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74c850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009c40 tx=0x7f8ac8005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930ebf5b0e3143b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8accc00640 1 -- 172.21.15.37:0/616577215 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8030070 con 0x5633aa7420f0 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8accc00640 1 -- 172.21.15.37:0/616577215 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8003910 con 0x5633aa7420f0 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8accc00640 1 -- 172.21.15.37:0/616577215 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8027070 con 0x5633aa7420f0 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/616577215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 msgr2=0x5633aa74c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74c850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009c40 tx=0x7f8ac8005970 comp rx=0 tx=0).stop 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/616577215 shutdown_connections 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa74b460 0x5633aa74aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa742dd0 0x5633aa74a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/616577215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa7420f0 0x5633aa74c850 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.430+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/616577215 >> 172.21.15.37:0/616577215 conn(0x5633aa709200 msgr2=0x5633aa73c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.431+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/616577215 shutdown_connections 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.431+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/616577215 wait complete. 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.431+0000 7f8ae1b7d740 1 -- start start 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.431+0000 7f8ae1b7d740 1 Processor -- start 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.431+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa7420f0 0x5633aa74f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.431+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa74f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ae1b7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa74b460 0x5633aa7b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81af0b0 con 0x5633aa7420f0 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81b03b0 con 0x5633aa74b460 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ae1b7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81af6f0 con 0x5633aa742dd0 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ae1b7d740 10 client.0 _ll_register_callbacks cb 0x7f8ac8009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:14:47.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa7420f0 0x5633aa74f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa74f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa74f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47062/0 (socket says 172.21.15.37:47062) 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa7420f0 0x5633aa74f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35860/0 (socket says 172.21.15.37:35860) 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 -- 172.21.15.37:0/1345182678 learned_addr learned my addr 172.21.15.37:0/1345182678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8adce00640 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa74b460 0x5633aa7b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 -- 172.21.15.37:0/1345182678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa74b460 msgr2=0x5633aa7b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa74b460 0x5633aa7b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 -- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa7420f0 msgr2=0x5633aa74f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa7420f0 0x5633aa74f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 -- 172.21.15.37:0/1345182678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633aa7b5eb0 con 0x5633aa742dd0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ad7400640 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa74f770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ac402ce00 tx=0x7f8ac402bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c24bc128edc0e6fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.432+0000 7f8ac2000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ac2000640 1 -- 172.21.15.37:0/1345182678 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4059070 con 0x5633aa742dd0 2024-03-21T09:14:47.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ac8003910 con 0x5633aa742dd0 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ac2000640 1 -- 172.21.15.37:0/1345182678 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8ac40048c0 con 0x5633aa742dd0 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ac2000640 1 -- 172.21.15.37:0/1345182678 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac4065050 con 0x5633aa742dd0 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ae1b7d740 10 client.26389 fetch_fsmap learned FSMap version 26 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5633aa7b71f0 con 0x5633aa742dd0 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ac2000640 1 -- 172.21.15.37:0/1345182678 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8003910 con 0x5633aa742dd0 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ac2000640 1 -- 172.21.15.37:0/1345182678 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8ac4052050 con 0x5633aa742dd0 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ae1b7d740 10 client.26389 fetch_fsmap finished waiting for FSMap version 26 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ae1b7d740 10 client.26389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.433+0000 7f8ae1b7d740 20 client.26389 populate_metadata read hostname 'smithi037' 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.434+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa7b7960 0x5633aa7b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.434+0000 7f8ae1b7d740 4 client.26389 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.434+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633aa6adae0 con 0x5633aa7b7960 2024-03-21T09:14:47.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.434+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa7b7960 0x5633aa7b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.434+0000 7f8ad7e00640 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa7b7960 0x5633aa7b9d80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e806a49c79c7c6 server_cookie=322a0e3a0593e6 in_seq=0 out_seq=0 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.434+0000 7f8ac2000640 10 client.26389 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ac2000640 1 -- 172.21.15.37:0/1345182678 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5633aa6adae0 con 0x5633aa7b7960 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ac2000640 10 client.26389 handle_command_reply: tid=0 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 client.26389 shutdown 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 msgr2=0x5633aa74f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa74f770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ac402ce00 tx=0x7f8ac402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 shutdown_connections 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5633aa7b7960 0x5633aa7b9d80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5633aa74b460 0x5633aa7b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ac2000640 0 client.26389 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:14:47.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633aa742dd0 0x5633aa74f770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 --2- 172.21.15.37:0/1345182678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633aa7420f0 0x5633aa74f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 >> 172.21.15.37:0/1345182678 conn(0x5633aa709200 msgr2=0x5633aa73b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 shutdown_connections 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 1 -- 172.21.15.37:0/1345182678 wait complete. 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.435+0000 7f8ae1b7d740 20 client.26389 trim_cache size 0 max 16384 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "fe11a699-210e-4c48-9052-dece200b0334": { 2024-03-21T09:14:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:14:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "fe11a699-210e-4c48-9052-dece200b0334", 2024-03-21T09:14:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:14:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:14:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:14:47.435 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:14:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8abc0a3090 msgr2=0x7f8abc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8abc0a3090 0x7f8abc0a5550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8ac402efd0 tx=0x7f8ac4005fd0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 msgr2=0x7f8ad819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad819e820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000efd0 tx=0x7f8ad000c6a0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 shutdown_connections 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ad810ba50 0x7f8ad819e820 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8abc0a3090 0x7f8abc0a5550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad8102f50 0x7f8ad819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 --2- 172.21.15.37:0/3714438923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8102630 0x7f8ad8199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 >> 172.21.15.37:0/3714438923 conn(0x7f8ad8077130 msgr2=0x7f8ad8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 shutdown_connections 2024-03-21T09:14:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:14:47.436+0000 7f8add800640 1 -- 172.21.15.37:0/3714438923 wait complete. 2024-03-21T09:14:47.447 INFO:tasks.cephfs.filesystem:scrub status for tag:fe11a699-210e-4c48-9052-dece200b0334 - {'path': '/', 'tag': 'fe11a699-210e-4c48-9052-dece200b0334', 'options': 'recursive,force'} 2024-03-21T09:14:48.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:48 smithi037 ceph-mon[31830]: pgmap v81: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 714 KiB/s wr, 230 op/s 2024-03-21T09:14:48.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:48 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:14:48.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:48 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:48.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:48 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:14:48.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:48 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:48 smithi071 ceph-mon[36597]: pgmap v81: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 714 KiB/s wr, 230 op/s 2024-03-21T09:14:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:48 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:14:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:48 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:48 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:14:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:48 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:48.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:48 smithi053 ceph-mon[36169]: pgmap v81: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 714 KiB/s wr, 230 op/s 2024-03-21T09:14:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:48 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:14:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:48 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:48 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:14:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:48 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:14:49.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:49 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:14:49.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:49 smithi071 ceph-mon[36597]: pgmap v82: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 463 KiB/s wr, 151 op/s 2024-03-21T09:14:49.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:49 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:14:49.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:49 smithi037 ceph-mon[31830]: pgmap v82: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 463 KiB/s wr, 151 op/s 2024-03-21T09:14:49.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:49 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:14:49.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:49 smithi053 ceph-mon[36169]: pgmap v82: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 463 KiB/s wr, 151 op/s 2024-03-21T09:14:52.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:52 smithi037 ceph-mon[31830]: pgmap v83: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 463 KiB/s wr, 151 op/s 2024-03-21T09:14:52.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:52 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:14:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:52 smithi053 ceph-mon[36169]: pgmap v83: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 463 KiB/s wr, 151 op/s 2024-03-21T09:14:52.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:52 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:14:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:52 smithi071 ceph-mon[36597]: pgmap v83: 129 pgs: 129 active+clean; 24 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 463 KiB/s wr, 151 op/s 2024-03-21T09:14:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:52 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:14:54.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:54 smithi037 ceph-mon[31830]: pgmap v84: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 464 KiB/s wr, 164 op/s 2024-03-21T09:14:54.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:54 smithi053 ceph-mon[36169]: pgmap v84: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 464 KiB/s wr, 164 op/s 2024-03-21T09:14:54.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:54 smithi071 ceph-mon[36597]: pgmap v84: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 464 KiB/s wr, 164 op/s 2024-03-21T09:14:56.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:56 smithi037 ceph-mon[31830]: pgmap v85: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 214 KiB/s wr, 83 op/s 2024-03-21T09:14:56.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:56 smithi053 ceph-mon[36169]: pgmap v85: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 214 KiB/s wr, 83 op/s 2024-03-21T09:14:56.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:56 smithi071 ceph-mon[36597]: pgmap v85: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 214 KiB/s wr, 83 op/s 2024-03-21T09:14:58.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:14:58 smithi053 ceph-mon[36169]: pgmap v86: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 220 KiB/s wr, 87 op/s 2024-03-21T09:14:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:14:58 smithi071 ceph-mon[36597]: pgmap v86: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 220 KiB/s wr, 87 op/s 2024-03-21T09:14:58.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:14:58 smithi037 ceph-mon[31830]: pgmap v86: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 220 KiB/s wr, 87 op/s 2024-03-21T09:15:00.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:00 smithi053 ceph-mon[36169]: pgmap v87: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T09:15:00.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:00 smithi071 ceph-mon[36597]: pgmap v87: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T09:15:00.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:00 smithi037 ceph-mon[31830]: pgmap v87: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T09:15:02.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:02 smithi053 ceph-mon[36169]: pgmap v88: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T09:15:02.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:02 smithi071 ceph-mon[36597]: pgmap v88: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T09:15:02.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:02 smithi037 ceph-mon[31830]: pgmap v88: 129 pgs: 129 active+clean; 20 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-21T09:15:04.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:04 smithi053 ceph-mon[36169]: pgmap v89: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 33 op/s 2024-03-21T09:15:04.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:04 smithi071 ceph-mon[36597]: pgmap v89: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 33 op/s 2024-03-21T09:15:04.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:04 smithi037 ceph-mon[31830]: pgmap v89: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 33 op/s 2024-03-21T09:15:06.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:06 smithi071 ceph-mon[36597]: pgmap v90: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-03-21T09:15:06.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:06 smithi037 ceph-mon[31830]: pgmap v90: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-03-21T09:15:06.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:06 smithi053 ceph-mon[36169]: pgmap v90: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-03-21T09:15:08.493 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:08 smithi053 ceph-mon[36169]: pgmap v91: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 23 op/s 2024-03-21T09:15:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:08 smithi071 ceph-mon[36597]: pgmap v91: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 23 op/s 2024-03-21T09:15:08.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:08 smithi037 ceph-mon[31830]: pgmap v91: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 23 op/s 2024-03-21T09:15:10.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:10 smithi071 ceph-mon[36597]: pgmap v92: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:15:10.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:10 smithi037 ceph-mon[31830]: pgmap v92: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:15:10.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:10 smithi053 ceph-mon[36169]: pgmap v92: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:15:12.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:12 smithi071 ceph-mon[36597]: pgmap v93: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:15:12.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:12 smithi037 ceph-mon[31830]: pgmap v93: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:15:12.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:12 smithi053 ceph-mon[36169]: pgmap v93: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:15:14.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:14 smithi071 ceph-mon[36597]: pgmap v94: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:15:14.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:14 smithi037 ceph-mon[31830]: pgmap v94: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:15:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:14 smithi053 ceph-mon[36169]: pgmap v94: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:15:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:16 smithi071 ceph-mon[36597]: pgmap v95: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:15:16.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:16 smithi037 ceph-mon[31830]: pgmap v95: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:15:16.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:16 smithi053 ceph-mon[36169]: pgmap v95: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:15:17.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:15:17.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:15:17.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 -- 172.21.15.37:0/3467214497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 msgr2=0x7f33dc114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 --2- 172.21.15.37:0/3467214497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc114c30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009cd0 tx=0x7f33d402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 -- 172.21.15.37:0/3467214497 shutdown_connections 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 --2- 172.21.15.37:0/3467214497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc114c30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 --2- 172.21.15.37:0/3467214497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33dc107dc0 0x7f33dc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 --2- 172.21.15.37:0/3467214497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc109b70 0x7f33dc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 -- 172.21.15.37:0/3467214497 >> 172.21.15.37:0/3467214497 conn(0x7f33dc077540 msgr2=0x7f33dc1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 -- 172.21.15.37:0/3467214497 shutdown_connections 2024-03-21T09:15:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.703+0000 7f33e2800640 1 -- 172.21.15.37:0/3467214497 wait complete. 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.704+0000 7f33e2800640 1 Processor -- start 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.704+0000 7f33e2800640 1 -- start start 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33e2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc107dc0 0x7f33dc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33e2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33e2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33dc109b70 0x7f33dc1a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33e2800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc118100 con 0x7f33dc107dc0 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33e2800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc117f80 con 0x7f33dc108760 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33e2800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc118280 con 0x7f33dc109b70 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33daa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33daa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44720/0 (socket says 172.21.15.37:44720) 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33daa00640 1 -- 172.21.15.37:0/4189837102 learned_addr learned my addr 172.21.15.37:0/4189837102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33db400640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc107dc0 0x7f33dc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.705+0000 7f33dbe00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33dc109b70 0x7f33dc1a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33daa00640 1 -- 172.21.15.37:0/4189837102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33dc109b70 msgr2=0x7f33dc1a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33daa00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33dc109b70 0x7f33dc1a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33daa00640 1 -- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc107dc0 msgr2=0x7f33dc19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33daa00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc107dc0 0x7f33dc19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33daa00640 1 -- 172.21.15.37:0/4189837102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc1a3580 con 0x7f33dc108760 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33db400640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc107dc0 0x7f33dc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33daa00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc19eb20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f33c800e9f0 tx=0x7f33c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33d3e00640 1 -- 172.21.15.37:0/4189837102 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c800cd80 con 0x7f33dc108760 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33d3e00640 1 -- 172.21.15.37:0/4189837102 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c8010910 con 0x7f33dc108760 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33d3e00640 1 -- 172.21.15.37:0/4189837102 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8018990 con 0x7f33dc108760 2024-03-21T09:15:17.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.706+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc1a3870 con 0x7f33dc108760 2024-03-21T09:15:17.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.707+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc1ab100 con 0x7f33dc108760 2024-03-21T09:15:17.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 Processor -- start 2024-03-21T09:15:17.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 -- start start 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf58680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1beb1250 con 0x561f1bf4cf20 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1be59a30 con 0x561f1bf56290 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33e5576740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1bead2f0 con 0x561f1bf4dc00 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33d3e00640 1 -- 172.21.15.37:0/4189837102 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f33c8010ac0 con 0x7f33dc108760 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.710+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44550/0 (socket says 172.21.15.37:44550) 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 -- 172.21.15.37:0/3838683346 learned_addr learned my addr 172.21.15.37:0/3838683346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33db400640 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33daa00640 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf58680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 -- 172.21.15.37:0/3838683346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 msgr2=0x561f1bf58680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf58680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 -- 172.21.15.37:0/3838683346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 msgr2=0x561f1bf4d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf4d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 -- 172.21.15.37:0/3838683346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1be44690 con 0x561f1bf4dc00 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33dbe00640 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4c950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33d4004af0 tx=0x7f33d40098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db65c94081730f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.711+0000 7f33d0c00640 1 -- 172.21.15.37:0/3838683346 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4043070 con 0x561f1bf4dc00 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33d0c00640 1 -- 172.21.15.37:0/3838683346 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33d4032c90 con 0x561f1bf4dc00 2024-03-21T09:15:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33d0c00640 1 -- 172.21.15.37:0/3838683346 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4039040 con 0x561f1bf4dc00 2024-03-21T09:15:17.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 -- 172.21.15.37:0/3838683346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 msgr2=0x561f1bf4c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4c950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33d4004af0 tx=0x7f33d40098a0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 -- 172.21.15.37:0/3838683346 shutdown_connections 2024-03-21T09:15:17.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf4d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 --2- 172.21.15.37:0/3838683346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4c950 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 -- 172.21.15.37:0/3838683346 >> 172.21.15.37:0/3838683346 conn(0x561f1bf14200 msgr2=0x561f1bf47970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 -- 172.21.15.37:0/3838683346 shutdown_connections 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.712+0000 7f33e5576740 1 -- 172.21.15.37:0/3838683346 wait complete. 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.713+0000 7f33d3e00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f33c00a3020 0x7f33c00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.713+0000 7f33d3e00640 1 -- 172.21.15.37:0/4189837102 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f33c80cfec0 con 0x7f33dc108760 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.713+0000 7f33e5576740 1 -- start start 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.713+0000 7f33dbe00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f33c00a3020 0x7f33c00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.713+0000 7f33e5576740 1 Processor -- start 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33dbe00640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f33c00a3020 0x7f33c00a54e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33d4037040 tx=0x7f33d4005df0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 0x561f1bf558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4dc00 0x561f1bf52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1bf590a0 con 0x561f1bf4dc00 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1be6ce80 con 0x561f1bf56290 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1bf59220 con 0x561f1bf4cf20 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33e5576740 10 client.0 _ll_register_callbacks cb 0x561f1be7f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 0x561f1bf558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 0x561f1bf558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44556/0 (socket says 172.21.15.37:44556) 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33db400640 1 -- 172.21.15.37:0/192802804 learned_addr learned my addr 172.21.15.37:0/192802804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33dbe00640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.714+0000 7f33daa00640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4dc00 0x561f1bf52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33db400640 1 -- 172.21.15.37:0/192802804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 msgr2=0x561f1bf531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33db400640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33db400640 1 -- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4dc00 msgr2=0x561f1bf52c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33db400640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4dc00 0x561f1bf52c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33db400640 1 -- 172.21.15.37:0/192802804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f1bc8f7d0 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33db400640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 0x561f1bf558e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f33cc013790 tx=0x7f33cc013c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d32a2f1c20505b2f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33c5600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33c5600640 1 -- 172.21.15.37:0/192802804 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc011dd0 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33c5600640 1 -- 172.21.15.37:0/192802804 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f33cc004020 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.715+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33d4032c90 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33c5600640 1 -- 172.21.15.37:0/192802804 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33cc025050 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 10 client.26401 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f1bbc53f0 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33c5600640 1 -- 172.21.15.37:0/192802804 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33d4032c90 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33c5600640 1 -- 172.21.15.37:0/192802804 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f33cc01a030 con 0x561f1bf4cf20 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 10 client.26401 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 20 client.26401 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:17.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bf66160 0x561f1bf68600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 4 client.26401 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.716+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f1bebe790 con 0x561f1bf66160 2024-03-21T09:15:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.717+0000 7f33daa00640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bf66160 0x561f1bf68600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.717+0000 7f33daa00640 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bf66160 0x561f1bf68600 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd488601d38c91ed server_cookie=9f4acbcc51eafd8d in_seq=0 out_seq=0 2024-03-21T09:15:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.717+0000 7f33c5600640 10 client.26401 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33c5600640 1 -- 172.21.15.37:0/192802804 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561f1bebe790 con 0x561f1bf66160 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33c5600640 10 client.26401 handle_command_reply: tid=0 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33e5576740 1 client.26401 shutdown 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 msgr2=0x561f1bf558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33e5576740 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 0x561f1bf558e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f33cc013790 tx=0x7f33cc013c50 comp rx=0 tx=0).stop 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 shutdown_connections 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.718+0000 7f33e5576740 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bf66160 0x561f1bf68600 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4dc00 0x561f1bf52c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33c5600640 0 client.26401 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 1 --2- 172.21.15.37:0/192802804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4cf20 0x561f1bf558e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 >> 172.21.15.37:0/192802804 conn(0x561f1bf14200 msgr2=0x561f1bf496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 shutdown_connections 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 1 -- 172.21.15.37:0/192802804 wait complete. 2024-03-21T09:15:17.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.719+0000 7f33e5576740 20 client.26401 trim_cache size 0 max 16384 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.736+0000 7f33e5576740 1 Processor -- start 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 -- start start 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf57680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1b74e0 con 0x561f1bf4cf20 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1b4fb0 con 0x561f1bf56290 2024-03-21T09:15:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33e5576740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1b3050 con 0x561f1bf4dc00 2024-03-21T09:15:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf57680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44568/0 (socket says 172.21.15.37:44568) 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 -- 172.21.15.37:0/698609071 learned_addr learned my addr 172.21.15.37:0/698609071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 -- 172.21.15.37:0/698609071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 msgr2=0x561f1bf57680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf57680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 -- 172.21.15.37:0/698609071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 msgr2=0x561f1bf553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.737+0000 7f33daa00640 1 -- 172.21.15.37:0/698609071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1bb850f0 con 0x561f1bf4dc00 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.738+0000 7f33daa00640 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf558e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c6a0 tx=0x7f33c80d23d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985fd0ea939ccb20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.738+0000 7f33c7e00640 1 -- 172.21.15.37:0/698609071 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c80a0060 con 0x561f1bf4dc00 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.738+0000 7f33c7e00640 1 -- 172.21.15.37:0/698609071 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c80a4920 con 0x561f1bf4dc00 2024-03-21T09:15:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.738+0000 7f33c7e00640 1 -- 172.21.15.37:0/698609071 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8018b30 con 0x561f1bf4dc00 2024-03-21T09:15:17.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 -- 172.21.15.37:0/698609071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 msgr2=0x561f1bf558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf558e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c6a0 tx=0x7f33c80d23d0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 -- 172.21.15.37:0/698609071 shutdown_connections 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf56290 0x561f1bf57680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf558e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 --2- 172.21.15.37:0/698609071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf4cf20 0x561f1bf553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 -- 172.21.15.37:0/698609071 >> 172.21.15.37:0/698609071 conn(0x561f1bf14200 msgr2=0x561f1bf495e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 -- 172.21.15.37:0/698609071 shutdown_connections 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.739+0000 7f33e5576740 1 -- 172.21.15.37:0/698609071 wait complete. 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.740+0000 7f33e5576740 1 -- start start 2024-03-21T09:15:17.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.740+0000 7f33e5576740 1 Processor -- start 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 0x561f1bf5e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf56290 0x561f1bf50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1b3650 con 0x561f1bf56290 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1b4950 con 0x561f1bf4cf20 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1b3c90 con 0x561f1bf4dc00 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33e5576740 10 client.0 _ll_register_callbacks cb 0x7f33cc009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 0x561f1bf5e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33daa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 0x561f1bf5e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44768/0 (socket says 172.21.15.37:44768) 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 -- 172.21.15.37:0/1390916378 learned_addr learned my addr 172.21.15.37:0/1390916378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33dbe00640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf56290 0x561f1bf50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 -- 172.21.15.37:0/1390916378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 msgr2=0x561f1bf4fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 -- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf56290 msgr2=0x561f1bf50370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf56290 0x561f1bf50370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33db400640 1 -- 172.21.15.37:0/1390916378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f1bf508e0 con 0x561f1bf4cf20 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.741+0000 7f33daa00640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33db400640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 0x561f1bf5e020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00f550 tx=0x7f33cc0118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6475ec58e2094a5f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33c5600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33c5600640 1 -- 172.21.15.37:0/1390916378 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc00f900 con 0x561f1bf4cf20 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33c5600640 1 -- 172.21.15.37:0/1390916378 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f33cc00faf0 con 0x561f1bf4cf20 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33c80a4920 con 0x561f1bf4cf20 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33c5600640 1 -- 172.21.15.37:0/1390916378 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33cc03b050 con 0x561f1bf4cf20 2024-03-21T09:15:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33e5576740 10 client.26381 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f1bfc2830 con 0x561f1bf4cf20 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.742+0000 7f33c5600640 1 -- 172.21.15.37:0/1390916378 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c80a4920 con 0x561f1bf4cf20 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33c5600640 1 -- 172.21.15.37:0/1390916378 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f33cc01a040 con 0x561f1bf4cf20 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33e5576740 10 client.26381 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33e5576740 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33e5576740 20 client.26381 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33e5576740 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bfc2f60 0x561f1bfc5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33e5576740 4 client.26381 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f1beb8ae0 con 0x561f1bfc2f60 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33daa00640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bfc2f60 0x561f1bfc5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33daa00640 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bfc2f60 0x561f1bfc5380 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1b7048e063ebe87 server_cookie=b1633031885805f4 in_seq=0 out_seq=0 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.743+0000 7f33c5600640 10 client.26381 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33c5600640 1 -- 172.21.15.37:0/1390916378 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561f1beb8ae0 con 0x561f1bfc2f60 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33c5600640 10 client.26381 handle_command_reply: tid=0 2024-03-21T09:15:17.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 client.26381 shutdown 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 msgr2=0x561f1bf5e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 0x561f1bf5e020 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00f550 tx=0x7f33cc0118f0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 shutdown_connections 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f1bf56290 0x561f1bf50370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f1bf4dc00 0x561f1bf4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x561f1bfc2f60 0x561f1bfc5380 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 --2- 172.21.15.37:0/1390916378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f1bf4cf20 0x561f1bf5e020 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33c5600640 0 client.26381 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 >> 172.21.15.37:0/1390916378 conn(0x561f1bf14200 msgr2=0x561f1bf46200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 shutdown_connections 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 1 -- 172.21.15.37:0/1390916378 wait complete. 2024-03-21T09:15:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.744+0000 7f33e5576740 20 client.26381 trim_cache size 0 max 16384 2024-03-21T09:15:17.742 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:15:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:15:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:15:17.742 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f33c00a3020 msgr2=0x7f33c00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f33c00a3020 0x7f33c00a54e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33d4037040 tx=0x7f33d4005df0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 msgr2=0x7f33dc19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc19eb20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f33c800e9f0 tx=0x7f33c800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 shutdown_connections 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33dc109b70 0x7f33dc1a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f33c00a3020 0x7f33c00a54e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33dc108760 0x7f33dc19eb20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 --2- 172.21.15.37:0/4189837102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc107dc0 0x7f33dc19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 >> 172.21.15.37:0/4189837102 conn(0x7f33dc077540 msgr2=0x7f33dc077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 shutdown_connections 2024-03-21T09:15:17.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.745+0000 7f33e2800640 1 -- 172.21.15.37:0/4189837102 wait complete. 2024-03-21T09:15:17.757 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:15:17.757 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 -- 172.21.15.37:0/2844487309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d81006b0 msgr2=0x7f84d8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 --2- 172.21.15.37:0/2844487309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d81006b0 0x7f84d8100a90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009890 tx=0x7f84cc02bc50 comp rx=0 tx=0).stop 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 -- 172.21.15.37:0/2844487309 shutdown_connections 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 --2- 172.21.15.37:0/2844487309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84d8105ab0 0x7f84d8105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 --2- 172.21.15.37:0/2844487309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d8100fd0 0x7f84d8105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 --2- 172.21.15.37:0/2844487309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d81006b0 0x7f84d8100a90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 -- 172.21.15.37:0/2844487309 >> 172.21.15.37:0/2844487309 conn(0x7f84d8077720 msgr2=0x7f84d8077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:17.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 -- 172.21.15.37:0/2844487309 shutdown_connections 2024-03-21T09:15:17.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.978+0000 7f84dfa00640 1 -- 172.21.15.37:0/2844487309 wait complete. 2024-03-21T09:15:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.979+0000 7f84dfa00640 1 Processor -- start 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.979+0000 7f84dfa00640 1 -- start start 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dfa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d81006b0 0x7f84d819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dfa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 0x7f84d819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dfa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84d8105ab0 0x7f84d81a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dfa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8118030 con 0x7f84d81006b0 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dfa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8117eb0 con 0x7f84d8105ab0 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dfa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81181b0 con 0x7f84d8100fd0 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 0x7f84d819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d81006b0 0x7f84d819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 0x7f84d819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44606/0 (socket says 172.21.15.37:44606) 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84d7e00640 1 -- 172.21.15.37:0/868379844 learned_addr learned my addr 172.21.15.37:0/868379844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.980+0000 7f84dda00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84d8105ab0 0x7f84d81a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d7e00640 1 -- 172.21.15.37:0/868379844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84d8105ab0 msgr2=0x7f84d81a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d7e00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84d8105ab0 0x7f84d81a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d7e00640 1 -- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d81006b0 msgr2=0x7f84d819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d7e00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d81006b0 0x7f84d819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d7e00640 1 -- 172.21.15.37:0/868379844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d81a3530 con 0x7f84d8100fd0 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84dd000640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d81006b0 0x7f84d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d7e00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 0x7f84d819ead0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9c0 tx=0x7f84c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400ccc0 con 0x7f84d8100fd0 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84c4005500 con 0x7f84d8100fd0 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4010430 con 0x7f84d8100fd0 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d81a3820 con 0x7f84d8100fd0 2024-03-21T09:15:17.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.981+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d81ab0b0 con 0x7f84d8100fd0 2024-03-21T09:15:17.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.983+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d8068520 con 0x7f84d8100fd0 2024-03-21T09:15:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.988+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f84c40046c0 con 0x7f84d8100fd0 2024-03-21T09:15:17.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.989+0000 7f84d5600640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c00a3090 0x7f84c00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.989+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f84c40cfed0 con 0x7f84d8100fd0 2024-03-21T09:15:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.990+0000 7f84dd000640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c00a3090 0x7f84c00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.990+0000 7f84dd000640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c00a3090 0x7f84c00a5550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02c260 tx=0x7f84cc02e040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:17.998+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f84c409c5c0 con 0x7f84d8100fd0 2024-03-21T09:15:18.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.247+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84d81ab3b0 con 0x7f84d8100fd0 2024-03-21T09:15:18.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.248+0000 7f84d5600640 1 -- 172.21.15.37:0/868379844 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11376 (secure 0 0 0) 0x7f84c409bec0 con 0x7f84d8100fd0 2024-03-21T09:15:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:15:18.242 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:14:48.220597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:15:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-03-21T09:15:18.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c00a3090 msgr2=0x7f84c00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c00a3090 0x7f84c00a5550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02c260 tx=0x7f84cc02e040 comp rx=0 tx=0).stop 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 msgr2=0x7f84d819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 0x7f84d819ead0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9c0 tx=0x7f84c400ee80 comp rx=0 tx=0).stop 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 shutdown_connections 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c00a3090 0x7f84c00a5550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84d8105ab0 0x7f84d81a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84d8100fd0 0x7f84d819ead0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 --2- 172.21.15.37:0/868379844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d81006b0 0x7f84d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 >> 172.21.15.37:0/868379844 conn(0x7f84d8077720 msgr2=0x7f84d8103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 shutdown_connections 2024-03-21T09:15:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.252+0000 7f84dfa00640 1 -- 172.21.15.37:0/868379844 wait complete. 2024-03-21T09:15:18.273 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:15:18.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 -- 172.21.15.37:0/304624515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 msgr2=0x7fd2d810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 --2- 172.21.15.37:0/304624515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d810ded0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009ca0 tx=0x7fd2d402c190 comp rx=0 tx=0).stop 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 -- 172.21.15.37:0/304624515 shutdown_connections 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 --2- 172.21.15.37:0/304624515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d810ded0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 --2- 172.21.15.37:0/304624515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d8102f50 0x7fd2d810b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 --2- 172.21.15.37:0/304624515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 0x7fd2d8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 -- 172.21.15.37:0/304624515 >> 172.21.15.37:0/304624515 conn(0x7fd2d8077130 msgr2=0x7fd2d80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 -- 172.21.15.37:0/304624515 shutdown_connections 2024-03-21T09:15:18.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.519+0000 7fd2df200640 1 -- 172.21.15.37:0/304624515 wait complete. 2024-03-21T09:15:18.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.520+0000 7fd2df200640 1 Processor -- start 2024-03-21T09:15:18.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.520+0000 7fd2df200640 1 -- start start 2024-03-21T09:15:18.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2df200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 0x7fd2d819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2df200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d8102f50 0x7fd2d819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2df200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2df200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8100ab0 con 0x7fd2d8102f50 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2df200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8100930 con 0x7fd2d8102630 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2df200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8100c30 con 0x7fd2d810ba50 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2dd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d8102f50 0x7fd2d819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 0x7fd2d819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44628/0 (socket says 172.21.15.37:44628) 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 0x7fd2d819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44796/0 (socket says 172.21.15.37:44796) 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 -- 172.21.15.37:0/58098292 learned_addr learned my addr 172.21.15.37:0/58098292 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 -- 172.21.15.37:0/58098292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 msgr2=0x7fd2d819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 0x7fd2d819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 -- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d8102f50 msgr2=0x7fd2d819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d8102f50 0x7fd2d819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.521+0000 7fd2de800640 1 -- 172.21.15.37:0/58098292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d819a8d0 con 0x7fd2d810ba50 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.522+0000 7fd2de800640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d819a010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4037040 tx=0x7fd2d4035cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.522+0000 7fd2d2a00640 1 -- 172.21.15.37:0/58098292 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4039040 con 0x7fd2d810ba50 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.522+0000 7fd2d2a00640 1 -- 172.21.15.37:0/58098292 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2d4044070 con 0x7fd2d810ba50 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.522+0000 7fd2d2a00640 1 -- 172.21.15.37:0/58098292 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40328d0 con 0x7fd2d810ba50 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.522+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d819ab00 con 0x7fd2d810ba50 2024-03-21T09:15:18.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.522+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d81a6a00 con 0x7fd2d810ba50 2024-03-21T09:15:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.525+0000 7fd2d2a00640 1 -- 172.21.15.37:0/58098292 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd2d4004040 con 0x7fd2d810ba50 2024-03-21T09:15:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 Processor -- start 2024-03-21T09:15:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 -- start start 2024-03-21T09:15:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 0x5602260bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022601a2c0 con 0x5602260bfde0 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560225fc2c90 con 0x5602260b7260 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560226016360 con 0x5602260b68b0 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 0x5602260bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 0x5602260bf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44812/0 (socket says 172.21.15.37:44812) 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.526+0000 7fd2dde00640 1 -- 172.21.15.37:0/1420103247 learned_addr learned my addr 172.21.15.37:0/1420103247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dd400640 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dd400640 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dde00640 1 -- 172.21.15.37:0/1420103247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 msgr2=0x5602260b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dde00640 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dde00640 1 -- 172.21.15.37:0/1420103247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 msgr2=0x5602260c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dde00640 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dde00640 1 -- 172.21.15.37:0/1420103247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560225fe7080 con 0x5602260b7260 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dd400640 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2dde00640 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 0x5602260bf810 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009cf0 tx=0x7fd2cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b0e00273c50f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2c7000640 1 -- 172.21.15.37:0/1420103247 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004230 con 0x5602260b7260 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2c7000640 1 -- 172.21.15.37:0/1420103247 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc0043d0 con 0x5602260b7260 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.527+0000 7fd2c7000640 1 -- 172.21.15.37:0/1420103247 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc019960 con 0x5602260b7260 2024-03-21T09:15:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2d2a00640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2c00a3090 0x7fd2c00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2d2a00640 1 -- 172.21.15.37:0/58098292 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd2d40f1f70 con 0x7fd2d810ba50 2024-03-21T09:15:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2dd400640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2c00a3090 0x7fd2c00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1420103247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 msgr2=0x5602260bf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 0x5602260bf810 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009cf0 tx=0x7fd2cc011810 comp rx=0 tx=0).stop 2024-03-21T09:15:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1420103247 shutdown_connections 2024-03-21T09:15:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b7260 0x5602260bf810 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1420103247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1420103247 >> 172.21.15.37:0/1420103247 conn(0x5602260b0f80 msgr2=0x5602260b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.528+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1420103247 shutdown_connections 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.529+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1420103247 wait complete. 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.529+0000 7fd2dd400640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2c00a3090 0x7fd2c00a5550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2d810b7f0 tx=0x7fd2c8004730 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.529+0000 7fd2e35e6740 1 -- start start 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.529+0000 7fd2e35e6740 1 Processor -- start 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260bf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260bfde0 0x5602260bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602260c2d50 con 0x5602260b7260 2024-03-21T09:15:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560225fd60d0 con 0x5602260b68b0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602260c2ed0 con 0x5602260bfde0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2e35e6740 10 client.0 _ll_register_callbacks cb 0x560225fdae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260bf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260bf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44828/0 (socket says 172.21.15.37:44828) 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.530+0000 7fd2dde00640 1 -- 172.21.15.37:0/4173837702 learned_addr learned my addr 172.21.15.37:0/4173837702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dd400640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2de800640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260bfde0 0x5602260bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dde00640 1 -- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260bfde0 msgr2=0x5602260bcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dde00640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260bfde0 0x5602260bcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dde00640 1 -- 172.21.15.37:0/4173837702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 msgr2=0x5602260bca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dde00640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260bca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dde00640 1 -- 172.21.15.37:0/4173837702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560225d2e3f0 con 0x5602260b68b0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dd400640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260bca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2dde00640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260bf6e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc019460 tx=0x7fd2cc003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c774b6be652d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2bb400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2bb400640 1 -- 172.21.15.37:0/4173837702 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc027070 con 0x5602260b68b0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2bb400640 1 -- 172.21.15.37:0/4173837702 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd2cc005240 con 0x5602260b68b0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.531+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2cc0043d0 con 0x5602260b68b0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2bb400640 1 -- 172.21.15.37:0/4173837702 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc033050 con 0x5602260b68b0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 10 client.26399 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560225df87d0 con 0x5602260b68b0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2bb400640 1 -- 172.21.15.37:0/4173837702 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc0043d0 con 0x5602260b68b0 2024-03-21T09:15:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2bb400640 1 -- 172.21.15.37:0/4173837702 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fd2cc004ba0 con 0x5602260b68b0 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 10 client.26399 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 10 client.26399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 20 client.26399 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5602260cfaa0 0x5602260d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 4 client.26399 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.532+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560226027800 con 0x5602260cfaa0 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.533+0000 7fd2dd400640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5602260cfaa0 0x5602260d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.533+0000 7fd2dd400640 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5602260cfaa0 0x5602260d1e90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38648d4a415eb42e server_cookie=e352829e6eb456fb in_seq=0 out_seq=0 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.533+0000 7fd2bb400640 10 client.26399 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2bb400640 1 -- 172.21.15.37:0/4173837702 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560226027800 con 0x5602260cfaa0 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2bb400640 10 client.26399 handle_command_reply: tid=0 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 client.26399 shutdown 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 msgr2=0x5602260bf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260bf6e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc019460 tx=0x7fd2cc003820 comp rx=0 tx=0).stop 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 shutdown_connections 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260bfde0 0x5602260bcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5602260cfaa0 0x5602260d1e90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260bca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/4173837702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260bf6e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2bb400640 0 client.26399 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.534+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 >> 172.21.15.37:0/4173837702 conn(0x5602260b0f80 msgr2=0x5602260b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.535+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 shutdown_connections 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.535+0000 7fd2e35e6740 1 -- 172.21.15.37:0/4173837702 wait complete. 2024-03-21T09:15:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.535+0000 7fd2e35e6740 20 client.26399 trim_cache size 0 max 16384 2024-03-21T09:15:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 Processor -- start 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 -- start start 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b7260 0x5602260abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81b2fc0 con 0x5602260bfde0 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81b0a90 con 0x5602260b68b0 2024-03-21T09:15:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.558+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81aeb30 con 0x5602260b7260 2024-03-21T09:15:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44830/0 (socket says 172.21.15.37:44830) 2024-03-21T09:15:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 -- 172.21.15.37:0/591432563 learned_addr learned my addr 172.21.15.37:0/591432563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2de800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dd400640 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b7260 0x5602260abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 -- 172.21.15.37:0/591432563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b7260 msgr2=0x5602260abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b7260 0x5602260abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 -- 172.21.15.37:0/591432563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 msgr2=0x5602260bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260bf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 -- 172.21.15.37:0/591432563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602260bd360 con 0x5602260b68b0 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.559+0000 7fd2dde00640 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260b0160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009730 tx=0x7fd2cc011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeff94e9df1890ee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.560+0000 7fd2c6600640 1 -- 172.21.15.37:0/591432563 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc03c070 con 0x5602260b68b0 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.560+0000 7fd2c6600640 1 -- 172.21.15.37:0/591432563 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc0043d0 con 0x5602260b68b0 2024-03-21T09:15:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.560+0000 7fd2c6600640 1 -- 172.21.15.37:0/591432563 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc027040 con 0x5602260b68b0 2024-03-21T09:15:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.560+0000 7fd2e35e6740 1 -- 172.21.15.37:0/591432563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 msgr2=0x5602260b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.560+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260b0160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009730 tx=0x7fd2cc011d10 comp rx=0 tx=0).stop 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 -- 172.21.15.37:0/591432563 shutdown_connections 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260bfde0 0x5602260bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b7260 0x5602260abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/591432563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260b68b0 0x5602260b0160 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 -- 172.21.15.37:0/591432563 >> 172.21.15.37:0/591432563 conn(0x5602260b0f80 msgr2=0x5602260b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 -- 172.21.15.37:0/591432563 shutdown_connections 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 -- 172.21.15.37:0/591432563 wait complete. 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.561+0000 7fd2e35e6740 1 -- start start 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 Processor -- start 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260bfde0 0x5602260c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81af130 con 0x5602260b7260 2024-03-21T09:15:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81b0430 con 0x5602260bfde0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81af770 con 0x5602260b68b0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2e35e6740 10 client.0 _ll_register_callbacks cb 0x7fd2cc009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2dde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44670/0 (socket says 172.21.15.37:44670) 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2dde00640 1 -- 172.21.15.37:0/1741692225 learned_addr learned my addr 172.21.15.37:0/1741692225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.562+0000 7fd2dd400640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2de800640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260bfde0 0x5602260c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2dde00640 1 -- 172.21.15.37:0/1741692225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260bfde0 msgr2=0x5602260c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2dde00640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260bfde0 0x5602260c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2dde00640 1 -- 172.21.15.37:0/1741692225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 msgr2=0x5602260c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2dde00640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2dde00640 1 -- 172.21.15.37:0/1741692225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602260c6120 con 0x5602260b68b0 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2de800640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260bfde0 0x5602260c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:15:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2dde00640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260c7f70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0059f0 tx=0x7fd2cc012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8565f8d5c1bc30 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2bb400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2bb400640 1 -- 172.21.15.37:0/1741692225 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc03c070 con 0x5602260b68b0 2024-03-21T09:15:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2bb400640 1 -- 172.21.15.37:0/1741692225 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd2cc019990 con 0x5602260b68b0 2024-03-21T09:15:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.563+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2cc0043d0 con 0x5602260b68b0 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2bb400640 1 -- 172.21.15.37:0/1741692225 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc049050 con 0x5602260b68b0 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 10 client.26428 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56022612b2a0 con 0x5602260b68b0 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2bb400640 1 -- 172.21.15.37:0/1741692225 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc0043d0 con 0x5602260b68b0 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2bb400640 1 -- 172.21.15.37:0/1741692225 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fd2cc004660 con 0x5602260b68b0 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 10 client.26428 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 10 client.26428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 20 client.26428 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x56022612b800 0x56022612dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 4 client.26428 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560226021b50 con 0x56022612b800 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.564+0000 7fd2dd400640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x56022612b800 0x56022612dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2dd400640 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x56022612b800 0x56022612dc20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a48dcf9ea23ed54c server_cookie=6d3207a1aa5f916f in_seq=0 out_seq=0 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2bb400640 10 client.26428 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2bb400640 1 -- 172.21.15.37:0/1741692225 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560226021b50 con 0x56022612b800 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2bb400640 10 client.26428 handle_command_reply: tid=0 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 client.26428 shutdown 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 msgr2=0x5602260c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260c7f70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0059f0 tx=0x7fd2cc012f30 comp rx=0 tx=0).stop 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 shutdown_connections 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5602260bfde0 0x5602260c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x56022612b800 0x56022612dc20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602260b7260 0x5602260c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 --2- 172.21.15.37:0/1741692225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602260b68b0 0x5602260c7f70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2bb400640 0 client.26428 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.565+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 >> 172.21.15.37:0/1741692225 conn(0x5602260b0f80 msgr2=0x5602260af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.566+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 shutdown_connections 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.566+0000 7fd2e35e6740 1 -- 172.21.15.37:0/1741692225 wait complete. 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.566+0000 7fd2e35e6740 20 client.26428 trim_cache size 0 max 16384 2024-03-21T09:15:18.563 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:15:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.566+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2c00a3090 msgr2=0x7fd2c00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.566+0000 7fd2df200640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2c00a3090 0x7fd2c00a5550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2d810b7f0 tx=0x7fd2c8004730 comp rx=0 tx=0).stop 2024-03-21T09:15:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 msgr2=0x7fd2d819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d819a010 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4037040 tx=0x7fd2d4035cf0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 shutdown_connections 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d810ba50 0x7fd2d819a010 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d8102f50 0x7fd2d819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2c00a3090 0x7fd2c00a5550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 --2- 172.21.15.37:0/58098292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2d8102630 0x7fd2d819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 >> 172.21.15.37:0/58098292 conn(0x7fd2d8077130 msgr2=0x7fd2d810d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 shutdown_connections 2024-03-21T09:15:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.567+0000 7fd2df200640 1 -- 172.21.15.37:0/58098292 wait complete. 2024-03-21T09:15:18.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:18 smithi037 ceph-mon[31830]: pgmap v96: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:15:18.579 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:15:18.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:15:18.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:18 smithi071 ceph-mon[36597]: pgmap v96: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:15:18.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:18 smithi053 ceph-mon[36169]: pgmap v96: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 -- 172.21.15.37:0/2707475736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f413810c980 msgr2=0x7f413810cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 --2- 172.21.15.37:0/2707475736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f413810c980 0x7f413810cd60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4128009900 tx=0x7f412802bcc0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 -- 172.21.15.37:0/2707475736 shutdown_connections 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 --2- 172.21.15.37:0/2707475736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138108440 0x7f4138114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 --2- 172.21.15.37:0/2707475736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4138107aa0 0x7f4138107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 --2- 172.21.15.37:0/2707475736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f413810c980 0x7f413810cd60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 -- 172.21.15.37:0/2707475736 >> 172.21.15.37:0/2707475736 conn(0x7f41380772a0 msgr2=0x7f41380796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 -- 172.21.15.37:0/2707475736 shutdown_connections 2024-03-21T09:15:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.805+0000 7f4137e00640 1 -- 172.21.15.37:0/2707475736 wait complete. 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.806+0000 7f4137e00640 1 Processor -- start 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.806+0000 7f4137e00640 1 -- start start 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 0x7f413819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4138108440 0x7f413819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f413810c980 0x7f41381a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138117e90 con 0x7f413810c980 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138117d10 con 0x7f4138108440 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138118010 con 0x7f4138107aa0 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4136a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 0x7f413819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f412fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4138108440 0x7f413819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4136a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 0x7f413819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44690/0 (socket says 172.21.15.37:44690) 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4136a00640 1 -- 172.21.15.37:0/1221271293 learned_addr learned my addr 172.21.15.37:0/1221271293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.807+0000 7f4137400640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f413810c980 0x7f41381a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f4136a00640 1 -- 172.21.15.37:0/1221271293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4138108440 msgr2=0x7f413819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f4136a00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4138108440 0x7f413819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f4136a00640 1 -- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f413810c980 msgr2=0x7f41381a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f4136a00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f413810c980 0x7f41381a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f4136a00640 1 -- 172.21.15.37:0/1221271293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41381a33c0 con 0x7f4138107aa0 2024-03-21T09:15:18.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f4136a00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 0x7f413819e3d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4128000c00 tx=0x7f41280043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f412f400640 1 -- 172.21.15.37:0/1221271293 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128042020 con 0x7f4138107aa0 2024-03-21T09:15:18.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f412f400640 1 -- 172.21.15.37:0/1221271293 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4128005000 con 0x7f4138107aa0 2024-03-21T09:15:18.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.808+0000 7f412f400640 1 -- 172.21.15.37:0/1221271293 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128003b20 con 0x7f4138107aa0 2024-03-21T09:15:18.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.809+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41381a3650 con 0x7f4138107aa0 2024-03-21T09:15:18.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.809+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41381aaf40 con 0x7f4138107aa0 2024-03-21T09:15:18.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 Processor -- start 2024-03-21T09:15:18.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 -- start start 2024-03-21T09:15:18.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2ad810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1fa2082c0 con 0x55d1fa2a48b0 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1fa1b0c90 con 0x55d1fa2a5260 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.812+0000 7f4140277740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1fa204360 con 0x55d1fa2adde0 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f4137400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f4137400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46738/0 (socket says 172.21.15.37:46738) 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f4137400640 1 -- 172.21.15.37:0/981078948 learned_addr learned my addr 172.21.15.37:0/981078948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 -- 172.21.15.37:0/981078948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 msgr2=0x55d1fa2ad810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2ad810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 -- 172.21.15.37:0/981078948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 msgr2=0x55d1fa2a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 -- 172.21.15.37:0/981078948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1fa1d5080 con 0x55d1fa2adde0 2024-03-21T09:15:18.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412f400640 1 -- 172.21.15.37:0/1221271293 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4128045020 con 0x7f4138107aa0 2024-03-21T09:15:18.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.813+0000 7f412fe00640 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2b0230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4120005d00 tx=0x7f41200107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b194b0126c53d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4127400640 1 -- 172.21.15.37:0/981078948 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120004020 con 0x55d1fa2adde0 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4127400640 1 -- 172.21.15.37:0/981078948 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41200041c0 con 0x55d1fa2adde0 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4127400640 1 -- 172.21.15.37:0/981078948 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120011800 con 0x55d1fa2adde0 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 -- 172.21.15.37:0/981078948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 msgr2=0x55d1fa2b0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2b0230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4120005d00 tx=0x7f41200107a0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 -- 172.21.15.37:0/981078948 shutdown_connections 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2b0230 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 --2- 172.21.15.37:0/981078948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.814+0000 7f4140277740 1 -- 172.21.15.37:0/981078948 >> 172.21.15.37:0/981078948 conn(0x55d1fa29ef80 msgr2=0x55d1fa29f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.815+0000 7f4140277740 1 -- 172.21.15.37:0/981078948 shutdown_connections 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.815+0000 7f4140277740 1 -- 172.21.15.37:0/981078948 wait complete. 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.815+0000 7f4140277740 1 -- start start 2024-03-21T09:15:18.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.815+0000 7f412f400640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f411c0a2fa0 0x7f411c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.815+0000 7f4140277740 1 Processor -- start 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.815+0000 7f412f400640 1 -- 172.21.15.37:0/1221271293 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f41280f1820 con 0x7f4138107aa0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4137400640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f411c0a2fa0 0x7f411c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1fa2b0d50 con 0x55d1fa2a48b0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1fa1c40d0 con 0x55d1fa2a5260 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1fa2b0ed0 con 0x55d1fa2adde0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4140277740 10 client.0 _ll_register_callbacks cb 0x55d1fa1c8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4136a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f412fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4136a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46754/0 (socket says 172.21.15.37:46754) 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f412fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2aaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44860/0 (socket says 172.21.15.37:44860) 2024-03-21T09:15:18.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.816+0000 7f4136a00640 1 -- 172.21.15.37:0/2294956325 learned_addr learned my addr 172.21.15.37:0/2294956325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f4137400640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f411c0a2fa0 0x7f411c0a5460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41300099a0 tx=0x7f4130009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f4137400640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f412fe00640 1 -- 172.21.15.37:0/2294956325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 msgr2=0x55d1fa2ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f412fe00640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f412fe00640 1 -- 172.21.15.37:0/2294956325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 msgr2=0x55d1fa2ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f412fe00640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f412fe00640 1 -- 172.21.15.37:0/2294956325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1f9f1c3f0 con 0x55d1fa2a5260 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f4137400640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f412fe00640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2aaaf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4120000c00 tx=0x7f4120003dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5b20e7742e26eac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f411be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f411be00640 1 -- 172.21.15.37:0/2294956325 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412002a060 con 0x55d1fa2a5260 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f411be00640 1 -- 172.21.15.37:0/2294956325 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f41200037f0 con 0x55d1fa2a5260 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.817+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41200041c0 con 0x55d1fa2a5260 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f411be00640 1 -- 172.21.15.37:0/2294956325 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4120031050 con 0x55d1fa2a5260 2024-03-21T09:15:18.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 10 client.26411 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d1f9fe67d0 con 0x55d1fa2a5260 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f411be00640 1 -- 172.21.15.37:0/2294956325 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41200041c0 con 0x55d1fa2a5260 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f411be00640 1 -- 172.21.15.37:0/2294956325 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f4120004790 con 0x55d1fa2a5260 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 10 client.26411 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 10 client.26411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 20 client.26411 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa2bdbc0 0x55d1fa2bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 4 client.26411 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.818+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1fa215800 con 0x55d1fa2bdbc0 2024-03-21T09:15:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.819+0000 7f4136a00640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa2bdbc0 0x55d1fa2bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.819+0000 7f4136a00640 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa2bdbc0 0x55d1fa2bffb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ade182c2a9388a4 server_cookie=4f9a4b517725d24f in_seq=0 out_seq=0 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.819+0000 7f411be00640 10 client.26411 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f411be00640 1 -- 172.21.15.37:0/2294956325 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d1fa215800 con 0x55d1fa2bdbc0 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f411be00640 10 client.26411 handle_command_reply: tid=0 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 client.26411 shutdown 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 msgr2=0x55d1fa2aaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2aaaf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4120000c00 tx=0x7f4120003dc0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 shutdown_connections 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa2bdbc0 0x55d1fa2bffb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2aaaf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 --2- 172.21.15.37:0/2294956325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 >> 172.21.15.37:0/2294956325 conn(0x55d1fa29ef80 msgr2=0x55d1fa2a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f411be00640 0 client.26411 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 shutdown_connections 2024-03-21T09:15:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.820+0000 7f4140277740 1 -- 172.21.15.37:0/2294956325 wait complete. 2024-03-21T09:15:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.821+0000 7f4140277740 20 client.26411 trim_cache size 0 max 16384 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.834+0000 7f4140277740 1 Processor -- start 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 -- start start 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2a5260 0x55d1fa2a8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381b73c0 con 0x55d1fa2adde0 2024-03-21T09:15:18.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381b4e90 con 0x55d1fa2a48b0 2024-03-21T09:15:18.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4140277740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381b2f30 con 0x55d1fa2a5260 2024-03-21T09:15:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4136a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f412fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2a5260 0x55d1fa2a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4136a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44864/0 (socket says 172.21.15.37:44864) 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4136a00640 1 -- 172.21.15.37:0/4002788721 learned_addr learned my addr 172.21.15.37:0/4002788721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.835+0000 7f4137400640 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4136a00640 1 -- 172.21.15.37:0/4002788721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2a5260 msgr2=0x55d1fa2a8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4136a00640 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2a5260 0x55d1fa2a8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4136a00640 1 -- 172.21.15.37:0/4002788721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2adde0 msgr2=0x55d1fa2ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4136a00640 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4136a00640 1 -- 172.21.15.37:0/4002788721 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1fa2ada40 con 0x55d1fa2a48b0 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4137400640 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f412fe00640 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2a5260 0x55d1fa2a8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4136a00640 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a7d50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41280ef1d0 tx=0x7f4128005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3721ed4dcc3e3e66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4126a00640 1 -- 172.21.15.37:0/4002788721 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280d5030 con 0x55d1fa2a48b0 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4126a00640 1 -- 172.21.15.37:0/4002788721 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41280bd980 con 0x55d1fa2a48b0 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.836+0000 7f4126a00640 1 -- 172.21.15.37:0/4002788721 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280eee50 con 0x55d1fa2a48b0 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 -- 172.21.15.37:0/4002788721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 msgr2=0x55d1fa2a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a7d50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41280ef1d0 tx=0x7f4128005e60 comp rx=0 tx=0).stop 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 -- 172.21.15.37:0/4002788721 shutdown_connections 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2adde0 0x55d1fa2ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2a5260 0x55d1fa2a8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 --2- 172.21.15.37:0/4002788721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2a7d50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 -- 172.21.15.37:0/4002788721 >> 172.21.15.37:0/4002788721 conn(0x55d1fa29ef80 msgr2=0x55d1fa2aa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 -- 172.21.15.37:0/4002788721 shutdown_connections 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 -- 172.21.15.37:0/4002788721 wait complete. 2024-03-21T09:15:18.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.837+0000 7f4140277740 1 -- start start 2024-03-21T09:15:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 Processor -- start 2024-03-21T09:15:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381b3530 con 0x55d1fa2a48b0 2024-03-21T09:15:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381b4830 con 0x55d1fa2a5260 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.838+0000 7f4140277740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381b3b70 con 0x55d1fa2adde0 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f4140277740 10 client.0 _ll_register_callbacks cb 0x7f4130009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f4137400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2c0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44870/0 (socket says 172.21.15.37:44870) 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 -- 172.21.15.37:0/2453684968 learned_addr learned my addr 172.21.15.37:0/2453684968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f4136a00640 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 -- 172.21.15.37:0/2453684968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 msgr2=0x55d1fa2c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 -- 172.21.15.37:0/2453684968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 msgr2=0x55d1fa2c06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2c06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 -- 172.21.15.37:0/2453684968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1fa318410 con 0x55d1fa2a5260 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f412fe00640 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2c0c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55d1fa29fdc0 tx=0x7f4120010fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c41c935d4fb2a3f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:18.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.839+0000 7f411be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f411be00640 1 -- 172.21.15.37:0/2453684968 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412002a040 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f411be00640 1 -- 172.21.15.37:0/2453684968 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4120004860 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41280bd980 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f411be00640 1 -- 172.21.15.37:0/2453684968 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4120044050 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f4140277740 10 client.26417 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d1fa319620 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f411be00640 1 -- 172.21.15.37:0/2453684968 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41280bd980 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f411be00640 1 -- 172.21.15.37:0/2453684968 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f4120011680 con 0x55d1fa2a5260 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f4140277740 10 client.26417 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.840+0000 7f4140277740 10 client.26417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4140277740 20 client.26417 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4140277740 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa319da0 0x55d1fa31c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4140277740 4 client.26417 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:15:18.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d1fa20fb50 con 0x55d1fa319da0 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4136a00640 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa319da0 0x55d1fa31c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4136a00640 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa319da0 0x55d1fa31c1c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36830b72d0bde375 server_cookie=893c665813d6f5c0 in_seq=0 out_seq=0 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f411be00640 10 client.26417 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f411be00640 1 -- 172.21.15.37:0/2453684968 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d1fa20fb50 con 0x55d1fa319da0 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f411be00640 10 client.26417 handle_command_reply: tid=0 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.841+0000 7f4140277740 1 client.26417 shutdown 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 msgr2=0x55d1fa2c0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2c0c30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55d1fa29fdc0 tx=0x7f4120010fb0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 shutdown_connections 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55d1fa319da0 0x55d1fa31c1c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1fa2adde0 0x55d1fa2c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d1fa2a5260 0x55d1fa2c0c30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 --2- 172.21.15.37:0/2453684968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1fa2a48b0 0x55d1fa2c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f411be00640 0 client.26417 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 >> 172.21.15.37:0/2453684968 conn(0x55d1fa29ef80 msgr2=0x55d1fa2aa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 shutdown_connections 2024-03-21T09:15:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 1 -- 172.21.15.37:0/2453684968 wait complete. 2024-03-21T09:15:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.842+0000 7f4140277740 20 client.26417 trim_cache size 0 max 16384 2024-03-21T09:15:18.840 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f411c0a2fa0 msgr2=0x7f411c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f411c0a2fa0 0x7f411c0a5460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41300099a0 tx=0x7f4130009500 comp rx=0 tx=0).stop 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 msgr2=0x7f413819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 0x7f413819e3d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4128000c00 tx=0x7f41280043c0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 shutdown_connections 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f413810c980 0x7f41381a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f411c0a2fa0 0x7f411c0a5460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4138108440 0x7f413819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 --2- 172.21.15.37:0/1221271293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4138107aa0 0x7f413819e3d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:18.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 >> 172.21.15.37:0/1221271293 conn(0x7f41380772a0 msgr2=0x7f4138077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:18.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 shutdown_connections 2024-03-21T09:15:18.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:18.843+0000 7f4137e00640 1 -- 172.21.15.37:0/1221271293 wait complete. 2024-03-21T09:15:18.860 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:15:18.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:15:19.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.079+0000 7fac27800640 1 -- 172.21.15.37:0/203839281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20075bd0 msgr2=0x7fac20075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.079+0000 7fac27800640 1 --2- 172.21.15.37:0/203839281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20075bd0 0x7fac20075fb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac08009930 tx=0x7fac0802bc20 comp rx=0 tx=0).stop 2024-03-21T09:15:19.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 -- 172.21.15.37:0/203839281 shutdown_connections 2024-03-21T09:15:19.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 --2- 172.21.15.37:0/203839281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20112500 0x7fac201148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 --2- 172.21.15.37:0/203839281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac200764f0 0x7fac20111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 --2- 172.21.15.37:0/203839281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20075bd0 0x7fac20075fb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 -- 172.21.15.37:0/203839281 >> 172.21.15.37:0/203839281 conn(0x7fac200ff310 msgr2=0x7fac20101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:19.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 -- 172.21.15.37:0/203839281 shutdown_connections 2024-03-21T09:15:19.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.080+0000 7fac27800640 1 -- 172.21.15.37:0/203839281 wait complete. 2024-03-21T09:15:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 Processor -- start 2024-03-21T09:15:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 -- start start 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 0x7fac2019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac200764f0 0x7fac2019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20112500 0x7fac201a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201180c0 con 0x7fac200764f0 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20117f40 con 0x7fac20075bd0 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac27800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20118240 con 0x7fac20112500 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 0x7fac2019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 0x7fac2019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44886/0 (socket says 172.21.15.37:44886) 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac24e00640 1 -- 172.21.15.37:0/2281437210 learned_addr learned my addr 172.21.15.37:0/2281437210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac17e00640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac200764f0 0x7fac2019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.081+0000 7fac25800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20112500 0x7fac201a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.082+0000 7fac24e00640 1 -- 172.21.15.37:0/2281437210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20112500 msgr2=0x7fac201a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.082+0000 7fac24e00640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20112500 0x7fac201a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.082+0000 7fac24e00640 1 -- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac200764f0 msgr2=0x7fac2019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.082+0000 7fac24e00640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac200764f0 0x7fac2019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.082+0000 7fac24e00640 1 -- 172.21.15.37:0/2281437210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac201a35c0 con 0x7fac20075bd0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.082+0000 7fac24e00640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 0x7fac2019e620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac08005ab0 tx=0x7fac080042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.083+0000 7fac1e000640 1 -- 172.21.15.37:0/2281437210 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08043020 con 0x7fac20075bd0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.083+0000 7fac1e000640 1 -- 172.21.15.37:0/2281437210 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac080044d0 con 0x7fac20075bd0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.083+0000 7fac1e000640 1 -- 172.21.15.37:0/2281437210 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08003bb0 con 0x7fac20075bd0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.083+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac201a3850 con 0x7fac20075bd0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.083+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201ab140 con 0x7fac20075bd0 2024-03-21T09:15:19.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.086+0000 7fac1e000640 1 -- 172.21.15.37:0/2281437210 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fac08046020 con 0x7fac20075bd0 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.087+0000 7fac1e000640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fac000a3090 0x7fac000a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.087+0000 7fac1e000640 1 -- 172.21.15.37:0/2281437210 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fac080f1ae0 con 0x7fac20075bd0 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 Processor -- start 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac17e00640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fac000a3090 0x7fac000a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 -- start start 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33dc302c0 con 0x55c33dccc6d0 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33dbd8c90 con 0x55c33dccd080 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.088+0000 7fac2a463740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33dc2c360 con 0x55c33dcd5c00 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac17e00640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fac000a3090 0x7fac000a5550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac201017e0 tx=0x7fac10005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac24e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44736/0 (socket says 172.21.15.37:44736) 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 -- 172.21.15.37:0/3721434408 learned_addr learned my addr 172.21.15.37:0/3721434408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 -- 172.21.15.37:0/3721434408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 msgr2=0x55c33dcd5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 -- 172.21.15.37:0/3721434408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 msgr2=0x55c33dcccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 -- 172.21.15.37:0/3721434408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c33dbfd080 con 0x55c33dcd5c00 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac25800640 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd8050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fac18009f40 tx=0x7fac18010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dea6705f87b3041 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.089+0000 7fac16000640 1 -- 172.21.15.37:0/3721434408 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18016070 con 0x55c33dcd5c00 2024-03-21T09:15:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac16000640 1 -- 172.21.15.37:0/3721434408 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18004d20 con 0x55c33dcd5c00 2024-03-21T09:15:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac16000640 1 -- 172.21.15.37:0/3721434408 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18012850 con 0x55c33dcd5c00 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 -- 172.21.15.37:0/3721434408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 msgr2=0x55c33dcd8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd8050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fac18009f40 tx=0x7fac18010800 comp rx=0 tx=0).stop 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 -- 172.21.15.37:0/3721434408 shutdown_connections 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd8050 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 --2- 172.21.15.37:0/3721434408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 -- 172.21.15.37:0/3721434408 >> 172.21.15.37:0/3721434408 conn(0x55c33dcc6da0 msgr2=0x55c33dcc7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 -- 172.21.15.37:0/3721434408 shutdown_connections 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.090+0000 7fac2a463740 1 -- 172.21.15.37:0/3721434408 wait complete. 2024-03-21T09:15:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.091+0000 7fac2a463740 1 -- start start 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.091+0000 7fac2a463740 1 Processor -- start 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.091+0000 7fac2a463740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac2a463740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac2a463740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac2a463740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33dcd8b70 con 0x55c33dccc6d0 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac2a463740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33dbec0d0 con 0x55c33dccd080 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac2a463740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33dcd8cf0 con 0x55c33dcd5c00 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac2a463740 10 client.0 _ll_register_callbacks cb 0x55c33dbf0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac24e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44740/0 (socket says 172.21.15.37:44740) 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 -- 172.21.15.37:0/3473998844 learned_addr learned my addr 172.21.15.37:0/3473998844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 -- 172.21.15.37:0/3473998844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 msgr2=0x55c33dcd2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 -- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 msgr2=0x55c33dcd5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.092+0000 7fac25800640 1 -- 172.21.15.37:0/3473998844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c33d9443f0 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac25800640 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd2f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac18011010 tx=0x7fac18004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961cc0ef1b0c779b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac0ea00640 1 -- 172.21.15.37:0/3473998844 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18016070 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac0ea00640 1 -- 172.21.15.37:0/3473998844 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fac18003970 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac18004d20 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac0ea00640 1 -- 172.21.15.37:0/3473998844 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac18003b70 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac2a463740 10 client.26458 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c33da0e7d0 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.093+0000 7fac0ea00640 1 -- 172.21.15.37:0/3473998844 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18004d20 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac0ea00640 1 -- 172.21.15.37:0/3473998844 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fac1800f040 con 0x55c33dcd5c00 2024-03-21T09:15:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac2a463740 10 client.26458 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac2a463740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac2a463740 20 client.26458 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac2a463740 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dce5a70 0x55c33dce7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac2a463740 4 client.26458 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c33dc3d800 con 0x55c33dce5a70 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.094+0000 7fac24e00640 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dce5a70 0x55c33dce7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.095+0000 7fac24e00640 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dce5a70 0x55c33dce7e60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78931922a063f10b server_cookie=a93d19032f79020 in_seq=0 out_seq=0 2024-03-21T09:15:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.095+0000 7fac0ea00640 10 client.26458 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:19.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.095+0000 7fac0ea00640 1 -- 172.21.15.37:0/3473998844 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c33dc3d800 con 0x55c33dce5a70 2024-03-21T09:15:19.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.095+0000 7fac0ea00640 10 client.26458 handle_command_reply: tid=0 2024-03-21T09:15:19.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.095+0000 7fac2a463740 1 client.26458 shutdown 2024-03-21T09:15:19.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 msgr2=0x55c33dcd2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd2f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac18011010 tx=0x7fac18004c70 comp rx=0 tx=0).stop 2024-03-21T09:15:19.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 shutdown_connections 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dcd2f20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dce5a70 0x55c33dce7e60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 --2- 172.21.15.37:0/3473998844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac0ea00640 0 client.26458 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 >> 172.21.15.37:0/3473998844 conn(0x55c33dcc6da0 msgr2=0x55c33dcc8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 shutdown_connections 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 1 -- 172.21.15.37:0/3473998844 wait complete. 2024-03-21T09:15:19.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.096+0000 7fac2a463740 20 client.26458 trim_cache size 0 max 16384 2024-03-21T09:15:19.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 Processor -- start 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 -- start start 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dccfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201b75e0 con 0x55c33dccc6d0 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201b50b0 con 0x55c33dccd080 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac2a463740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201b3150 con 0x55c33dcd5c00 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac25800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dccfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac24e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac17e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44910/0 (socket says 172.21.15.37:44910) 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac17e00640 1 -- 172.21.15.37:0/2774894707 learned_addr learned my addr 172.21.15.37:0/2774894707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.110+0000 7fac25800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dccfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44742/0 (socket says 172.21.15.37:44742) 2024-03-21T09:15:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac25800640 1 -- 172.21.15.37:0/2774894707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 msgr2=0x55c33dcd5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac25800640 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac25800640 1 -- 172.21.15.37:0/2774894707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 msgr2=0x55c33dcd5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac25800640 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac25800640 1 -- 172.21.15.37:0/2774894707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c33dcc68d0 con 0x55c33dcd5c00 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac24e00640 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac25800640 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dccfb70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac18012b70 tx=0x7fac18010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef52928e6ee11050 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac15600640 1 -- 172.21.15.37:0/2774894707 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18016030 con 0x55c33dcd5c00 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac15600640 1 -- 172.21.15.37:0/2774894707 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18003970 con 0x55c33dcd5c00 2024-03-21T09:15:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.111+0000 7fac15600640 1 -- 172.21.15.37:0/2774894707 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180248b0 con 0x55c33dcd5c00 2024-03-21T09:15:19.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 -- 172.21.15.37:0/2774894707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 msgr2=0x55c33dccfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dccfb70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac18012b70 tx=0x7fac18010800 comp rx=0 tx=0).stop 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 -- 172.21.15.37:0/2774894707 shutdown_connections 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dcd5c00 0x55c33dccfb70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccd080 0x55c33dcd5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 --2- 172.21.15.37:0/2774894707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dccc6d0 0x55c33dcd5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 -- 172.21.15.37:0/2774894707 >> 172.21.15.37:0/2774894707 conn(0x55c33dcc6da0 msgr2=0x55c33dcc8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 -- 172.21.15.37:0/2774894707 shutdown_connections 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.112+0000 7fac2a463740 1 -- 172.21.15.37:0/2774894707 wait complete. 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 -- start start 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 Processor -- start 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 0x55c33dcdb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dccd080 0x55c33dcdbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dcd5c00 0x55c33dcdc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201b3750 con 0x55c33dcd5c00 2024-03-21T09:15:19.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201b4a50 con 0x55c33dccc6d0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201b3d90 con 0x55c33dccd080 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.113+0000 7fac2a463740 10 client.0 _ll_register_callbacks cb 0x7fac08009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 0x55c33dcdb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 0x55c33dcdb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44912/0 (socket says 172.21.15.37:44912) 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 -- 172.21.15.37:0/1875207123 learned_addr learned my addr 172.21.15.37:0/1875207123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac17e00640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dccd080 0x55c33dcdbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac25800640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dcd5c00 0x55c33dcdc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 -- 172.21.15.37:0/1875207123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dccd080 msgr2=0x55c33dcdbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dccd080 0x55c33dcdbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 -- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dcd5c00 msgr2=0x55c33dcdc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dcd5c00 0x55c33dcdc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 -- 172.21.15.37:0/1875207123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c33dd40470 con 0x55c33dccc6d0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac24e00640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 0x55c33dcdb6a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac080ba6d0 tx=0x7fac080f1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19d36c26a7a712a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.114+0000 7fac0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:19.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac0ea00640 1 -- 172.21.15.37:0/1875207123 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac080a1060 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac0ea00640 1 -- 172.21.15.37:0/1875207123 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fac080ba370 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac18003970 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac0ea00640 1 -- 172.21.15.37:0/1875207123 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac080c6710 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 10 client.26435 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c33dd417d0 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac0ea00640 1 -- 172.21.15.37:0/1875207123 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18003970 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac0ea00640 1 -- 172.21.15.37:0/1875207123 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7fac080c6280 con 0x55c33dccc6d0 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 10 client.26435 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 20 client.26435 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dd41e30 0x55c33dd44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 4 client.26435 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.115+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c33dc37b50 con 0x55c33dd41e30 2024-03-21T09:15:19.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.116+0000 7fac17e00640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dd41e30 0x55c33dd44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:19.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.116+0000 7fac17e00640 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dd41e30 0x55c33dd44250 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6791bf5d850fdc1 server_cookie=ce63444074a22056 in_seq=0 out_seq=0 2024-03-21T09:15:19.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.116+0000 7fac0ea00640 10 client.26435 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac0ea00640 1 -- 172.21.15.37:0/1875207123 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c33dc37b50 con 0x55c33dd41e30 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac0ea00640 10 client.26435 handle_command_reply: tid=0 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 client.26435 shutdown 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 msgr2=0x55c33dcdb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 0x55c33dcdb6a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac080ba6d0 tx=0x7fac080f1040 comp rx=0 tx=0).stop 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 shutdown_connections 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c33dcd5c00 0x55c33dcdc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c33dccd080 0x55c33dcdbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c33dd41e30 0x55c33dd44250 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 --2- 172.21.15.37:0/1875207123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c33dccc6d0 0x55c33dcdb6a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac0ea00640 0 client.26435 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:19.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 >> 172.21.15.37:0/1875207123 conn(0x55c33dcc6da0 msgr2=0x55c33dcc5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:19.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 shutdown_connections 2024-03-21T09:15:19.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 1 -- 172.21.15.37:0/1875207123 wait complete. 2024-03-21T09:15:19.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.117+0000 7fac2a463740 20 client.26435 trim_cache size 0 max 16384 2024-03-21T09:15:19.115 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fac000a3090 msgr2=0x7fac000a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fac000a3090 0x7fac000a5550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac201017e0 tx=0x7fac10005f90 comp rx=0 tx=0).stop 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 msgr2=0x7fac2019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 0x7fac2019e620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac08005ab0 tx=0x7fac080042e0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 shutdown_connections 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac20112500 0x7fac201a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac200764f0 0x7fac2019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fac000a3090 0x7fac000a5550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 --2- 172.21.15.37:0/2281437210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac20075bd0 0x7fac2019e620 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 >> 172.21.15.37:0/2281437210 conn(0x7fac200ff310 msgr2=0x7fac20113c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:19.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 shutdown_connections 2024-03-21T09:15:19.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:19.118+0000 7fac27800640 1 -- 172.21.15.37:0/2281437210 wait complete. 2024-03-21T09:15:19.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:19 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/868379844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:15:19.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:19 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/868379844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:15:19.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:19 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/868379844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:15:20.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:15:20.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:15:20.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:15:20.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 -- 172.21.15.37:0/1577148779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100d60 msgr2=0x7f010810dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 --2- 172.21.15.37:0/1577148779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100d60 0x7f010810dab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009900 tx=0x7f00f002bc50 comp rx=0 tx=0).stop 2024-03-21T09:15:20.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 -- 172.21.15.37:0/1577148779 shutdown_connections 2024-03-21T09:15:20.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 --2- 172.21.15.37:0/1577148779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f010810dff0 0x7f01081103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 --2- 172.21.15.37:0/1577148779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100d60 0x7f010810dab0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 --2- 172.21.15.37:0/1577148779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0108100440 0x7f0108100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 -- 172.21.15.37:0/1577148779 >> 172.21.15.37:0/1577148779 conn(0x7f01080776f0 msgr2=0x7f0108077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 -- 172.21.15.37:0/1577148779 shutdown_connections 2024-03-21T09:15:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.378+0000 7f010cc00640 1 -- 172.21.15.37:0/1577148779 wait complete. 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.379+0000 7f010cc00640 1 Processor -- start 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.379+0000 7f010cc00640 1 -- start start 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f010cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 0x7f010819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f010cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 0x7f010819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f010cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010810dff0 0x7f010819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f010cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108113ca0 con 0x7f010810dff0 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f010cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108113b20 con 0x7f0108100d60 2024-03-21T09:15:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f010cc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108113e20 con 0x7f0108100440 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 0x7f010819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f00fd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 0x7f010819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 0x7f010819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44766/0 (socket says 172.21.15.37:44766) 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f00fd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 0x7f010819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44922/0 (socket says 172.21.15.37:44922) 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 -- 172.21.15.37:0/3757548855 learned_addr learned my addr 172.21.15.37:0/3757548855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0106800640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010810dff0 0x7f010819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 -- 172.21.15.37:0/3757548855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 msgr2=0x7f010819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 0x7f010819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 -- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010810dff0 msgr2=0x7f010819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010810dff0 0x7f010819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f0105e00640 1 -- 172.21.15.37:0/3757548855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010819f1a0 con 0x7f0108100440 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.380+0000 7f00fd400640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 0x7f010819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.381+0000 7f0105e00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 0x7f010819a200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9f0 tx=0x7f00f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.381+0000 7f00ff400640 1 -- 172.21.15.37:0/3757548855 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cd80 con 0x7f0108100440 2024-03-21T09:15:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.381+0000 7f00ff400640 1 -- 172.21.15.37:0/3757548855 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f8010910 con 0x7f0108100440 2024-03-21T09:15:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.381+0000 7f00ff400640 1 -- 172.21.15.37:0/3757548855 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8018990 con 0x7f0108100440 2024-03-21T09:15:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.381+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010819f490 con 0x7f0108100440 2024-03-21T09:15:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.381+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01081a6d20 con 0x7f0108100440 2024-03-21T09:15:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.384+0000 7f00ff400640 1 -- 172.21.15.37:0/3757548855 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f00f8010ac0 con 0x7f0108100440 2024-03-21T09:15:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.384+0000 7f010f991740 1 Processor -- start 2024-03-21T09:15:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 -- start start 2024-03-21T09:15:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69f4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 0x5561d69fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d69ff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d6958eb0 con 0x5561d69f4080 2024-03-21T09:15:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d6901a60 con 0x5561d69fd580 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f010f991740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d6954f50 con 0x5561d69f49a0 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f0105e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 0x5561d69fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f00fd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d69ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f0106800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f0105e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 0x5561d69fd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44780/0 (socket says 172.21.15.37:44780) 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f0105e00640 1 -- 172.21.15.37:0/574441380 learned_addr learned my addr 172.21.15.37:0/574441380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.385+0000 7f00fd400640 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d69ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44930/0 (socket says 172.21.15.37:44930) 2024-03-21T09:15:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0105e00640 1 -- 172.21.15.37:0/574441380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 msgr2=0x5561d69ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0105e00640 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d69ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0105e00640 1 -- 172.21.15.37:0/574441380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 msgr2=0x5561d69f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0105e00640 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0105e00640 1 -- 172.21.15.37:0/574441380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d67357e0 con 0x5561d69f49a0 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f00ff400640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f00ec0a3110 0x7f00ec0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f00ff400640 1 -- 172.21.15.37:0/3757548855 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f00f80cfe10 con 0x7f0108100440 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0105e00640 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 0x5561d69fd040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00f80cd130 tx=0x7f00f80d3f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5d2a768ba9eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f0106800640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f00ec0a3110 0x7f00ec0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f00f7000640 1 -- 172.21.15.37:0/574441380 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80981b0 con 0x5561d69f49a0 2024-03-21T09:15:20.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f00f7000640 1 -- 172.21.15.37:0/574441380 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f809bf10 con 0x5561d69f49a0 2024-03-21T09:15:20.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.386+0000 7f00f7000640 1 -- 172.21.15.37:0/574441380 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f809b870 con 0x5561d69f49a0 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 -- 172.21.15.37:0/574441380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 msgr2=0x5561d69fd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 0x5561d69fd040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00f80cd130 tx=0x7f00f80d3f90 comp rx=0 tx=0).stop 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f0106800640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f00ec0a3110 0x7f00ec0a55d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0108068810 tx=0x7f01000096a0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 -- 172.21.15.37:0/574441380 shutdown_connections 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d69ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f49a0 0x5561d69fd040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 --2- 172.21.15.37:0/574441380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.387+0000 7f010f991740 1 -- 172.21.15.37:0/574441380 >> 172.21.15.37:0/574441380 conn(0x5561d69ee7b0 msgr2=0x5561d69eeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.388+0000 7f010f991740 1 -- 172.21.15.37:0/574441380 shutdown_connections 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.388+0000 7f010f991740 1 -- 172.21.15.37:0/574441380 wait complete. 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.388+0000 7f010f991740 1 -- start start 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.388+0000 7f010f991740 1 Processor -- start 2024-03-21T09:15:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f49a0 0x5561d69fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d6a00370 con 0x5561d69f4080 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d6914eb0 con 0x5561d69f49a0 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d6a004f0 con 0x5561d69fd580 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f010f991740 10 client.0 _ll_register_callbacks cb 0x5561d6909150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f0105e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f00fd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f49a0 0x5561d69fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.389+0000 7f0105e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46852/0 (socket says 172.21.15.37:46852) 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 -- 172.21.15.37:0/3233785835 learned_addr learned my addr 172.21.15.37:0/3233785835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0106800640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 -- 172.21.15.37:0/3233785835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 msgr2=0x5561d69fa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69fa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 -- 172.21.15.37:0/3233785835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f49a0 msgr2=0x5561d69fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f49a0 0x5561d69fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 -- 172.21.15.37:0/3233785835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561d6795f70 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0106800640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f00fd400640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f49a0 0x5561d69fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f0105e00640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69fcef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f00f80cd270 tx=0x7f00f80d0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e562a1c1e1b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f00eb400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.390+0000 7f00eb400640 1 -- 172.21.15.37:0/3233785835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f809ba50 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f00eb400640 1 -- 172.21.15.37:0/3233785835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f00f80d8d00 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f809bf10 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f00eb400640 1 -- 172.21.15.37:0/3233785835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f809bce0 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f010f991740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5561d662d170 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f00eb400640 1 -- 172.21.15.37:0/3233785835 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f809bf10 con 0x5561d69f4080 2024-03-21T09:15:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f00eb400640 1 -- 172.21.15.37:0/3233785835 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f00f809c170 con 0x5561d69f4080 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f010f991740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f010f991740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.391+0000 7f010f991740 20 client.16248 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.392+0000 7f010f991740 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a0d460 0x5561d6a0f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.392+0000 7f010f991740 4 client.16248 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.392+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561d69663f0 con 0x5561d6a0d460 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.392+0000 7f00fd400640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a0d460 0x5561d6a0f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.392+0000 7f00fd400640 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a0d460 0x5561d6a0f850 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae15fefb93d926f5 server_cookie=5da4acbad5ff05b5 in_seq=0 out_seq=0 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.392+0000 7f00eb400640 10 client.16248 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f00eb400640 1 -- 172.21.15.37:0/3233785835 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5561d69663f0 con 0x5561d6a0d460 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f00eb400640 10 client.16248 handle_command_reply: tid=0 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 client.16248 shutdown 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 msgr2=0x5561d69fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69fcef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f00f80cd270 tx=0x7f00f80d0180 comp rx=0 tx=0).stop 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 shutdown_connections 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a0d460 0x5561d6a0f850 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f49a0 0x5561d69fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 --2- 172.21.15.37:0/3233785835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f4080 0x5561d69fcef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 >> 172.21.15.37:0/3233785835 conn(0x5561d69ee7b0 msgr2=0x5561d69f0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.393+0000 7f00eb400640 0 client.16248 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.394+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 shutdown_connections 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.394+0000 7f010f991740 1 -- 172.21.15.37:0/3233785835 wait complete. 2024-03-21T09:15:20.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.394+0000 7f010f991740 20 client.16248 trim_cache size 0 max 16384 2024-03-21T09:15:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 Processor -- start 2024-03-21T09:15:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 -- start start 2024-03-21T09:15:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 0x5561d69fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d69fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081b3190 con 0x5561d69f49a0 2024-03-21T09:15:20.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081b0c60 con 0x5561d69f4080 2024-03-21T09:15:20.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f010f991740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081aed00 con 0x5561d69fd580 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f0105e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 0x5561d69fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f0106800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f00fd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d69fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f0105e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 0x5561d69fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44946/0 (socket says 172.21.15.37:44946) 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f0106800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44792/0 (socket says 172.21.15.37:44792) 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.412+0000 7f0105e00640 1 -- 172.21.15.37:0/3693815713 learned_addr learned my addr 172.21.15.37:0/3693815713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f0105e00640 1 -- 172.21.15.37:0/3693815713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 msgr2=0x5561d69f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f0105e00640 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f0105e00640 1 -- 172.21.15.37:0/3693815713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 msgr2=0x5561d69fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f0105e00640 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d69fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f0105e00640 1 -- 172.21.15.37:0/3693815713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d6a003f0 con 0x5561d69f4080 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f0105e00640 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 0x5561d69fc9b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00f809bc10 tx=0x7f00f80d0110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10638b21acf8aea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f00f6600640 1 -- 172.21.15.37:0/3693815713 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80a9070 con 0x5561d69f4080 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f00f6600640 1 -- 172.21.15.37:0/3693815713 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f80bac20 con 0x5561d69f4080 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.413+0000 7f00f6600640 1 -- 172.21.15.37:0/3693815713 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f809add0 con 0x5561d69f4080 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 -- 172.21.15.37:0/3693815713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 msgr2=0x5561d69fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 0x5561d69fc9b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00f809bc10 tx=0x7f00f80d0110 comp rx=0 tx=0).stop 2024-03-21T09:15:20.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 -- 172.21.15.37:0/3693815713 shutdown_connections 2024-03-21T09:15:20.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69fd580 0x5561d69f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d69fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 --2- 172.21.15.37:0/3693815713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69f4080 0x5561d69fc9b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 -- 172.21.15.37:0/3693815713 >> 172.21.15.37:0/3693815713 conn(0x5561d69ee7b0 msgr2=0x5561d6a08fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 -- 172.21.15.37:0/3693815713 shutdown_connections 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.414+0000 7f010f991740 1 -- 172.21.15.37:0/3693815713 wait complete. 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.415+0000 7f010f991740 1 -- start start 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.415+0000 7f010f991740 1 Processor -- start 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 0x5561d6a122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d6a106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d6a10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081af300 con 0x5561d69f49a0 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081b0600 con 0x5561d69fd580 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081af940 con 0x5561d69f4080 2024-03-21T09:15:20.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f010f991740 10 client.0 _ll_register_callbacks cb 0x7f0100009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 0x5561d6a122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 0x5561d6a122e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44804/0 (socket says 172.21.15.37:44804) 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 -- 172.21.15.37:0/3090724483 learned_addr learned my addr 172.21.15.37:0/3090724483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0106800640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d6a10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f00fd400640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d6a106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 -- 172.21.15.37:0/3090724483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 msgr2=0x5561d6a10c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d6a10c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 -- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 msgr2=0x5561d6a106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d6a106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.416+0000 7f0105e00640 1 -- 172.21.15.37:0/3090724483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561d6a11150 con 0x5561d69f4080 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f0105e00640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 0x5561d6a122e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00f80bae00 tx=0x7f00f8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0a24da65caa0c20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f00eb400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f00eb400640 1 -- 172.21.15.37:0/3090724483 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80a9030 con 0x5561d69f4080 2024-03-21T09:15:20.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f00eb400640 1 -- 172.21.15.37:0/3090724483 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f00f809a210 con 0x5561d69f4080 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561d6960740 con 0x5561d69f4080 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f00eb400640 1 -- 172.21.15.37:0/3090724483 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f809b870 con 0x5561d69f4080 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f010f991740 10 client.26482 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5561d6a11150 con 0x5561d69f4080 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.417+0000 7f00eb400640 1 -- 172.21.15.37:0/3090724483 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5561d6960740 con 0x5561d69f4080 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f00eb400640 1 -- 172.21.15.37:0/3090724483 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f00f80a43e0 con 0x5561d69f4080 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f010f991740 10 client.26482 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f010f991740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f010f991740 20 client.26482 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f010f991740 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a699c0 0x5561d6a6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f010f991740 4 client.26482 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:15:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5561d68dea40 con 0x5561d6a699c0 2024-03-21T09:15:20.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.418+0000 7f00fd400640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a699c0 0x5561d6a6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.419+0000 7f00fd400640 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a699c0 0x5561d6a6bde0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b15aaab72993c102 server_cookie=ecdb5e7938c4ff11 in_seq=0 out_seq=0 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.419+0000 7f00eb400640 10 client.26482 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.420+0000 7f00eb400640 1 -- 172.21.15.37:0/3090724483 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5561d68dea40 con 0x5561d6a699c0 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.420+0000 7f00eb400640 10 client.26482 handle_command_reply: tid=0 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.420+0000 7f010f991740 1 client.26482 shutdown 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 msgr2=0x5561d6a122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 0x5561d6a122e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00f80bae00 tx=0x7f00f8009310 comp rx=0 tx=0).stop 2024-03-21T09:15:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 shutdown_connections 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5561d6a699c0 0x5561d6a6bde0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561d69fd580 0x5561d6a10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561d69f49a0 0x5561d6a106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 --2- 172.21.15.37:0/3090724483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561d69f4080 0x5561d6a122e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 >> 172.21.15.37:0/3090724483 conn(0x5561d69ee7b0 msgr2=0x5561d69efb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f00eb400640 0 client.26482 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 shutdown_connections 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 1 -- 172.21.15.37:0/3090724483 wait complete. 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.421+0000 7f010f991740 20 client.26482 trim_cache size 0 max 16384 2024-03-21T09:15:20.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:15:20.418 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:15:20.418 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "63eaecb7-5a85-49de-8fb0-83222ef61d19", 2024-03-21T09:15:20.418 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:15:20.418 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:15:20.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f00ec0a3110 msgr2=0x7f00ec0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f00ec0a3110 0x7f00ec0a55d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0108068810 tx=0x7f01000096a0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 msgr2=0x7f010819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 0x7f010819a200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9f0 tx=0x7f00f800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 shutdown_connections 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010810dff0 0x7f010819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f00ec0a3110 0x7f00ec0a55d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0108100d60 0x7f010819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 --2- 172.21.15.37:0/3757548855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0108100440 0x7f010819a200 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.422+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 >> 172.21.15.37:0/3757548855 conn(0x7f01080776f0 msgr2=0x7f010810f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.423+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 shutdown_connections 2024-03-21T09:15:20.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.423+0000 7f010cc00640 1 -- 172.21.15.37:0/3757548855 wait complete. 2024-03-21T09:15:20.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:15:20.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:15:20.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:20 smithi037 ceph-mon[31830]: pgmap v97: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:15:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:20 smithi071 ceph-mon[36597]: pgmap v97: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:15:20.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:20 smithi053 ceph-mon[36169]: pgmap v97: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:15:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 -- 172.21.15.37:0/3657430937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 msgr2=0x7f892810d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 --2- 172.21.15.37:0/3657430937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892810d7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8910009900 tx=0x7f891002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 -- 172.21.15.37:0/3657430937 shutdown_connections 2024-03-21T09:15:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 --2- 172.21.15.37:0/3657430937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f89281100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 --2- 172.21.15.37:0/3657430937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892810d7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 --2- 172.21.15.37:0/3657430937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8928073a70 0x7f8928073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 -- 172.21.15.37:0/3657430937 >> 172.21.15.37:0/3657430937 conn(0x7f89280fcf50 msgr2=0x7f89280ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 -- 172.21.15.37:0/3657430937 shutdown_connections 2024-03-21T09:15:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.709+0000 7f8927e00640 1 -- 172.21.15.37:0/3657430937 wait complete. 2024-03-21T09:15:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.711+0000 7f8927e00640 1 Processor -- start 2024-03-21T09:15:20.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.711+0000 7f8927e00640 1 -- start start 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8928073a70 0x7f8928199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f892819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8928113a40 con 0x7f8928073a70 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281138c0 con 0x7f8928074390 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8928113bc0 con 0x7f892810dce0 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f891fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8927400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f892819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.712+0000 7f8926a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8928073a70 0x7f8928199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44952/0 (socket says 172.21.15.37:44952) 2024-03-21T09:15:20.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 -- 172.21.15.37:0/528943973 learned_addr learned my addr 172.21.15.37:0/528943973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f8927400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f892819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44828/0 (socket says 172.21.15.37:44828) 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 -- 172.21.15.37:0/528943973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 msgr2=0x7f892819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f892819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 -- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8928073a70 msgr2=0x7f8928199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8928073a70 0x7f8928199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 -- 172.21.15.37:0/528943973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892819efd0 con 0x7f8928074390 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f8927400640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f892819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f891fe00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892819a520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8910005940 tx=0x7f891002cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.713+0000 7f892ca00640 1 -- 172.21.15.37:0/528943973 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910041070 con 0x7f8928074390 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.714+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892819f260 con 0x7f8928074390 2024-03-21T09:15:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.714+0000 7f892ca00640 1 -- 172.21.15.37:0/528943973 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8910004450 con 0x7f8928074390 2024-03-21T09:15:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.714+0000 7f892ca00640 1 -- 172.21.15.37:0/528943973 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910003d80 con 0x7f8928074390 2024-03-21T09:15:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.714+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89281a6b50 con 0x7f8928074390 2024-03-21T09:15:20.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.717+0000 7f892ca00640 1 -- 172.21.15.37:0/528943973 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8910044070 con 0x7f8928074390 2024-03-21T09:15:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.718+0000 7f8930a36740 1 Processor -- start 2024-03-21T09:15:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.718+0000 7f892ca00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f890c0a3090 0x7f890c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 -- start start 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f892ca00640 1 -- 172.21.15.37:0/528943973 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f89100f15a0 con 0x7f8928074390 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8927400640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f890c0a3090 0x7f890c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a69b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca69ce250 con 0x55eca6a6add0 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca6976a30 con 0x55eca6a73460 2024-03-21T09:15:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8930a36740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca69ca2f0 con 0x55eca6a6a0f0 2024-03-21T09:15:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f8926a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a69b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f891fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f891fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44844/0 (socket says 172.21.15.37:44844) 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.719+0000 7f891fe00640 1 -- 172.21.15.37:0/3326802040 learned_addr learned my addr 172.21.15.37:0/3326802040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f8927400640 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f891fe00640 1 -- 172.21.15.37:0/3326802040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 msgr2=0x55eca6a75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f891fe00640 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f8927400640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f890c0a3090 0x7f890c0a5550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f892806abb0 tx=0x7f8920009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f891fe00640 1 -- 172.21.15.37:0/3326802040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 msgr2=0x55eca6a69b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f891fe00640 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a69b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f891fe00640 1 -- 172.21.15.37:0/3326802040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca6961690 con 0x55eca6a6a0f0 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f891fe00640 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6a570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f89100ba120 tx=0x7f8910009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2b9d6bea2ca5f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f8917e00640 1 -- 172.21.15.37:0/3326802040 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891004b040 con 0x55eca6a6a0f0 2024-03-21T09:15:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f8917e00640 1 -- 172.21.15.37:0/3326802040 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8910004c80 con 0x55eca6a6a0f0 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.720+0000 7f8917e00640 1 -- 172.21.15.37:0/3326802040 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100bbae0 con 0x55eca6a6a0f0 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 -- 172.21.15.37:0/3326802040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 msgr2=0x55eca6a6a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6a570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f89100ba120 tx=0x7f8910009680 comp rx=0 tx=0).stop 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 -- 172.21.15.37:0/3326802040 shutdown_connections 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6a570 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 --2- 172.21.15.37:0/3326802040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a69b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.721+0000 7f8930a36740 1 -- 172.21.15.37:0/3326802040 >> 172.21.15.37:0/3326802040 conn(0x55eca6a31200 msgr2=0x55eca6a64b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.722+0000 7f8930a36740 1 -- 172.21.15.37:0/3326802040 shutdown_connections 2024-03-21T09:15:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.722+0000 7f8930a36740 1 -- 172.21.15.37:0/3326802040 wait complete. 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.722+0000 7f8930a36740 1 -- start start 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 Processor -- start 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 0x55eca6a6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a70450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca6a76090 con 0x55eca6a6a0f0 2024-03-21T09:15:20.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca6989e80 con 0x55eca6a6add0 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca6a76210 con 0x55eca6a73460 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.723+0000 7f8930a36740 10 client.0 _ll_register_callbacks cb 0x55eca699c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8926a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 0x55eca6a6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8927400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a70450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8926a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46910/0 (socket says 172.21.15.37:46910) 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 0x55eca6a6ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44970/0 (socket says 172.21.15.37:44970) 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8926a00640 1 -- 172.21.15.37:0/51661088 learned_addr learned my addr 172.21.15.37:0/51661088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 -- 172.21.15.37:0/51661088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 msgr2=0x55eca6a70450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a70450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 -- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6a0f0 msgr2=0x55eca6a72b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 -- 172.21.15.37:0/51661088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eca67ac7d0 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f891fe00640 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 0x55eca6a6ff10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55eca6a6b320 tx=0x7f89100b93f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453e8f43876641a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8914c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8914c00640 1 -- 172.21.15.37:0/51661088 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891004b040 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8914c00640 1 -- 172.21.15.37:0/51661088 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f89100eeb70 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.724+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8910004c80 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8914c00640 1 -- 172.21.15.37:0/51661088 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89100c17f0 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 10 client.26465 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eca66e23f0 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8914c00640 1 -- 172.21.15.37:0/51661088 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8910004c80 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8914c00640 1 -- 172.21.15.37:0/51661088 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f89100c5c20 con 0x55eca6a6add0 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 10 client.26465 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 20 client.26465 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6a82f20 0x55eca6a85310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 4 client.26465 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.725+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eca69db790 con 0x55eca6a82f20 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.726+0000 7f8926a00640 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6a82f20 0x55eca6a85310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.726+0000 7f8926a00640 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6a82f20 0x55eca6a85310 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98e019a4de25847 server_cookie=d8d5ff8d2fa4b650 in_seq=0 out_seq=0 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.726+0000 7f8914c00640 10 client.26465 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.727+0000 7f8914c00640 1 -- 172.21.15.37:0/51661088 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eca69db790 con 0x55eca6a82f20 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.727+0000 7f8914c00640 10 client.26465 handle_command_reply: tid=0 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.727+0000 7f8930a36740 1 client.26465 shutdown 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 msgr2=0x55eca6a6ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 0x55eca6a6ff10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55eca6a6b320 tx=0x7f89100b93f0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 shutdown_connections 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a70450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6a82f20 0x55eca6a85310 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6add0 0x55eca6a6ff10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 --2- 172.21.15.37:0/51661088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 >> 172.21.15.37:0/51661088 conn(0x55eca6a31200 msgr2=0x55eca6a66890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8914c00640 0 client.26465 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 shutdown_connections 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 1 -- 172.21.15.37:0/51661088 wait complete. 2024-03-21T09:15:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.728+0000 7f8930a36740 20 client.26465 trim_cache size 0 max 16384 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 Processor -- start 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 -- start start 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a74850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281b3050 con 0x55eca6a6add0 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281b0b20 con 0x55eca6a6a0f0 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8930a36740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281aebc0 con 0x55eca6a73460 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8926a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f891fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a72b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.746+0000 7f8926a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44984/0 (socket says 172.21.15.37:44984) 2024-03-21T09:15:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 -- 172.21.15.37:0/2620610329 learned_addr learned my addr 172.21.15.37:0/2620610329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8927400640 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a74850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 -- 172.21.15.37:0/2620610329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 msgr2=0x55eca6a74850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a74850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 -- 172.21.15.37:0/2620610329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 msgr2=0x55eca6a72b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a72b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 -- 172.21.15.37:0/2620610329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca66a20f0 con 0x55eca6a6a0f0 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f891fe00640 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a72b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8926a00640 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8918009e30 tx=0x7f8918020b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b7f28c3e4b7006 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8917400640 1 -- 172.21.15.37:0/2620610329 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918020110 con 0x55eca6a6a0f0 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8917400640 1 -- 172.21.15.37:0/2620610329 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8918026070 con 0x55eca6a6a0f0 2024-03-21T09:15:20.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.747+0000 7f8917400640 1 -- 172.21.15.37:0/2620610329 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891801eaa0 con 0x55eca6a6a0f0 2024-03-21T09:15:20.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 -- 172.21.15.37:0/2620610329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 msgr2=0x55eca6a72640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8918009e30 tx=0x7f8918020b40 comp rx=0 tx=0).stop 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 -- 172.21.15.37:0/2620610329 shutdown_connections 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a73460 0x55eca6a74850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a72b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 --2- 172.21.15.37:0/2620610329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a6a0f0 0x55eca6a72640 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 -- 172.21.15.37:0/2620610329 >> 172.21.15.37:0/2620610329 conn(0x55eca6a31200 msgr2=0x55eca6a70030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 -- 172.21.15.37:0/2620610329 shutdown_connections 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.748+0000 7f8930a36740 1 -- 172.21.15.37:0/2620610329 wait complete. 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 -- start start 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 Processor -- start 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a7e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a7e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281af1c0 con 0x55eca6a6add0 2024-03-21T09:15:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281b04c0 con 0x55eca6a73460 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.749+0000 7f8930a36740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281af800 con 0x55eca6a6a0f0 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f8930a36740 10 client.0 _ll_register_callbacks cb 0x7f89100c1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a7e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a7e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46922/0 (socket says 172.21.15.37:46922) 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 -- 172.21.15.37:0/4180795153 learned_addr learned my addr 172.21.15.37:0/4180795153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f8927400640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a7e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f8926a00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 -- 172.21.15.37:0/4180795153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 msgr2=0x55eca6a6d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 -- 172.21.15.37:0/4180795153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 msgr2=0x55eca6a7e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a7e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f891fe00640 1 -- 172.21.15.37:0/4180795153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eca6a7ef30 con 0x55eca6a6add0 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f8927400640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a7e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:20.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.750+0000 7f8926a00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f891fe00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a7e3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8910036040 tx=0x7f89100c5e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ab81e5930fb882 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8914c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8914c00640 1 -- 172.21.15.37:0/4180795153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891004b040 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8914c00640 1 -- 172.21.15.37:0/4180795153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f89100eece0 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8918026070 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8914c00640 1 -- 172.21.15.37:0/4180795153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89100bd520 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8930a36740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eca6a7ef30 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.751+0000 7f8914c00640 1 -- 172.21.15.37:0/4180795153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8918026070 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8914c00640 1 -- 172.21.15.37:0/4180795153 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f89100bc610 con 0x55eca6a6add0 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8930a36740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8930a36740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:20.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8930a36740 20 client.16272 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:20.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8930a36740 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6adf700 0x55eca6ae1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:20.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8930a36740 4 client.16272 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:15:20.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eca69d5ae0 con 0x55eca6adf700 2024-03-21T09:15:20.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.752+0000 7f8926a00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6adf700 0x55eca6ae1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8926a00640 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6adf700 0x55eca6ae1af0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0bc61845c5d6e38 server_cookie=1aad379f59890f9f in_seq=0 out_seq=0 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8914c00640 10 client.16272 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8914c00640 1 -- 172.21.15.37:0/4180795153 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eca69d5ae0 con 0x55eca6adf700 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8914c00640 10 client.16272 handle_command_reply: tid=0 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8930a36740 1 client.16272 shutdown 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 msgr2=0x55eca6a7e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.753+0000 7f8930a36740 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a7e3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8910036040 tx=0x7f89100c5e50 comp rx=0 tx=0).stop 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 shutdown_connections 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55eca6adf700 0x55eca6ae1af0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eca6a73460 0x55eca6a7e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eca6a6add0 0x55eca6a7e3b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 --2- 172.21.15.37:0/4180795153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eca6a6a0f0 0x55eca6a6d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8914c00640 0 client.16272 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 >> 172.21.15.37:0/4180795153 conn(0x55eca6a31200 msgr2=0x55eca6a63330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 shutdown_connections 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 1 -- 172.21.15.37:0/4180795153 wait complete. 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8930a36740 20 client.16272 trim_cache size 0 max 16384 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:15:20.751 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout: "63eaecb7-5a85-49de-8fb0-83222ef61d19": { 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "63eaecb7-5a85-49de-8fb0-83222ef61d19", 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:15:20.752 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f890c0a3090 msgr2=0x7f890c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8927e00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f890c0a3090 0x7f890c0a5550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f892806abb0 tx=0x7f8920009500 comp rx=0 tx=0).stop 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 msgr2=0x7f892819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.754+0000 7f8927e00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892819a520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8910005940 tx=0x7f891002cbd0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 shutdown_connections 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f892810dce0 0x7f892819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f890c0a3090 0x7f890c0a5550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8928074390 0x7f892819a520 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 --2- 172.21.15.37:0/528943973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8928073a70 0x7f8928199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 >> 172.21.15.37:0/528943973 conn(0x7f89280fcf50 msgr2=0x7f89280fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 shutdown_connections 2024-03-21T09:15:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:20.755+0000 7f8927e00640 1 -- 172.21.15.37:0/528943973 wait complete. 2024-03-21T09:15:20.765 INFO:tasks.cephfs.filesystem:scrub status for tag:63eaecb7-5a85-49de-8fb0-83222ef61d19 - {'path': '/', 'tag': '63eaecb7-5a85-49de-8fb0-83222ef61d19', 'options': 'recursive,force'} 2024-03-21T09:15:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:21 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:15:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:21 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:21 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:15:21.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:21 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:21.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:21 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:15:21.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:21 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:21.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:21 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:15:21.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:21 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:21.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:21 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:15:21.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:21 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:21.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:21 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:15:21.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:21 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:22 smithi071 ceph-mon[36597]: pgmap v98: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:15:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:22 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:15:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:22 smithi037 ceph-mon[31830]: pgmap v98: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:15:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:22 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:15:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:22 smithi053 ceph-mon[36169]: pgmap v98: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:15:22.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:22 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:15:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:24 smithi071 ceph-mon[36597]: pgmap v99: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-03-21T09:15:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:24 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:15:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:24 smithi037 ceph-mon[31830]: pgmap v99: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-03-21T09:15:24.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:24 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:15:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:24 smithi053 ceph-mon[36169]: pgmap v99: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-03-21T09:15:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:24 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:15:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:26 smithi071 ceph-mon[36597]: pgmap v100: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:26 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:15:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:26 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:15:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:26 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:15:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:26 smithi037 ceph-mon[31830]: pgmap v100: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:26 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:15:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:26 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:15:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:26 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:15:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:26 smithi053 ceph-mon[36169]: pgmap v100: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:26 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:15:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:26 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:15:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:26 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:15:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:28 smithi071 ceph-mon[36597]: pgmap v101: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:28.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:28 smithi037 ceph-mon[31830]: pgmap v101: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:28.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:28 smithi053 ceph-mon[36169]: pgmap v101: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:30 smithi071 ceph-mon[36597]: pgmap v102: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:30.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:30 smithi037 ceph-mon[31830]: pgmap v102: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:30.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:30 smithi053 ceph-mon[36169]: pgmap v102: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:32.352 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:31 smithi071 ceph-mon[36597]: pgmap v103: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:32.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:31 smithi037 ceph-mon[31830]: pgmap v103: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:32.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:31 smithi053 ceph-mon[36169]: pgmap v103: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:15:34.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:34 smithi037 ceph-mon[31830]: pgmap v104: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:34.440 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:34 smithi053 ceph-mon[36169]: pgmap v104: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:34 smithi071 ceph-mon[36597]: pgmap v104: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:36.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:36 smithi053 ceph-mon[36169]: pgmap v105: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:36.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:36 smithi071 ceph-mon[36597]: pgmap v105: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:36.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:36 smithi037 ceph-mon[31830]: pgmap v105: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:38.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:38 smithi053 ceph-mon[36169]: pgmap v106: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:38 smithi071 ceph-mon[36597]: pgmap v106: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:38.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:38 smithi037 ceph-mon[31830]: pgmap v106: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:40.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:40 smithi053 ceph-mon[36169]: pgmap v107: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:40.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:40 smithi071 ceph-mon[36597]: pgmap v107: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:40.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:40 smithi037 ceph-mon[31830]: pgmap v107: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:42.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:42 smithi053 ceph-mon[36169]: pgmap v108: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:42.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:42 smithi071 ceph-mon[36597]: pgmap v108: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:42.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:42 smithi037 ceph-mon[31830]: pgmap v108: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:44.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:44 smithi071 ceph-mon[36597]: pgmap v109: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:44.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:44 smithi037 ceph-mon[31830]: pgmap v109: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:44.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:44 smithi053 ceph-mon[36169]: pgmap v109: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:46.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:46 smithi071 ceph-mon[36597]: pgmap v110: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:46.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:46 smithi037 ceph-mon[31830]: pgmap v110: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:46.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:46 smithi053 ceph-mon[36169]: pgmap v110: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:48.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:48 smithi037 ceph-mon[31830]: pgmap v111: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:48.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:48 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:15:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:48 smithi071 ceph-mon[36597]: pgmap v111: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:48.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:48 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:15:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:48 smithi053 ceph-mon[36169]: pgmap v111: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:48 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:15:50.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:50 smithi071 ceph-mon[36597]: pgmap v112: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:50.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:50 smithi037 ceph-mon[31830]: pgmap v112: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:50.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:50 smithi053 ceph-mon[36169]: pgmap v112: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:50.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:15:50.766 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 -- 172.21.15.37:0/3192696938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 msgr2=0x7f074c103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 --2- 172.21.15.37:0/3192696938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c103b00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0738009890 tx=0x7f073802bb70 comp rx=0 tx=0).stop 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 -- 172.21.15.37:0/3192696938 shutdown_connections 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 --2- 172.21.15.37:0/3192696938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 0x7f074c110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 --2- 172.21.15.37:0/3192696938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c103b00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 --2- 172.21.15.37:0/3192696938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f074c109690 0x7f074c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 -- 172.21.15.37:0/3192696938 >> 172.21.15.37:0/3192696938 conn(0x7f074c0ff110 msgr2=0x7f074c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 -- 172.21.15.37:0/3192696938 shutdown_connections 2024-03-21T09:15:50.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.001+0000 7f074b200640 1 -- 172.21.15.37:0/3192696938 wait complete. 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.002+0000 7f074b200640 1 Processor -- start 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.002+0000 7f074b200640 1 -- start start 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 0x7f074c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f074c109690 0x7f074c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c113a60 con 0x7f074c109690 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1138e0 con 0x7f074c104040 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c113be0 con 0x7f074c103680 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 0x7f074c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f074a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f074c109690 0x7f074c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45674/0 (socket says 172.21.15.37:45674) 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 -- 172.21.15.37:0/2666680905 learned_addr learned my addr 172.21.15.37:0/2666680905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 -- 172.21.15.37:0/2666680905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 msgr2=0x7f074c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 0x7f074c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 -- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f074c109690 msgr2=0x7f074c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f074c109690 0x7f074c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749e00640 1 -- 172.21.15.37:0/2666680905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c19ef90 con 0x7f074c103680 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.003+0000 7f0749400640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 0x7f074c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.004+0000 7f0749e00640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c199fa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f073c00e9f0 tx=0x7f073c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.004+0000 7f0742a00640 1 -- 172.21.15.37:0/2666680905 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00cd80 con 0x7f074c103680 2024-03-21T09:15:51.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.004+0000 7f0742a00640 1 -- 172.21.15.37:0/2666680905 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f073c010910 con 0x7f074c103680 2024-03-21T09:15:51.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.004+0000 7f0742a00640 1 -- 172.21.15.37:0/2666680905 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c018990 con 0x7f074c103680 2024-03-21T09:15:51.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.004+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c19f280 con 0x7f074c103680 2024-03-21T09:15:51.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.004+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c1a6b10 con 0x7f074c103680 2024-03-21T09:15:51.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.007+0000 7f0742a00640 1 -- 172.21.15.37:0/2666680905 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f073c010ac0 con 0x7f074c103680 2024-03-21T09:15:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.007+0000 7f07535ff740 1 Processor -- start 2024-03-21T09:15:51.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 -- start start 2024-03-21T09:15:51.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x56099fff5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x56099fff6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0001850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099ff5a250 con 0x56099fff6dd0 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099ff02a30 con 0x56099ffff460 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f07535ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099ff562f0 con 0x56099fff60f0 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f0749e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x56099fff6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f0749e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x56099fff6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45680/0 (socket says 172.21.15.37:45680) 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f0749e00640 1 -- 172.21.15.37:0/1317063226 learned_addr learned my addr 172.21.15.37:0/1317063226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f074a800640 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x56099fff5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.008+0000 7f0749400640 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0001850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0749e00640 1 -- 172.21.15.37:0/1317063226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 msgr2=0x5609a0001850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0749e00640 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0001850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0749e00640 1 -- 172.21.15.37:0/1317063226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 msgr2=0x56099fff5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0749e00640 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x56099fff5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0749e00640 1 -- 172.21.15.37:0/1317063226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56099feed690 con 0x56099fff60f0 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0749e00640 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x56099fff6570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f073c0cd110 tx=0x7f073c0d8570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e22c2ec7fcbbe6d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0742a00640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07300a3090 0x7f07300a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.009+0000 7f0737000640 1 -- 172.21.15.37:0/1317063226 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c0e3070 con 0x56099fff60f0 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.010+0000 7f0737000640 1 -- 172.21.15.37:0/1317063226 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f073c096810 con 0x56099fff60f0 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.010+0000 7f0737000640 1 -- 172.21.15.37:0/1317063226 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c088170 con 0x56099fff60f0 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.010+0000 7f0742a00640 1 -- 172.21.15.37:0/2666680905 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f073c0cfe80 con 0x7f074c103680 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.010+0000 7f074a800640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07300a3090 0x7f07300a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.010+0000 7f07535ff740 1 -- 172.21.15.37:0/1317063226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 msgr2=0x56099fff6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.010+0000 7f07535ff740 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x56099fff6570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f073c0cd110 tx=0x7f073c0d8570 comp rx=0 tx=0).stop 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 -- 172.21.15.37:0/1317063226 shutdown_connections 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0001850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x56099fff6570 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 --2- 172.21.15.37:0/1317063226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x56099fff5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 -- 172.21.15.37:0/1317063226 >> 172.21.15.37:0/1317063226 conn(0x56099ffbd200 msgr2=0x56099fff0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f074a800640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07300a3090 0x7f07300a5550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0744005fd0 tx=0x7f0744010040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 -- 172.21.15.37:0/1317063226 shutdown_connections 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 -- 172.21.15.37:0/1317063226 wait complete. 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 -- start start 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.011+0000 7f07535ff740 1 Processor -- start 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x56099fffc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0002320 con 0x56099fff60f0 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099ff15e80 con 0x56099fff6dd0 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a00024a0 con 0x56099ffff460 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f07535ff740 10 client.0 _ll_register_callbacks cb 0x56099ff28130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f0749400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f0749400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33174/0 (socket says 172.21.15.37:33174) 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f0749400640 1 -- 172.21.15.37:0/976961733 learned_addr learned my addr 172.21.15.37:0/976961733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f074a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x56099fffc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.012+0000 7f0749e00640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f074a800640 1 -- 172.21.15.37:0/976961733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 msgr2=0x56099fffbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f074a800640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f074a800640 1 -- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 msgr2=0x56099fffeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f074a800640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f074a800640 1 -- 172.21.15.37:0/976961733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56099fd387d0 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f0749400640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f074a800640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x56099fffc3b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f074400edc0 tx=0x7f0744004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98135103580c9d0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f072b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f072b400640 1 -- 172.21.15.37:0/976961733 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440177c0 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f073c096810 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f072b400640 1 -- 172.21.15.37:0/976961733 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0744017960 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.013+0000 7f072b400640 1 -- 172.21.15.37:0/976961733 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0744032050 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 10 client.26512 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56099fc6e3f0 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f072b400640 1 -- 172.21.15.37:0/976961733 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f073c096810 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f072b400640 1 -- 172.21.15.37:0/976961733 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f074400b040 con 0x56099ffff460 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 10 client.26512 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 10 client.26512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 20 client.26512 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a000f450 0x5609a0011840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 4 client.26512 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56099ff67790 con 0x5609a000f450 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.014+0000 7f0749e00640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a000f450 0x5609a0011840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.015+0000 7f0749e00640 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a000f450 0x5609a0011840 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fce03a53d6c379f server_cookie=95bbd7fee7d651d8 in_seq=0 out_seq=0 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.015+0000 7f072b400640 10 client.26512 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f072b400640 1 -- 172.21.15.37:0/976961733 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56099ff67790 con 0x5609a000f450 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f072b400640 10 client.26512 handle_command_reply: tid=0 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 client.26512 shutdown 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 msgr2=0x56099fffc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x56099fffc3b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f074400edc0 tx=0x7f0744004ab0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 shutdown_connections 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x56099fffc3b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a000f450 0x5609a0011840 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 --2- 172.21.15.37:0/976961733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 >> 172.21.15.37:0/976961733 conn(0x56099ffbd200 msgr2=0x56099fff2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f072b400640 0 client.26512 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:51.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 shutdown_connections 2024-03-21T09:15:51.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.016+0000 7f07535ff740 1 -- 172.21.15.37:0/976961733 wait complete. 2024-03-21T09:15:51.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.017+0000 7f07535ff740 20 client.26512 trim_cache size 0 max 16384 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 Processor -- start 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 -- start start 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x5609a0000850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1b3090 con 0x56099fff60f0 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1b0b60 con 0x56099fff6dd0 2024-03-21T09:15:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.039+0000 7f07535ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1aec00 con 0x56099ffff460 2024-03-21T09:15:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33188/0 (socket says 172.21.15.37:33188) 2024-03-21T09:15:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 -- 172.21.15.37:0/457014245 learned_addr learned my addr 172.21.15.37:0/457014245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f074a800640 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x5609a0000850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749e00640 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 -- 172.21.15.37:0/457014245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 msgr2=0x5609a0000850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x5609a0000850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 -- 172.21.15.37:0/457014245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 msgr2=0x56099fffe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749400640 1 -- 172.21.15.37:0/457014245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56099fc2e0f0 con 0x56099fff6dd0 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.040+0000 7f0749e00640 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.041+0000 7f0749400640 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffeae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f073802e010 tx=0x7f07380056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a559e73acd527d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.041+0000 7f0736600640 1 -- 172.21.15.37:0/457014245 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738039040 con 0x56099fff6dd0 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.041+0000 7f0736600640 1 -- 172.21.15.37:0/457014245 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f073802c510 con 0x56099fff6dd0 2024-03-21T09:15:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.041+0000 7f0736600640 1 -- 172.21.15.37:0/457014245 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380328a0 con 0x56099fff6dd0 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 -- 172.21.15.37:0/457014245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 msgr2=0x56099fffeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffeae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f073802e010 tx=0x7f07380056d0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 -- 172.21.15.37:0/457014245 shutdown_connections 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099ffff460 0x5609a0000850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099fff6dd0 0x56099fffeae0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 --2- 172.21.15.37:0/457014245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff60f0 0x56099fffe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 -- 172.21.15.37:0/457014245 >> 172.21.15.37:0/457014245 conn(0x56099ffbd200 msgr2=0x56099fff0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 -- 172.21.15.37:0/457014245 shutdown_connections 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.042+0000 7f07535ff740 1 -- 172.21.15.37:0/457014245 wait complete. 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.043+0000 7f07535ff740 1 -- start start 2024-03-21T09:15:51.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.043+0000 7f07535ff740 1 Processor -- start 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.043+0000 7f07535ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x5609a0002ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f07535ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x5609a00033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f07535ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f07535ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1af200 con 0x56099fff6dd0 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f07535ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1b0500 con 0x56099ffff460 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f07535ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1af840 con 0x56099fff60f0 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f07535ff740 10 client.0 _ll_register_callbacks cb 0x7f0744009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x5609a0002ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x5609a00033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x5609a0002ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45698/0 (socket says 172.21.15.37:45698) 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 -- 172.21.15.37:0/611800124 learned_addr learned my addr 172.21.15.37:0/611800124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f074a800640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 -- 172.21.15.37:0/611800124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 msgr2=0x5609a0069cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0069cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 -- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 msgr2=0x5609a00033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x5609a00033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749e00640 1 -- 172.21.15.37:0/611800124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a0003ce0 con 0x56099fff60f0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.044+0000 7f0749400640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x5609a00033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f0749e00640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x5609a0002ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f073c00c6a0 tx=0x7f073c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82447d6eb3fe1e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f072b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f072b400640 1 -- 172.21.15.37:0/611800124 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c016330 con 0x56099fff60f0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f072b400640 1 -- 172.21.15.37:0/611800124 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f073c010430 con 0x56099fff60f0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f073802c510 con 0x56099fff60f0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f072b400640 1 -- 172.21.15.37:0/611800124 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f073c016510 con 0x56099fff60f0 2024-03-21T09:15:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 10 client.26518 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609a006b370 con 0x56099fff60f0 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f072b400640 1 -- 172.21.15.37:0/611800124 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f073802c510 con 0x56099fff60f0 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f072b400640 1 -- 172.21.15.37:0/611800124 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f073c098590 con 0x56099fff60f0 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 10 client.26518 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 20 client.26518 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.045+0000 7f07535ff740 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a006b5a0 0x5609a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.046+0000 7f07535ff740 4 client.26518 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.046+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56099ff61ae0 con 0x5609a006b5a0 2024-03-21T09:15:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.046+0000 7f0749400640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a006b5a0 0x5609a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.046+0000 7f0749400640 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a006b5a0 0x5609a006d990 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d367393ef1bc47 server_cookie=6c4906df2ba1d0c7 in_seq=0 out_seq=0 2024-03-21T09:15:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.046+0000 7f072b400640 10 client.26518 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f072b400640 1 -- 172.21.15.37:0/611800124 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56099ff61ae0 con 0x5609a006b5a0 2024-03-21T09:15:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f072b400640 10 client.26518 handle_command_reply: tid=0 2024-03-21T09:15:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 client.26518 shutdown 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 msgr2=0x5609a0002ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x5609a0002ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f073c00c6a0 tx=0x7f073c000bc0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 shutdown_connections 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5609a006b5a0 0x5609a006d990 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56099ffff460 0x5609a0069cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56099fff6dd0 0x5609a00033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f072b400640 0 client.26518 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 --2- 172.21.15.37:0/611800124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56099fff60f0 0x5609a0002ea0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 >> 172.21.15.37:0/611800124 conn(0x56099ffbd200 msgr2=0x56099fffa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 shutdown_connections 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 1 -- 172.21.15.37:0/611800124 wait complete. 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.047+0000 7f07535ff740 20 client.26518 trim_cache size 0 max 16384 2024-03-21T09:15:51.044 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:15:51.045 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:15:51.045 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:15:51.045 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:15:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07300a3090 msgr2=0x7f07300a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07300a3090 0x7f07300a5550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0744005fd0 tx=0x7f0744010040 comp rx=0 tx=0).stop 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 msgr2=0x7f074c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c199fa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f073c00e9f0 tx=0x7f073c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 shutdown_connections 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f074c109690 0x7f074c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07300a3090 0x7f07300a5550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f074c104040 0x7f074c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 --2- 172.21.15.37:0/2666680905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f074c103680 0x7f074c199fa0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 >> 172.21.15.37:0/2666680905 conn(0x7f074c0ff110 msgr2=0x7f074c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 shutdown_connections 2024-03-21T09:15:51.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.048+0000 7f074b200640 1 -- 172.21.15.37:0/2666680905 wait complete. 2024-03-21T09:15:51.060 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:15:51.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 -- 172.21.15.37:0/1466267126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2704073ad0 msgr2=0x7f2704073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 --2- 172.21.15.37:0/1466267126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2704073ad0 0x7f2704073eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009ca0 tx=0x7f26f402c180 comp rx=0 tx=0).stop 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 -- 172.21.15.37:0/1466267126 shutdown_connections 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 --2- 172.21.15.37:0/1466267126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f270410dd60 0x7f27041101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 --2- 172.21.15.37:0/1466267126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27040743f0 0x7f270410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 --2- 172.21.15.37:0/1466267126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2704073ad0 0x7f2704073eb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 -- 172.21.15.37:0/1466267126 >> 172.21.15.37:0/1466267126 conn(0x7f27040fd070 msgr2=0x7f27040ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 -- 172.21.15.37:0/1466267126 shutdown_connections 2024-03-21T09:15:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.281+0000 7f2708a00640 1 -- 172.21.15.37:0/1466267126 wait complete. 2024-03-21T09:15:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 Processor -- start 2024-03-21T09:15:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 -- start start 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 0x7f270419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27040743f0 0x7f270419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f270410dd60 0x7f270419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704113b00 con 0x7f27040743f0 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704113980 con 0x7f2704073ad0 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.283+0000 7f2708a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704113c80 con 0x7f270410dd60 2024-03-21T09:15:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 0x7f270419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 0x7f270419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33208/0 (socket says 172.21.15.37:33208) 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 -- 172.21.15.37:0/1260016439 learned_addr learned my addr 172.21.15.37:0/1260016439 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703e00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f270410dd60 0x7f270419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f26faa00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27040743f0 0x7f270419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 -- 172.21.15.37:0/1260016439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f270410dd60 msgr2=0x7f270419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f270410dd60 0x7f270419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 -- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27040743f0 msgr2=0x7f270419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27040743f0 0x7f270419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 -- 172.21.15.37:0/1260016439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270419f070 con 0x7f2704073ad0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.284+0000 7f2703400640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 0x7f270419a080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005e30 tx=0x7f26f4035d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.285+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4039040 con 0x7f2704073ad0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.285+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26f4044070 con 0x7f2704073ad0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.285+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40328d0 con 0x7f2704073ad0 2024-03-21T09:15:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.285+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270419f300 con 0x7f2704073ad0 2024-03-21T09:15:51.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.285+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041a6bf0 con 0x7f2704073ad0 2024-03-21T09:15:51.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.286+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270404f470 con 0x7f2704073ad0 2024-03-21T09:15:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.288+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f26f4032a70 con 0x7f2704073ad0 2024-03-21T09:15:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.289+0000 7f2700c00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26e80a3080 0x7f26e80a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.289+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f26f40f1f30 con 0x7f2704073ad0 2024-03-21T09:15:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.289+0000 7f26faa00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26e80a3080 0x7f26e80a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.290+0000 7f26faa00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26e80a3080 0x7f26e80a5540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27040ff790 tx=0x7f26ec005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.293+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26f40be730 con 0x7f2704073ad0 2024-03-21T09:15:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.542+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2704108ee0 con 0x7f2704073ad0 2024-03-21T09:15:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.543+0000 7f2700c00640 1 -- 172.21.15.37:0/1260016439 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11374 (secure 0 0 0) 0x7f26f40be030 con 0x7f2704073ad0 2024-03-21T09:15:51.537 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:15:51.538 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:15:46.490283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:15:51.538 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-21T09:15:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26e80a3080 msgr2=0x7f26e80a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26e80a3080 0x7f26e80a5540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27040ff790 tx=0x7f26ec005f90 comp rx=0 tx=0).stop 2024-03-21T09:15:51.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 msgr2=0x7f270419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 0x7f270419a080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005e30 tx=0x7f26f4035d70 comp rx=0 tx=0).stop 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 shutdown_connections 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f270410dd60 0x7f270419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27040743f0 0x7f270419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f26e80a3080 0x7f26e80a5540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 --2- 172.21.15.37:0/1260016439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2704073ad0 0x7f270419a080 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 >> 172.21.15.37:0/1260016439 conn(0x7f27040fd070 msgr2=0x7f27040fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 shutdown_connections 2024-03-21T09:15:51.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.546+0000 7f2708a00640 1 -- 172.21.15.37:0/1260016439 wait complete. 2024-03-21T09:15:51.563 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:15:51.564 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 -- 172.21.15.37:0/3672028194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81031e0 msgr2=0x7f50d810b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 --2- 172.21.15.37:0/3672028194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81031e0 0x7f50d810b6c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009900 tx=0x7f50c402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 -- 172.21.15.37:0/3672028194 shutdown_connections 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 --2- 172.21.15.37:0/3672028194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50d810bc90 0x7f50d810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 --2- 172.21.15.37:0/3672028194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81031e0 0x7f50d810b6c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 --2- 172.21.15.37:0/3672028194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d81028c0 0x7f50d8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 -- 172.21.15.37:0/3672028194 >> 172.21.15.37:0/3672028194 conn(0x7f50d80776c0 msgr2=0x7f50d8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 -- 172.21.15.37:0/3672028194 shutdown_connections 2024-03-21T09:15:51.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.786+0000 7f50dfa00640 1 -- 172.21.15.37:0/3672028194 wait complete. 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.787+0000 7f50dfa00640 1 Processor -- start 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.787+0000 7f50dfa00640 1 -- start start 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dfa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 0x7f50d819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dfa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50d81031e0 0x7f50d819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dfa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 0x7f50d819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dfa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8111a70 con 0x7f50d81031e0 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dfa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81118f0 con 0x7f50d81028c0 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dfa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8111bf0 con 0x7f50d810bc90 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 0x7f50d819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dd000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 0x7f50d819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 0x7f50d819ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45730/0 (socket says 172.21.15.37:45730) 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dd000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 0x7f50d819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33224/0 (socket says 172.21.15.37:33224) 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50dda00640 1 -- 172.21.15.37:0/2628026349 learned_addr learned my addr 172.21.15.37:0/2628026349 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.788+0000 7f50d7e00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50d81031e0 0x7f50d819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dda00640 1 -- 172.21.15.37:0/2628026349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 msgr2=0x7f50d819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dda00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 0x7f50d819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dda00640 1 -- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50d81031e0 msgr2=0x7f50d819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dda00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50d81031e0 0x7f50d819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dda00640 1 -- 172.21.15.37:0/2628026349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d819f140 con 0x7f50d810bc90 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dd000640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 0x7f50d819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dda00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 0x7f50d819ea10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f50d000eb40 tx=0x7f50d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50d5600640 1 -- 172.21.15.37:0/2628026349 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0004b20 con 0x7f50d810bc90 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50d5600640 1 -- 172.21.15.37:0/2628026349 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50d0004cc0 con 0x7f50d810bc90 2024-03-21T09:15:51.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50d5600640 1 -- 172.21.15.37:0/2628026349 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0010850 con 0x7f50d810bc90 2024-03-21T09:15:51.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d819f430 con 0x7f50d810bc90 2024-03-21T09:15:51.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.789+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d81a6cc0 con 0x7f50d810bc90 2024-03-21T09:15:51.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.792+0000 7f50d5600640 1 -- 172.21.15.37:0/2628026349 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f50d0005020 con 0x7f50d810bc90 2024-03-21T09:15:51.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.792+0000 7f50e277b740 1 Processor -- start 2024-03-21T09:15:51.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 -- start start 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991a6d0 0x556da991aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9923630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da9923c00 0x556da9926050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da987e2c0 con 0x556da991b080 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da9826c90 con 0x556da9923c00 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50e277b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da987a360 con 0x556da991a6d0 2024-03-21T09:15:51.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991a6d0 0x556da991aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991a6d0 0x556da991aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45738/0 (socket says 172.21.15.37:45738) 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50d7e00640 1 -- 172.21.15.37:0/2554489285 learned_addr learned my addr 172.21.15.37:0/2554489285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.793+0000 7f50dd000640 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9923630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50d7e00640 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da9923c00 0x556da9926050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50dd000640 1 -- 172.21.15.37:0/2554489285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991a6d0 msgr2=0x556da991aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50dd000640 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991a6d0 0x556da991aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50dd000640 1 -- 172.21.15.37:0/2554489285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da9923c00 msgr2=0x556da9926050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50dd000640 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da9923c00 0x556da9926050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50dd000640 1 -- 172.21.15.37:0/2554489285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556da984b080 con 0x556da991b080 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50d7e00640 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da9923c00 0x556da9926050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50d5600640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f50c00a3090 0x7f50c00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50d5600640 1 -- 172.21.15.37:0/2628026349 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f50d00d0340 con 0x7f50d810bc90 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50dd000640 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9923630 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009fd0 tx=0x7f50cc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5179f00ef3546d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.794+0000 7f50d7e00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f50c00a3090 0x7f50c00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.795+0000 7f50ca000640 1 -- 172.21.15.37:0/2554489285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc004030 con 0x556da991b080 2024-03-21T09:15:51.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.795+0000 7f50ca000640 1 -- 172.21.15.37:0/2554489285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50cc005500 con 0x556da991b080 2024-03-21T09:15:51.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.795+0000 7f50ca000640 1 -- 172.21.15.37:0/2554489285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc004e10 con 0x556da991b080 2024-03-21T09:15:51.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.795+0000 7f50e277b740 1 -- 172.21.15.37:0/2554489285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 msgr2=0x556da9923630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.795+0000 7f50d7e00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f50c00a3090 0x7f50c00a5550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c402bcd0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.795+0000 7f50e277b740 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9923630 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009fd0 tx=0x7f50cc0118d0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 -- 172.21.15.37:0/2554489285 shutdown_connections 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da9923c00 0x556da9926050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9923630 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 --2- 172.21.15.37:0/2554489285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991a6d0 0x556da991aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 -- 172.21.15.37:0/2554489285 >> 172.21.15.37:0/2554489285 conn(0x556da9914da0 msgr2=0x556da9915180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 -- 172.21.15.37:0/2554489285 shutdown_connections 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 -- 172.21.15.37:0/2554489285 wait complete. 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 -- start start 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.796+0000 7f50e277b740 1 Processor -- start 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9923500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991b080 0x556da9920890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da9923c00 0x556da9920e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da9926b70 con 0x556da9923c00 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da983a0d0 con 0x556da991a6d0 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da9926cf0 con 0x556da991b080 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50e277b740 10 client.0 _ll_register_callbacks cb 0x556da983ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50dd000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9923500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50dd000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9923500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33240/0 (socket says 172.21.15.37:33240) 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991b080 0x556da9920890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50dd000640 1 -- 172.21.15.37:0/1937256223 learned_addr learned my addr 172.21.15.37:0/1937256223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.797+0000 7f50dda00640 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da9923c00 0x556da9920e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50dd000640 1 -- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991b080 msgr2=0x556da9920890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50dd000640 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991b080 0x556da9920890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50dd000640 1 -- 172.21.15.37:0/1937256223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da9923c00 msgr2=0x556da9920e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50dd000640 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da9923c00 0x556da9920e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50dd000640 1 -- 172.21.15.37:0/1937256223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556da95923f0 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50dd000640 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9923500 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009350 tx=0x7f50cc004d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3db110e2f0d3cd32 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50bea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50bea00640 1 -- 172.21.15.37:0/1937256223 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc028020 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50bea00640 1 -- 172.21.15.37:0/1937256223 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f50cc016040 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.798+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50cc005500 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50bea00640 1 -- 172.21.15.37:0/1937256223 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50cc033050 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 10 client.26507 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556da965c7d0 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50bea00640 1 -- 172.21.15.37:0/1937256223 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50cc005500 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50bea00640 1 -- 172.21.15.37:0/1937256223 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f50cc003ad0 con 0x556da991a6d0 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 10 client.26507 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 10 client.26507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 20 client.26507 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da9933a30 0x556da9935e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 4 client.26507 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.799+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556da988b800 con 0x556da9933a30 2024-03-21T09:15:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.800+0000 7f50d7e00640 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da9933a30 0x556da9935e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.800+0000 7f50d7e00640 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da9933a30 0x556da9935e20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f05d2a4127982f0 server_cookie=7548e0b732109c18 in_seq=0 out_seq=0 2024-03-21T09:15:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.800+0000 7f50bea00640 10 client.26507 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50bea00640 1 -- 172.21.15.37:0/1937256223 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556da988b800 con 0x556da9933a30 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50bea00640 10 client.26507 handle_command_reply: tid=0 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 client.26507 shutdown 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 msgr2=0x556da9923500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9923500 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009350 tx=0x7f50cc004d40 comp rx=0 tx=0).stop 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 shutdown_connections 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da9933a30 0x556da9935e20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da9923c00 0x556da9920e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da991b080 0x556da9920890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 --2- 172.21.15.37:0/1937256223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9923500 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 >> 172.21.15.37:0/1937256223 conn(0x556da9914da0 msgr2=0x556da9916af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50bea00640 0 client.26507 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 shutdown_connections 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 1 -- 172.21.15.37:0/1937256223 wait complete. 2024-03-21T09:15:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.801+0000 7f50e277b740 20 client.26507 trim_cache size 0 max 16384 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 Processor -- start 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 -- start start 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9920040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9922fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9923500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b31c0 con 0x556da991b080 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b0c90 con 0x556da991a6d0 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.815+0000 7f50e277b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81aed30 con 0x556da9923c00 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9920040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9920040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33244/0 (socket says 172.21.15.37:33244) 2024-03-21T09:15:51.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 -- 172.21.15.37:0/2239905970 learned_addr learned my addr 172.21.15.37:0/2239905970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50d7e00640 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9922fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dda00640 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9923500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 -- 172.21.15.37:0/2239905970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 msgr2=0x556da9923500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9923500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 -- 172.21.15.37:0/2239905970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 msgr2=0x556da9922fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9922fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 -- 172.21.15.37:0/2239905970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556da9926a50 con 0x556da991a6d0 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50dd000640 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9920040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f50cc011dd0 tx=0x7f50cc009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d91638df1f9d79f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50c9600640 1 -- 172.21.15.37:0/2239905970 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc028070 con 0x556da991a6d0 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50c9600640 1 -- 172.21.15.37:0/2239905970 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50cc005140 con 0x556da991a6d0 2024-03-21T09:15:51.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.816+0000 7f50c9600640 1 -- 172.21.15.37:0/2239905970 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc005510 con 0x556da991a6d0 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 -- 172.21.15.37:0/2239905970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 msgr2=0x556da9920040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9920040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f50cc011dd0 tx=0x7f50cc009fa0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 -- 172.21.15.37:0/2239905970 shutdown_connections 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9923500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991b080 0x556da9922fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 --2- 172.21.15.37:0/2239905970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991a6d0 0x556da9920040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 -- 172.21.15.37:0/2239905970 >> 172.21.15.37:0/2239905970 conn(0x556da9914da0 msgr2=0x556da991bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 -- 172.21.15.37:0/2239905970 shutdown_connections 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.817+0000 7f50e277b740 1 -- 172.21.15.37:0/2239905970 wait complete. 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 -- start start 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 Processor -- start 2024-03-21T09:15:51.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991a6d0 0x556da992bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 0x556da9928f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9929470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81af330 con 0x556da991a6d0 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b0630 con 0x556da991b080 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81af970 con 0x556da9923c00 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.818+0000 7f50e277b740 10 client.0 _ll_register_callbacks cb 0x7f50cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 0x556da9928f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 0x556da9928f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33248/0 (socket says 172.21.15.37:33248) 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 -- 172.21.15.37:0/3992582651 learned_addr learned my addr 172.21.15.37:0/3992582651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50dd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991a6d0 0x556da992bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50dda00640 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9929470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 -- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 msgr2=0x556da9929470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9929470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 -- 172.21.15.37:0/3992582651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991a6d0 msgr2=0x556da992bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991a6d0 0x556da992bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 -- 172.21.15.37:0/3992582651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556da9929b10 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50d7e00640 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 0x556da9928f30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f50c402c2e0 tx=0x7f50c4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c2ac1d965a8486 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.819+0000 7f50bea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50bea00640 1 -- 172.21.15.37:0/3992582651 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c405b030 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50bea00640 1 -- 172.21.15.37:0/3992582651 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f50c403a040 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50cc005140 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50bea00640 1 -- 172.21.15.37:0/3992582651 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c4068050 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 10 client.26513 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556da998efd0 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50bea00640 1 -- 172.21.15.37:0/3992582651 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50cc005140 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50bea00640 1 -- 172.21.15.37:0/3992582651 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f50c40035f0 con 0x556da991b080 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 10 client.26513 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 10 client.26513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:15:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 20 client.26513 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da998f710 0x556da9991b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 4 client.26513 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.820+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556da9885b50 con 0x556da998f710 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.821+0000 7f50dd000640 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da998f710 0x556da9991b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.821+0000 7f50dd000640 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da998f710 0x556da9991b30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d23b38ff934ca07d server_cookie=91bd59de05baf535 in_seq=0 out_seq=0 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.821+0000 7f50bea00640 10 client.26513 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.821+0000 7f50bea00640 1 -- 172.21.15.37:0/3992582651 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556da9885b50 con 0x556da998f710 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.821+0000 7f50bea00640 10 client.26513 handle_command_reply: tid=0 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 client.26513 shutdown 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 msgr2=0x556da9928f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 0x556da9928f30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f50c402c2e0 tx=0x7f50c4005fd0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 shutdown_connections 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556da9923c00 0x556da9929470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556da991b080 0x556da9928f30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x556da998f710 0x556da9991b30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 --2- 172.21.15.37:0/3992582651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556da991a6d0 0x556da992bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 >> 172.21.15.37:0/3992582651 conn(0x556da9914da0 msgr2=0x556da993a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50bea00640 0 client.26513 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 shutdown_connections 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 1 -- 172.21.15.37:0/3992582651 wait complete. 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.822+0000 7f50e277b740 20 client.26513 trim_cache size 0 max 16384 2024-03-21T09:15:51.821 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f50c00a3090 msgr2=0x7f50c00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f50c00a3090 0x7f50c00a5550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 msgr2=0x7f50d819ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 0x7f50d819ea10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f50d000eb40 tx=0x7f50d000c2d0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 shutdown_connections 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d810bc90 0x7f50d819ea10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50d81031e0 0x7f50d819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f50c00a3090 0x7f50c00a5550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 --2- 172.21.15.37:0/2628026349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50d81028c0 0x7f50d819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:51.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 >> 172.21.15.37:0/2628026349 conn(0x7f50d80776c0 msgr2=0x7f50d8101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:51.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 shutdown_connections 2024-03-21T09:15:51.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:51.823+0000 7f50dfa00640 1 -- 172.21.15.37:0/2628026349 wait complete. 2024-03-21T09:15:51.838 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:15:51.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 -- 172.21.15.37:0/2316606607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848107630 msgr2=0x7f4848107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 --2- 172.21.15.37:0/2316606607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848107630 0x7f4848107a10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f483c009890 tx=0x7f483c02bb70 comp rx=0 tx=0).stop 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 -- 172.21.15.37:0/2316606607 shutdown_connections 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 --2- 172.21.15.37:0/2316606607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848101fe0 0x7f484810e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 --2- 172.21.15.37:0/2316606607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4848101620 0x7f4848101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 --2- 172.21.15.37:0/2316606607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848107630 0x7f4848107a10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 -- 172.21.15.37:0/2316606607 >> 172.21.15.37:0/2316606607 conn(0x7f48480fd110 msgr2=0x7f48480ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 -- 172.21.15.37:0/2316606607 shutdown_connections 2024-03-21T09:15:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.077+0000 7f484f800640 1 -- 172.21.15.37:0/2316606607 wait complete. 2024-03-21T09:15:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 Processor -- start 2024-03-21T09:15:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 -- start start 2024-03-21T09:15:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4848101620 0x7f4848197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848101fe0 0x7f48481984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 0x7f484819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848111a00 con 0x7f4848101620 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848111880 con 0x7f4848107630 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.079+0000 7f484f800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848111b80 con 0x7f4848101fe0 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 0x7f484819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 0x7f484819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33266/0 (socket says 172.21.15.37:33266) 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 -- 172.21.15.37:0/491273642 learned_addr learned my addr 172.21.15.37:0/491273642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f4847e00640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848101fe0 0x7f48481984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484ce00640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4848101620 0x7f4848197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 -- 172.21.15.37:0/491273642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848101fe0 msgr2=0x7f48481984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848101fe0 0x7f48481984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 -- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4848101620 msgr2=0x7f4848197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4848101620 0x7f4848197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 -- 172.21.15.37:0/491273642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484819cf70 con 0x7f4848107630 2024-03-21T09:15:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.080+0000 7f484d800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 0x7f484819c840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f484000eb10 tx=0x7f484000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.081+0000 7f4845600640 1 -- 172.21.15.37:0/491273642 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484000ce20 con 0x7f4848107630 2024-03-21T09:15:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.081+0000 7f4845600640 1 -- 172.21.15.37:0/491273642 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4840005020 con 0x7f4848107630 2024-03-21T09:15:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.081+0000 7f4845600640 1 -- 172.21.15.37:0/491273642 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840010690 con 0x7f4848107630 2024-03-21T09:15:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.081+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484819d260 con 0x7f4848107630 2024-03-21T09:15:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.081+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48481a4af0 con 0x7f4848107630 2024-03-21T09:15:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.084+0000 7f4845600640 1 -- 172.21.15.37:0/491273642 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4840005540 con 0x7f4848107630 2024-03-21T09:15:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.085+0000 7f4845600640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f48300a3110 0x7f48300a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.085+0000 7f4845600640 1 -- 172.21.15.37:0/491273642 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f48400d0060 con 0x7f4848107630 2024-03-21T09:15:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f484ce00640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f48300a3110 0x7f48300a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 Processor -- start 2024-03-21T09:15:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 -- start start 2024-03-21T09:15:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f64ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f70460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e16ec42c0 con 0x557e16f6e010 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e16e6cc90 con 0x557e16f65490 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.086+0000 7f485259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e16ec0360 con 0x557e16f64ae0 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f484ce00640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f48300a3110 0x7f48300a55d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f483c02c180 tx=0x7f483c0056d0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f484ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f64ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33272/0 (socket says 172.21.15.37:33272) 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 -- 172.21.15.37:0/4258333177 learned_addr learned my addr 172.21.15.37:0/4258333177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f484d800640 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f70460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 -- 172.21.15.37:0/4258333177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 msgr2=0x557e16f64ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f64ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 -- 172.21.15.37:0/4258333177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 msgr2=0x557e16f70460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f70460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 -- 172.21.15.37:0/4258333177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e16e91080 con 0x557e16f65490 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f484d800640 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f70460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f4847e00640 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6da40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4834009ca0 tx=0x7f4834011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17c442904d005cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.087+0000 7f483a000640 1 -- 172.21.15.37:0/4258333177 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834004230 con 0x557e16f65490 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f483a000640 1 -- 172.21.15.37:0/4258333177 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48340043d0 con 0x557e16f65490 2024-03-21T09:15:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f483a000640 1 -- 172.21.15.37:0/4258333177 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834019870 con 0x557e16f65490 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 -- 172.21.15.37:0/4258333177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 msgr2=0x557e16f6da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6da40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4834009ca0 tx=0x7f4834011810 comp rx=0 tx=0).stop 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 -- 172.21.15.37:0/4258333177 shutdown_connections 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f70460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6da40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 --2- 172.21.15.37:0/4258333177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f64ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 -- 172.21.15.37:0/4258333177 >> 172.21.15.37:0/4258333177 conn(0x557e16f5f1b0 msgr2=0x557e16f5f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 -- 172.21.15.37:0/4258333177 shutdown_connections 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.088+0000 7f485259a740 1 -- 172.21.15.37:0/4258333177 wait complete. 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.089+0000 7f485259a740 1 -- start start 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.089+0000 7f485259a740 1 Processor -- start 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e16f70f80 con 0x557e16f64ae0 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e16e800d0 con 0x557e16f65490 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e16f71100 con 0x557e16f6e010 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f485259a740 10 client.0 _ll_register_callbacks cb 0x557e16e84e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f484ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f484d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33284/0 (socket says 172.21.15.37:33284) 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 -- 172.21.15.37:0/2525530709 learned_addr learned my addr 172.21.15.37:0/2525530709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 -- 172.21.15.37:0/2525530709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 msgr2=0x557e16f6b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 -- 172.21.15.37:0/2525530709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 msgr2=0x557e16f6d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f6d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f4847e00640 1 -- 172.21.15.37:0/2525530709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e16bd83f0 con 0x557e16f65490 2024-03-21T09:15:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.090+0000 7f484d800640 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f4847e00640 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6ace0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4834006010 tx=0x7f4834005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695fb6326fe9c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f482ea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f482ea00640 1 -- 172.21.15.37:0/2525530709 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834028020 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f482ea00640 1 -- 172.21.15.37:0/2525530709 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f48340035f0 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48340043d0 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f482ea00640 1 -- 172.21.15.37:0/2525530709 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4834033050 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f485259a740 10 client.26525 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.091+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557e16ca27d0 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f482ea00640 1 -- 172.21.15.37:0/2525530709 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48340043d0 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f482ea00640 1 -- 172.21.15.37:0/2525530709 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4834018070 con 0x557e16f65490 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f485259a740 10 client.26525 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:52.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f485259a740 10 client.26525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f485259a740 20 client.26525 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f485259a740 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16f7dd70 0x557e16f80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f485259a740 4 client.26525 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e16ed1800 con 0x557e16f7dd70 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.092+0000 7f484ce00640 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16f7dd70 0x557e16f80160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.093+0000 7f484ce00640 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16f7dd70 0x557e16f80160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2530b5ac2d0c312a server_cookie=cc784730a735c7be in_seq=0 out_seq=0 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.093+0000 7f482ea00640 10 client.26525 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.093+0000 7f482ea00640 1 -- 172.21.15.37:0/2525530709 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557e16ed1800 con 0x557e16f7dd70 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.093+0000 7f482ea00640 10 client.26525 handle_command_reply: tid=0 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.093+0000 7f485259a740 1 client.26525 shutdown 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 msgr2=0x557e16f6ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6ace0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4834006010 tx=0x7f4834005270 comp rx=0 tx=0).stop 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 shutdown_connections 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16f7dd70 0x557e16f80160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6ace0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 --2- 172.21.15.37:0/2525530709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f6d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f482ea00640 0 client.26525 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:52.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 >> 172.21.15.37:0/2525530709 conn(0x557e16f5f1b0 msgr2=0x557e16f60f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 shutdown_connections 2024-03-21T09:15:52.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 1 -- 172.21.15.37:0/2525530709 wait complete. 2024-03-21T09:15:52.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.094+0000 7f485259a740 20 client.26525 trim_cache size 0 max 16384 2024-03-21T09:15:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 Processor -- start 2024-03-21T09:15:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 -- start start 2024-03-21T09:15:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f59d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481aee40 con 0x557e16f64ae0 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481ac910 con 0x557e16f65490 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f485259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481aa9b0 con 0x557e16f6e010 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f484ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f59d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f484d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33294/0 (socket says 172.21.15.37:33294) 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.108+0000 7f4847e00640 1 -- 172.21.15.37:0/2371157808 learned_addr learned my addr 172.21.15.37:0/2371157808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4847e00640 1 -- 172.21.15.37:0/2371157808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 msgr2=0x557e16f6d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4847e00640 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4847e00640 1 -- 172.21.15.37:0/2371157808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 msgr2=0x557e16f59d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4847e00640 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f59d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4847e00640 1 -- 172.21.15.37:0/2371157808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e16f6b8f0 con 0x557e16f65490 2024-03-21T09:15:52.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4847e00640 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6d410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f48340092a0 tx=0x7f4834001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50549fe686713e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4839600640 1 -- 172.21.15.37:0/2371157808 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834028040 con 0x557e16f65490 2024-03-21T09:15:52.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4839600640 1 -- 172.21.15.37:0/2371157808 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4834004c30 con 0x557e16f65490 2024-03-21T09:15:52.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.109+0000 7f4839600640 1 -- 172.21.15.37:0/2371157808 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483401f040 con 0x557e16f65490 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 -- 172.21.15.37:0/2371157808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 msgr2=0x557e16f6d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6d410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f48340092a0 tx=0x7f4834001e50 comp rx=0 tx=0).stop 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 -- 172.21.15.37:0/2371157808 shutdown_connections 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f6e010 0x557e16f6d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f6d410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 --2- 172.21.15.37:0/2371157808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f64ae0 0x557e16f59d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 -- 172.21.15.37:0/2371157808 >> 172.21.15.37:0/2371157808 conn(0x557e16f5f1b0 msgr2=0x557e16f59620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 -- 172.21.15.37:0/2371157808 shutdown_connections 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 -- 172.21.15.37:0/2371157808 wait complete. 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.110+0000 7f485259a740 1 -- start start 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 Processor -- start 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f73770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f73cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481aafb0 con 0x557e16f6e010 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481ac2b0 con 0x557e16f65490 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481ab5f0 con 0x557e16f64ae0 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.111+0000 7f485259a740 10 client.0 _ll_register_callbacks cb 0x7f4834005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f484ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f73770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f73cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f484ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f73770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45806/0 (socket says 172.21.15.37:45806) 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f484ce00640 1 -- 172.21.15.37:0/4199354466 learned_addr learned my addr 172.21.15.37:0/4199354466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f484d800640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 -- 172.21.15.37:0/4199354466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 msgr2=0x557e16f73770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f73770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 -- 172.21.15.37:0/4199354466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 msgr2=0x557e16f741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 -- 172.21.15.37:0/4199354466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e16fd8470 con 0x557e16f65490 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f484ce00640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f73770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f4847e00640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f73cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x557e16f711a0 tx=0x7f4834009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8555840acd4bded8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.112+0000 7f482ea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f482ea00640 1 -- 172.21.15.37:0/4199354466 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834004380 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4834004c30 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f482ea00640 1 -- 172.21.15.37:0/4199354466 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4834004520 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f482ea00640 1 -- 172.21.15.37:0/4199354466 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48340035f0 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 10 client.26531 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557e16fd96f0 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f482ea00640 1 -- 172.21.15.37:0/4199354466 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4834004c30 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f482ea00640 1 -- 172.21.15.37:0/4199354466 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4834018070 con 0x557e16f65490 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 10 client.26531 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 10 client.26531 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 20 client.26531 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16fd9ad0 0x557e16fdbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 4 client.26531 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.113+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e16ecbb50 con 0x557e16fd9ad0 2024-03-21T09:15:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f484ce00640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16fd9ad0 0x557e16fdbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f484ce00640 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16fd9ad0 0x557e16fdbef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88ebccf53e0cf5cf server_cookie=142838ae68a20445 in_seq=0 out_seq=0 2024-03-21T09:15:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f482ea00640 10 client.26531 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f482ea00640 1 -- 172.21.15.37:0/4199354466 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e16ecbb50 con 0x557e16fd9ad0 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f482ea00640 10 client.26531 handle_command_reply: tid=0 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 client.26531 shutdown 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 msgr2=0x557e16f73cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f73cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x557e16f711a0 tx=0x7f4834009910 comp rx=0 tx=0).stop 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 shutdown_connections 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x557e16fd9ad0 0x557e16fdbef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e16f6e010 0x557e16f741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e16f65490 0x557e16f73cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 --2- 172.21.15.37:0/4199354466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e16f64ae0 0x557e16f73770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f482ea00640 0 client.26531 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.114+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 >> 172.21.15.37:0/4199354466 conn(0x557e16f5f1b0 msgr2=0x557e16f59eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 shutdown_connections 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f485259a740 1 -- 172.21.15.37:0/4199354466 wait complete. 2024-03-21T09:15:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f485259a740 20 client.26531 trim_cache size 0 max 16384 2024-03-21T09:15:52.113 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:15:52.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f48300a3110 msgr2=0x7f48300a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f484f800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f48300a3110 0x7f48300a55d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f483c02c180 tx=0x7f483c0056d0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 msgr2=0x7f484819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.115+0000 7f484f800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 0x7f484819c840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f484000eb10 tx=0x7f484000ef10 comp rx=0 tx=0).stop 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 shutdown_connections 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f48300a3110 0x7f48300a55d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848107630 0x7f484819c840 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4848101fe0 0x7f48481984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 --2- 172.21.15.37:0/491273642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4848101620 0x7f4848197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 >> 172.21.15.37:0/491273642 conn(0x7f48480fd110 msgr2=0x7f48480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 shutdown_connections 2024-03-21T09:15:52.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.116+0000 7f484f800640 1 -- 172.21.15.37:0/491273642 wait complete. 2024-03-21T09:15:52.129 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:15:52.129 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:15:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 -- 172.21.15.37:0/3763593132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 msgr2=0x7fef58107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3763593132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef58107960 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009890 tx=0x7fef4c02bb70 comp rx=0 tx=0).stop 2024-03-21T09:15:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 -- 172.21.15.37:0/3763593132 shutdown_connections 2024-03-21T09:15:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3763593132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef58101e80 0x7fef5810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3763593132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef581014c0 0x7fef58101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3763593132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef58107960 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 -- 172.21.15.37:0/3763593132 >> 172.21.15.37:0/3763593132 conn(0x7fef580fcfb0 msgr2=0x7fef580ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 -- 172.21.15.37:0/3763593132 shutdown_connections 2024-03-21T09:15:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.345+0000 7fef5ee00640 1 -- 172.21.15.37:0/3763593132 wait complete. 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 Processor -- start 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 -- start start 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 0x7fef58199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef58101e80 0x7fef5819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef5819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58111840 con 0x7fef58101e80 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581116c0 con 0x7fef581014c0 2024-03-21T09:15:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.347+0000 7fef5ee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581119c0 con 0x7fef58107580 2024-03-21T09:15:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef58101e80 0x7fef5819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef5819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 0x7fef58199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef5819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45826/0 (socket says 172.21.15.37:45826) 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5d000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef58101e80 0x7fef5819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40478/0 (socket says 172.21.15.37:40478) 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5d000640 1 -- 172.21.15.37:0/3445988786 learned_addr learned my addr 172.21.15.37:0/3445988786 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 0x7fef58199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33318/0 (socket says 172.21.15.37:33318) 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 -- 172.21.15.37:0/3445988786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 msgr2=0x7fef5819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef5819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 -- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef58101e80 msgr2=0x7fef5819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef58101e80 0x7fef5819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 -- 172.21.15.37:0/3445988786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5819ef50 con 0x7fef581014c0 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5e400640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef5819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.348+0000 7fef5da00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 0x7fef58199f60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0057c0 tx=0x7fef4c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.349+0000 7fef56000640 1 -- 172.21.15.37:0/3445988786 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c041020 con 0x7fef581014c0 2024-03-21T09:15:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.349+0000 7fef56000640 1 -- 172.21.15.37:0/3445988786 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4c02c9f0 con 0x7fef581014c0 2024-03-21T09:15:52.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.349+0000 7fef56000640 1 -- 172.21.15.37:0/3445988786 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c039040 con 0x7fef581014c0 2024-03-21T09:15:52.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.349+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5819f1e0 con 0x7fef581014c0 2024-03-21T09:15:52.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.349+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef581a6ad0 con 0x7fef581014c0 2024-03-21T09:15:52.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.352+0000 7fef56000640 1 -- 172.21.15.37:0/3445988786 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fef4c044020 con 0x7fef581014c0 2024-03-21T09:15:52.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.353+0000 7fef63164740 1 Processor -- start 2024-03-21T09:15:52.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.353+0000 7fef63164740 1 -- start start 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef56000640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef400a3110 0x7fef400a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef63164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c3fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef63164740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef63164740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef63164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f471ba32c0 con 0x55f471c3f8b0 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef63164740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f471b4bc90 con 0x55f471c48de0 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef63164740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f471b9f360 con 0x55f471c40260 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef56000640 1 -- 172.21.15.37:0/3445988786 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fef4c0f11c0 con 0x7fef581014c0 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c48810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45838/0 (socket says 172.21.15.37:45838) 2024-03-21T09:15:52.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5d000640 1 -- 172.21.15.37:0/3701758344 learned_addr learned my addr 172.21.15.37:0/3701758344 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c3fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5e400640 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5d000640 1 -- 172.21.15.37:0/3701758344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 msgr2=0x55f471c4b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.354+0000 7fef5e400640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef400a3110 0x7fef400a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef5d000640 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef5d000640 1 -- 172.21.15.37:0/3701758344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 msgr2=0x55f471c3fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef5d000640 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c3fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef5d000640 1 -- 172.21.15.37:0/3701758344 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f471b70080 con 0x55f471c40260 2024-03-21T09:15:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef5d000640 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c48810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef44009d60 tx=0x7fef44011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a952ee1e61e866e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef4aa00640 1 -- 172.21.15.37:0/3701758344 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44004230 con 0x55f471c40260 2024-03-21T09:15:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef4aa00640 1 -- 172.21.15.37:0/3701758344 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef440043d0 con 0x55f471c40260 2024-03-21T09:15:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.355+0000 7fef4aa00640 1 -- 172.21.15.37:0/3701758344 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440035f0 con 0x55f471c40260 2024-03-21T09:15:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 -- 172.21.15.37:0/3701758344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 msgr2=0x55f471c48810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef5e400640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef400a3110 0x7fef400a55d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef50005fd0 tx=0x7fef500095c0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c48810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef44009d60 tx=0x7fef44011810 comp rx=0 tx=0).stop 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 -- 172.21.15.37:0/3701758344 shutdown_connections 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c48810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 --2- 172.21.15.37:0/3701758344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c3fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.356+0000 7fef63164740 1 -- 172.21.15.37:0/3701758344 >> 172.21.15.37:0/3701758344 conn(0x55f471c39f80 msgr2=0x55f471c3a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.357+0000 7fef63164740 1 -- 172.21.15.37:0/3701758344 shutdown_connections 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.357+0000 7fef63164740 1 -- 172.21.15.37:0/3701758344 wait complete. 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.357+0000 7fef63164740 1 -- start start 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.357+0000 7fef63164740 1 Processor -- start 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c48680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c48de0 0x55f471c45f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f471c4bd50 con 0x55f471c3f8b0 2024-03-21T09:15:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f471b5f0d0 con 0x55f471c40260 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f471c4bed0 con 0x55f471c48de0 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c48680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef63164740 10 client.0 _ll_register_callbacks cb 0x55f471b63e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c48680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40492/0 (socket says 172.21.15.37:40492) 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 -- 172.21.15.37:0/2017467855 learned_addr learned my addr 172.21.15.37:0/2017467855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c48de0 0x55f471c45f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 -- 172.21.15.37:0/2017467855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c48de0 msgr2=0x55f471c45f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c48de0 0x55f471c45f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 -- 172.21.15.37:0/2017467855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 msgr2=0x55f471c45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5da00640 1 -- 172.21.15.37:0/2017467855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4718b73f0 con 0x55f471c3f8b0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.358+0000 7fef5d000640 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef5da00640 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c48680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef4c02c180 tx=0x7fef4c0bcec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d732d22c3640f398 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef3f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef3f400640 1 -- 172.21.15.37:0/2017467855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0bd9c0 con 0x55f471c3f8b0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef3f400640 1 -- 172.21.15.37:0/2017467855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fef4c0bb5c0 con 0x55f471c3f8b0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef440043d0 con 0x55f471c3f8b0 2024-03-21T09:15:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef3f400640 1 -- 172.21.15.37:0/2017467855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef4c0bd2c0 con 0x55f471c3f8b0 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef63164740 10 client.16344 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.359+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4719817d0 con 0x55f471c3f8b0 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef3f400640 1 -- 172.21.15.37:0/2017467855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef440043d0 con 0x55f471c3f8b0 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef3f400640 1 -- 172.21.15.37:0/2017467855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fef4c03c030 con 0x55f471c3f8b0 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef63164740 10 client.16344 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef63164740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef63164740 20 client.16344 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef63164740 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471c58c00 0x55f471c5b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef63164740 4 client.16344 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f471bb0800 con 0x55f471c58c00 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.360+0000 7fef5d000640 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471c58c00 0x55f471c5b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.361+0000 7fef5d000640 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471c58c00 0x55f471c5b050 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb35189ec919f3f server_cookie=a78afeeb8ea2eb9f in_seq=0 out_seq=0 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.361+0000 7fef3f400640 10 client.16344 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef3f400640 1 -- 172.21.15.37:0/2017467855 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f471bb0800 con 0x55f471c58c00 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef3f400640 10 client.16344 handle_command_reply: tid=0 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 client.16344 shutdown 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 msgr2=0x55f471c48680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c48680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef4c02c180 tx=0x7fef4c0bcec0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 shutdown_connections 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c48de0 0x55f471c45f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471c58c00 0x55f471c5b050 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 --2- 172.21.15.37:0/2017467855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c48680 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 >> 172.21.15.37:0/2017467855 conn(0x55f471c39f80 msgr2=0x55f471c3bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef3f400640 0 client.16344 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 shutdown_connections 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.362+0000 7fef63164740 1 -- 172.21.15.37:0/2017467855 wait complete. 2024-03-21T09:15:52.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.363+0000 7fef63164740 20 client.16344 trim_cache size 0 max 16384 2024-03-21T09:15:52.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 Processor -- start 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 -- start start 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 0x55f471c38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c48140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c48de0 0x55f471c48680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581b2f40 con 0x55f471c48de0 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581b0a10 con 0x55f471c40260 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.377+0000 7fef63164740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581aeab0 con 0x55f471c3f8b0 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 0x55f471c38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c48140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 0x55f471c38fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45860/0 (socket says 172.21.15.37:45860) 2024-03-21T09:15:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5da00640 1 -- 172.21.15.37:0/2853107288 learned_addr learned my addr 172.21.15.37:0/2853107288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5e400640 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c48de0 0x55f471c48680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 -- 172.21.15.37:0/2853107288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 msgr2=0x55f471c38fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 0x55f471c38fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 -- 172.21.15.37:0/2853107288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c48de0 msgr2=0x55f471c48680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c48de0 0x55f471c48680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 -- 172.21.15.37:0/2853107288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f471c60b20 con 0x55f471c40260 2024-03-21T09:15:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5da00640 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 0x55f471c38fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.378+0000 7fef5d000640 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c48140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55f471c3aff0 tx=0x7fef44005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd2fcbe565453e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.379+0000 7fef4a000640 1 -- 172.21.15.37:0/2853107288 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44031070 con 0x55f471c40260 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.379+0000 7fef4a000640 1 -- 172.21.15.37:0/2853107288 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4401f040 con 0x55f471c40260 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.379+0000 7fef4a000640 1 -- 172.21.15.37:0/2853107288 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4401e040 con 0x55f471c40260 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 -- 172.21.15.37:0/2853107288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 msgr2=0x55f471c48140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c48140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55f471c3aff0 tx=0x7fef44005760 comp rx=0 tx=0).stop 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 -- 172.21.15.37:0/2853107288 shutdown_connections 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c48de0 0x55f471c48680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c40260 0x55f471c48140 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 --2- 172.21.15.37:0/2853107288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c3f8b0 0x55f471c38fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 -- 172.21.15.37:0/2853107288 >> 172.21.15.37:0/2853107288 conn(0x55f471c39f80 msgr2=0x55f471c388d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 -- 172.21.15.37:0/2853107288 shutdown_connections 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 -- 172.21.15.37:0/2853107288 wait complete. 2024-03-21T09:15:52.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 -- start start 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.380+0000 7fef63164740 1 Processor -- start 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c50d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c4f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581af0b0 con 0x55f471c3f8b0 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581b03b0 con 0x55f471c48de0 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581af6f0 con 0x55f471c40260 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef63164740 10 client.0 _ll_register_callbacks cb 0x7fef4c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef5da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c50d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef5e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef5d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c4f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef5e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33358/0 (socket says 172.21.15.37:33358) 2024-03-21T09:15:52.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.381+0000 7fef5e400640 1 -- 172.21.15.37:0/2828236413 learned_addr learned my addr 172.21.15.37:0/2828236413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5e400640 1 -- 172.21.15.37:0/2828236413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 msgr2=0x55f471c4f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5e400640 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c4f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5e400640 1 -- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 msgr2=0x55f471c50d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5e400640 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c50d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5e400640 1 -- 172.21.15.37:0/2828236413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f471c4fc90 con 0x55f471c48de0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5d000640 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c4f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef5e400640 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4f650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fef5000c740 tx=0x7fef50004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0c86dfa4a618c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef3f400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef3f400640 1 -- 172.21.15.37:0/2828236413 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50003c40 con 0x55f471c48de0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef3f400640 1 -- 172.21.15.37:0/2828236413 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fef50024040 con 0x55f471c48de0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef4401f040 con 0x55f471c48de0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef3f400640 1 -- 172.21.15.37:0/2828236413 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef50036050 con 0x55f471c48de0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.382+0000 7fef63164740 10 client.26549 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f471cb53e0 con 0x55f471c48de0 2024-03-21T09:15:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef3f400640 1 -- 172.21.15.37:0/2828236413 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4401f040 con 0x55f471c48de0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef3f400640 1 -- 172.21.15.37:0/2828236413 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fef5000b040 con 0x55f471c48de0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 10 client.26549 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 20 client.26549 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471cb5bc0 0x55f471cb7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 4 client.26549 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f471baab50 con 0x55f471cb5bc0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.383+0000 7fef5da00640 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471cb5bc0 0x55f471cb7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef5da00640 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471cb5bc0 0x55f471cb7fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0f989e759b9ad4 server_cookie=b3e82d29d03c51df in_seq=0 out_seq=0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef3f400640 10 client.26549 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef3f400640 1 -- 172.21.15.37:0/2828236413 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f471baab50 con 0x55f471cb5bc0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef3f400640 10 client.26549 handle_command_reply: tid=0 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 client.26549 shutdown 2024-03-21T09:15:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 msgr2=0x55f471c4f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4f650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fef5000c740 tx=0x7fef50004ab0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 shutdown_connections 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f471cb5bc0 0x55f471cb7fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f471c48de0 0x55f471c4f650 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f471c40260 0x55f471c4f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef3f400640 0 client.26549 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 --2- 172.21.15.37:0/2828236413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f471c3f8b0 0x55f471c50d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.384+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 >> 172.21.15.37:0/2828236413 conn(0x55f471c39f80 msgr2=0x55f471c601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 shutdown_connections 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef63164740 1 -- 172.21.15.37:0/2828236413 wait complete. 2024-03-21T09:15:52.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef63164740 20 client.26549 trim_cache size 0 max 16384 2024-03-21T09:15:52.386 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef400a3110 msgr2=0x7fef400a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef400a3110 0x7fef400a55d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef50005fd0 tx=0x7fef500095c0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 msgr2=0x7fef58199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.385+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 0x7fef58199f60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0057c0 tx=0x7fef4c004360 comp rx=0 tx=0).stop 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 shutdown_connections 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef58107580 0x7fef5819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef58101e80 0x7fef5819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fef400a3110 0x7fef400a55d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 --2- 172.21.15.37:0/3445988786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef581014c0 0x7fef58199f60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 >> 172.21.15.37:0/3445988786 conn(0x7fef580fcfb0 msgr2=0x7fef5810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 shutdown_connections 2024-03-21T09:15:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:52.386+0000 7fef5ee00640 1 -- 172.21.15.37:0/3445988786 wait complete. 2024-03-21T09:15:52.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:52 smithi037 ceph-mon[31830]: pgmap v113: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:52.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:52 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1260016439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:15:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:52 smithi071 ceph-mon[36597]: pgmap v113: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:52 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1260016439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:15:52.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:52 smithi053 ceph-mon[36169]: pgmap v113: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:52.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:52 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1260016439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:15:53.398 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:15:53.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:15:53.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:15:53.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 -- 172.21.15.37:0/1308089481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa0100d60 msgr2=0x7f6aa010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 --2- 172.21.15.37:0/1308089481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa0100d60 0x7f6aa010dab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009900 tx=0x7f6a8c02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 -- 172.21.15.37:0/1308089481 shutdown_connections 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 --2- 172.21.15.37:0/1308089481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010dff0 0x7f6aa01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 --2- 172.21.15.37:0/1308089481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa0100d60 0x7f6aa010dab0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 --2- 172.21.15.37:0/1308089481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aa0100440 0x7f6aa0100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 -- 172.21.15.37:0/1308089481 >> 172.21.15.37:0/1308089481 conn(0x7f6aa0077750 msgr2=0x7f6aa0077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 -- 172.21.15.37:0/1308089481 shutdown_connections 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.655+0000 7f6aa7400640 1 -- 172.21.15.37:0/1308089481 wait complete. 2024-03-21T09:15:53.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.656+0000 7f6aa7400640 1 Processor -- start 2024-03-21T09:15:53.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.656+0000 7f6aa7400640 1 -- start start 2024-03-21T09:15:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 0x7f6aa019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aa0100d60 0x7f6aa019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa010dff0 0x7f6aa019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa7400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0113ca0 con 0x7f6aa0100d60 2024-03-21T09:15:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa7400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0113b20 con 0x7f6aa0100440 2024-03-21T09:15:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa7400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0113e20 con 0x7f6aa010dff0 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 0x7f6aa019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa010dff0 0x7f6aa019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 0x7f6aa019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33382/0 (socket says 172.21.15.37:33382) 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6aa4a00640 1 -- 172.21.15.37:0/418513596 learned_addr learned my addr 172.21.15.37:0/418513596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.657+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aa0100d60 0x7f6aa019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa4a00640 1 -- 172.21.15.37:0/418513596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa010dff0 msgr2=0x7f6aa019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa010dff0 0x7f6aa019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa4a00640 1 -- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aa0100d60 msgr2=0x7f6aa019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aa0100d60 0x7f6aa019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa4a00640 1 -- 172.21.15.37:0/418513596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa019f220 con 0x7f6aa0100440 2024-03-21T09:15:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa5400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa010dff0 0x7f6aa019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:53.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.658+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 0x7f6aa019a280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d950 tx=0x7f6a9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.659+0000 7f6a9d600640 1 -- 172.21.15.37:0/418513596 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94017070 con 0x7f6aa0100440 2024-03-21T09:15:53.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.659+0000 7f6a9d600640 1 -- 172.21.15.37:0/418513596 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a94009c30 con 0x7f6aa0100440 2024-03-21T09:15:53.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.659+0000 7f6a9d600640 1 -- 172.21.15.37:0/418513596 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400f6b0 con 0x7f6aa0100440 2024-03-21T09:15:53.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.659+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa019f510 con 0x7f6aa0100440 2024-03-21T09:15:53.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.659+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01a6da0 con 0x7f6aa0100440 2024-03-21T09:15:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.662+0000 7f6aaa1ba740 1 Processor -- start 2024-03-21T09:15:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.662+0000 7f6aaa1ba740 1 -- start start 2024-03-21T09:15:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d09a0 0x557dce9d9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d9580 0x557dce9db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dce934eb0 con 0x557dce9d0080 2024-03-21T09:15:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dce8dda60 con 0x557dce9d9580 2024-03-21T09:15:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dce930f50 con 0x557dce9d09a0 2024-03-21T09:15:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aa5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d9580 0x557dce9db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aa5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40532/0 (socket says 172.21.15.37:40532) 2024-03-21T09:15:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aa5400640 1 -- 172.21.15.37:0/3055387450 learned_addr learned my addr 172.21.15.37:0/3055387450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9fe00640 1 -- 172.21.15.37:0/3055387450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d09a0 msgr2=0x557dce9d9040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d09a0 0x557dce9d9040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9fe00640 1 -- 172.21.15.37:0/3055387450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 msgr2=0x557dce9d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9fe00640 1 -- 172.21.15.37:0/3055387450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dce7117e0 con 0x557dce9d9580 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6aa5400640 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.663+0000 7f6a9d600640 1 -- 172.21.15.37:0/418513596 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6a940109d0 con 0x7f6aa0100440 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.664+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d9580 0x557dce9db970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02c200 tx=0x7f6a8c02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53a0f25788b4803 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.664+0000 7f6a91c00640 1 -- 172.21.15.37:0/3055387450 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c043070 con 0x557dce9d9580 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.664+0000 7f6a91c00640 1 -- 172.21.15.37:0/3055387450 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c02c910 con 0x557dce9d9580 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.664+0000 7f6a91c00640 1 -- 172.21.15.37:0/3055387450 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004c70 con 0x557dce9d9580 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/3055387450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d9580 msgr2=0x557dce9db970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d9580 0x557dce9db970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02c200 tx=0x7f6a8c02c660 comp rx=0 tx=0).stop 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/3055387450 shutdown_connections 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d9580 0x557dce9db970 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d09a0 0x557dce9d9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/3055387450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/3055387450 >> 172.21.15.37:0/3055387450 conn(0x557dce9ca7b0 msgr2=0x557dce9cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6a9d600640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6a880a3110 0x7f6a880a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/3055387450 shutdown_connections 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6a9d600640 1 -- 172.21.15.37:0/418513596 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6a940cf730 con 0x7f6aa0100440 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/3055387450 wait complete. 2024-03-21T09:15:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.665+0000 7f6aa5400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6a880a3110 0x7f6a880a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 -- start start 2024-03-21T09:15:53.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 Processor -- start 2024-03-21T09:15:53.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aa5400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6a880a3110 0x7f6a880a55d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a980117d0 tx=0x7f6a98009550 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dce9dc310 con 0x557dce9d9580 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dce8f0eb0 con 0x557dce9d09a0 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.666+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dce9dc490 con 0x557dce9d0080 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6aaa1ba740 10 client.0 _ll_register_callbacks cb 0x557dce8e5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45886/0 (socket says 172.21.15.37:45886) 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33394/0 (socket says 172.21.15.37:33394) 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6aa4a00640 1 -- 172.21.15.37:0/593247582 learned_addr learned my addr 172.21.15.37:0/593247582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6aa5400640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 -- 172.21.15.37:0/593247582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 msgr2=0x557dce9d9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 -- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 msgr2=0x557dce9d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 -- 172.21.15.37:0/593247582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dce771f70 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d63c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c005ec0 tx=0x7f6a8c005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6bbbd896477b55 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.667+0000 7f6a86000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6a86000640 1 -- 172.21.15.37:0/593247582 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c043070 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a8c02c910 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6a86000640 1 -- 172.21.15.37:0/593247582 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6a8c004300 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6a86000640 1 -- 172.21.15.37:0/593247582 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c056050 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6aaa1ba740 10 client.26561 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557dce609170 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.668+0000 7f6a86000640 1 -- 172.21.15.37:0/593247582 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c02c910 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6a86000640 1 -- 172.21.15.37:0/593247582 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f6a8c0038f0 con 0x557dce9d09a0 2024-03-21T09:15:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aaa1ba740 10 client.26561 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aaa1ba740 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aaa1ba740 20 client.26561 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dce9e9540 0x557dce9eb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aaa1ba740 4 client.26561 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dce9423f0 con 0x557dce9e9540 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.669+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dce9e9540 0x557dce9eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.670+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dce9e9540 0x557dce9eb930 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62700233c582aa9a server_cookie=51da0c283d4197c4 in_seq=0 out_seq=0 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.670+0000 7f6a86000640 10 client.26561 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.670+0000 7f6a86000640 1 -- 172.21.15.37:0/593247582 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557dce9423f0 con 0x557dce9e9540 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.670+0000 7f6a86000640 10 client.26561 handle_command_reply: tid=0 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.670+0000 7f6aaa1ba740 1 client.26561 shutdown 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 msgr2=0x557dce9d63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d63c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c005ec0 tx=0x7f6a8c005900 comp rx=0 tx=0).stop 2024-03-21T09:15:53.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 shutdown_connections 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dce9e9540 0x557dce9eb930 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d63c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/593247582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6a86000640 0 client.26561 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 >> 172.21.15.37:0/593247582 conn(0x557dce9ca7b0 msgr2=0x557dce9cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 shutdown_connections 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/593247582 wait complete. 2024-03-21T09:15:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.671+0000 7f6aaa1ba740 20 client.26561 trim_cache size 0 max 16384 2024-03-21T09:15:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 Processor -- start 2024-03-21T09:15:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 -- start start 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01b3160 con 0x557dce9d9580 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01b0c30 con 0x557dce9d09a0 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.692+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01aecd0 con 0x557dce9d0080 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45902/0 (socket says 172.21.15.37:45902) 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 -- 172.21.15.37:0/1502708429 learned_addr learned my addr 172.21.15.37:0/1502708429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa5400640 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 -- 172.21.15.37:0/1502708429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 msgr2=0x557dce9d9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 -- 172.21.15.37:0/1502708429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 msgr2=0x557dce9cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 -- 172.21.15.37:0/1502708429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dce9d4950 con 0x557dce9d0080 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa5400640 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d8af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6a94097c30 tx=0x7f6a94086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14357c2bb600bf1a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.693+0000 7f6a91200640 1 -- 172.21.15.37:0/1502708429 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a940b3070 con 0x557dce9d0080 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6a91200640 1 -- 172.21.15.37:0/1502708429 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a940a4130 con 0x557dce9d0080 2024-03-21T09:15:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6a91200640 1 -- 172.21.15.37:0/1502708429 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94020040 con 0x557dce9d0080 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1502708429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 msgr2=0x557dce9d8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d8af0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6a94097c30 tx=0x7f6a94086040 comp rx=0 tx=0).stop 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1502708429 shutdown_connections 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d9580 0x557dce9cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9d9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1502708429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d0080 0x557dce9d8af0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1502708429 >> 172.21.15.37:0/1502708429 conn(0x557dce9ca7b0 msgr2=0x557dce9cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1502708429 shutdown_connections 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.694+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1502708429 wait complete. 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 -- start start 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 Processor -- start 2024-03-21T09:15:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d9580 0x557dce9df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01af2d0 con 0x557dce9d0080 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01b05d0 con 0x557dce9d09a0 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01af910 con 0x557dce9d9580 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.695+0000 7f6aaa1ba740 10 client.0 _ll_register_callbacks cb 0x7f6a98009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6aa4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6aa5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d9580 0x557dce9df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:33418/0 (socket says 172.21.15.37:33418) 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 -- 172.21.15.37:0/1623517234 learned_addr learned my addr 172.21.15.37:0/1623517234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 -- 172.21.15.37:0/1623517234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d9580 msgr2=0x557dce9df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d9580 0x557dce9df130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 -- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 msgr2=0x557dce9de6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9de6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 -- 172.21.15.37:0/1623517234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dce9d4440 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a9fe00640 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9debf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009900 tx=0x7f6a8c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d6cacc61dea75d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a86000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.696+0000 7f6a86000640 1 -- 172.21.15.37:0/1623517234 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c043070 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a940a4130 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6a86000640 1 -- 172.21.15.37:0/1623517234 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6a8c0037f0 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6a86000640 1 -- 172.21.15.37:0/1623517234 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c068050 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 10 client.26567 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557dcea45b20 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6a86000640 1 -- 172.21.15.37:0/1623517234 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a940a4130 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6a86000640 1 -- 172.21.15.37:0/1623517234 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f6a8c0051a0 con 0x557dce9d09a0 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 10 client.26567 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 10 client.26567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 20 client.26567 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dcea46240 0x557dcea48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 4 client.26567 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:15:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.697+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557dce93c740 con 0x557dcea46240 2024-03-21T09:15:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.698+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dcea46240 0x557dcea48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.698+0000 7f6aa4a00640 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dcea46240 0x557dcea48660 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edea8c7acbb352bc server_cookie=423b5b6984f0b388 in_seq=0 out_seq=0 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.698+0000 7f6a86000640 10 client.26567 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.699+0000 7f6a86000640 1 -- 172.21.15.37:0/1623517234 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557dce93c740 con 0x557dcea46240 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.699+0000 7f6a86000640 10 client.26567 handle_command_reply: tid=0 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 client.26567 shutdown 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 msgr2=0x557dce9debf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9debf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009900 tx=0x7f6a8c005a20 comp rx=0 tx=0).stop 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 shutdown_connections 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557dce9d9580 0x557dce9df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557dcea46240 0x557dcea48660 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dce9d09a0 0x557dce9debf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 --2- 172.21.15.37:0/1623517234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dce9d0080 0x557dce9de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 >> 172.21.15.37:0/1623517234 conn(0x557dce9ca7b0 msgr2=0x557dce9cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6a86000640 0 client.26567 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 shutdown_connections 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.700+0000 7f6aaa1ba740 1 -- 172.21.15.37:0/1623517234 wait complete. 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.701+0000 7f6aaa1ba740 20 client.26567 trim_cache size 0 max 16384 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "710059c1-974c-41bd-90c2-dc9be99b79f8", 2024-03-21T09:15:53.697 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:15:53.698 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:15:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6a880a3110 msgr2=0x7f6a880a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6a880a3110 0x7f6a880a55d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a980117d0 tx=0x7f6a98009550 comp rx=0 tx=0).stop 2024-03-21T09:15:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 msgr2=0x7f6aa019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 0x7f6aa019a280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d950 tx=0x7f6a9400de10 comp rx=0 tx=0).stop 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 shutdown_connections 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa010dff0 0x7f6aa019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6aa0100d60 0x7f6aa019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6a880a3110 0x7f6a880a55d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 --2- 172.21.15.37:0/418513596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0100440 0x7f6aa019a280 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 >> 172.21.15.37:0/418513596 conn(0x7f6aa0077750 msgr2=0x7f6aa010f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 shutdown_connections 2024-03-21T09:15:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.702+0000 7f6aa7400640 1 -- 172.21.15.37:0/418513596 wait complete. 2024-03-21T09:15:53.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:15:53.714 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 -- 172.21.15.37:0/4129108382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4104040 msgr2=0x7f3df4110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 --2- 172.21.15.37:0/4129108382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4104040 0x7f3df4110550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009c10 tx=0x7f3dec02bef0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 -- 172.21.15.37:0/4129108382 shutdown_connections 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 --2- 172.21.15.37:0/4129108382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4104040 0x7f3df4110550 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 --2- 172.21.15.37:0/4129108382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4103680 0x7f3df4103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 --2- 172.21.15.37:0/4129108382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3df4109690 0x7f3df4109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 -- 172.21.15.37:0/4129108382 >> 172.21.15.37:0/4129108382 conn(0x7f3df40ff110 msgr2=0x7f3df4101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 -- 172.21.15.37:0/4129108382 shutdown_connections 2024-03-21T09:15:53.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.925+0000 7f3df8a00640 1 -- 172.21.15.37:0/4129108382 wait complete. 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.926+0000 7f3df8a00640 1 Processor -- start 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 -- start start 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4103680 0x7f3df4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 0x7f3df419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3df4109690 0x7f3df419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4113af0 con 0x7f3df4103680 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4113970 con 0x7f3df4104040 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4113c70 con 0x7f3df4109690 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4103680 0x7f3df4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 0x7f3df419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3df4109690 0x7f3df419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 0x7f3df419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44782/0 (socket says 172.21.15.37:44782) 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.927+0000 7f3df2a00640 1 -- 172.21.15.37:0/2774674452 learned_addr learned my addr 172.21.15.37:0/2774674452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df2a00640 1 -- 172.21.15.37:0/2774674452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3df4109690 msgr2=0x7f3df419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df2a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3df4109690 0x7f3df419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df2a00640 1 -- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4103680 msgr2=0x7f3df4199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df2a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4103680 0x7f3df4199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df2a00640 1 -- 172.21.15.37:0/2774674452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df419ef40 con 0x7f3df4104040 2024-03-21T09:15:53.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df2a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 0x7f3df419a490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e9c0 tx=0x7f3de000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3debe00640 1 -- 172.21.15.37:0/2774674452 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de000cde0 con 0x7f3df4104040 2024-03-21T09:15:53.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3debe00640 1 -- 172.21.15.37:0/2774674452 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de00035f0 con 0x7f3df4104040 2024-03-21T09:15:53.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3debe00640 1 -- 172.21.15.37:0/2774674452 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0010690 con 0x7f3df4104040 2024-03-21T09:15:53.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df419f230 con 0x7f3df4104040 2024-03-21T09:15:53.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.928+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a6ac0 con 0x7f3df4104040 2024-03-21T09:15:53.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 Processor -- start 2024-03-21T09:15:53.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 -- start start 2024-03-21T09:15:53.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b56863b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b568640f0 0x558b56864570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3debe00640 1 -- 172.21.15.37:0/2774674452 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3de00037a0 con 0x7f3df4104040 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b5686d460 0x558b5686f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b567c8250 con 0x558b568640f0 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56770a30 con 0x558b56864dd0 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.932+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b567c42f0 con 0x558b5686d460 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b56863b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b56863b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44788/0 (socket says 172.21.15.37:44788) 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 -- 172.21.15.37:0/4178924491 learned_addr learned my addr 172.21.15.37:0/4178924491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df2a00640 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b5686d460 0x558b5686f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3400640 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b568640f0 0x558b56864570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 -- 172.21.15.37:0/4178924491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b5686d460 msgr2=0x558b5686f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b5686d460 0x558b5686f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 -- 172.21.15.37:0/4178924491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b568640f0 msgr2=0x558b56864570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b568640f0 0x558b56864570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 -- 172.21.15.37:0/4178924491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5675b690 con 0x558b56864dd0 2024-03-21T09:15:53.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.933+0000 7f3df3e00640 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b56863b20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009be0 tx=0x7f3dec009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c647fef8284d818 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3ddfe00640 1 -- 172.21.15.37:0/4178924491 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec043030 con 0x558b56864dd0 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3ddfe00640 1 -- 172.21.15.37:0/4178924491 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dec033c00 con 0x558b56864dd0 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3ddfe00640 1 -- 172.21.15.37:0/4178924491 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec02c890 con 0x558b56864dd0 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3debe00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3dd80a3090 0x7f3dd80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3debe00640 1 -- 172.21.15.37:0/2774674452 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3de00cff10 con 0x7f3df4104040 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/4178924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 msgr2=0x558b56863b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b56863b20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009be0 tx=0x7f3dec009840 comp rx=0 tx=0).stop 2024-03-21T09:15:53.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3df3400640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3dd80a3090 0x7f3dd80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/4178924491 shutdown_connections 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b5686d460 0x558b5686f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b568640f0 0x558b56864570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/4178924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b56863b20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.934+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/4178924491 >> 172.21.15.37:0/4178924491 conn(0x558b5682b200 msgr2=0x558b5685eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.935+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/4178924491 shutdown_connections 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.935+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/4178924491 wait complete. 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.935+0000 7f3df3400640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3dd80a3090 0x7f3dd80a5550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x558b56865320 tx=0x7f3de4005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.935+0000 7f3dfcdc3740 1 -- start start 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.935+0000 7f3dfcdc3740 1 Processor -- start 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b568640f0 0x558b5686cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b56864dd0 0x558b56869ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56870270 con 0x558b5686d460 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56783e80 con 0x558b568640f0 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b568703f0 con 0x558b56864dd0 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3dfcdc3740 10 client.0 _ll_register_callbacks cb 0x558b56796130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:53.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b568640f0 0x558b5686cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b56864dd0 0x558b56869ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b568640f0 0x558b5686cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44792/0 (socket says 172.21.15.37:44792) 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df3400640 1 -- 172.21.15.37:0/2494373088 learned_addr learned my addr 172.21.15.37:0/2494373088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df3e00640 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df2a00640 1 -- 172.21.15.37:0/2494373088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b568640f0 msgr2=0x558b5686cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df2a00640 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b568640f0 0x558b5686cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df2a00640 1 -- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 msgr2=0x558b5686a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.936+0000 7f3df2a00640 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3df2a00640 1 -- 172.21.15.37:0/2494373088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b565a67d0 con 0x558b56864dd0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3df2a00640 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b56864dd0 0x558b56869ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3de00cd260 tx=0x7f3de0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25241c9637488f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3ddcc00640 1 -- 172.21.15.37:0/2494373088 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de009b860 con 0x558b56864dd0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3ddcc00640 1 -- 172.21.15.37:0/2494373088 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3de00cd4e0 con 0x558b56864dd0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dec033c00 con 0x558b56864dd0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3ddcc00640 1 -- 172.21.15.37:0/2494373088 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de00cd6e0 con 0x558b56864dd0 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3dfcdc3740 10 client.26605 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:53.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.937+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b564dc3f0 con 0x558b56864dd0 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3ddcc00640 1 -- 172.21.15.37:0/2494373088 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dec033c00 con 0x558b56864dd0 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3ddcc00640 1 -- 172.21.15.37:0/2494373088 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f3de00d3020 con 0x558b56864dd0 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3dfcdc3740 10 client.26605 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3dfcdc3740 10 client.26605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3dfcdc3740 20 client.26605 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b5687d370 0x558b5687f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3dfcdc3740 4 client.26605 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:15:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b567d5790 con 0x558b5687d370 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.938+0000 7f3df3400640 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b5687d370 0x558b5687f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.939+0000 7f3df3400640 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b5687d370 0x558b5687f760 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872651202c99296c server_cookie=15b07557910e09c in_seq=0 out_seq=0 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.939+0000 7f3ddcc00640 10 client.26605 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3ddcc00640 1 -- 172.21.15.37:0/2494373088 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558b567d5790 con 0x558b5687d370 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3ddcc00640 10 client.26605 handle_command_reply: tid=0 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 client.26605 shutdown 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b56864dd0 msgr2=0x558b56869ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b56864dd0 0x558b56869ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3de00cd260 tx=0x7f3de0004730 comp rx=0 tx=0).stop 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 shutdown_connections 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b56864dd0 0x558b56869ee0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b5687d370 0x558b5687f760 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/2494373088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b568640f0 0x558b5686cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 >> 172.21.15.37:0/2494373088 conn(0x558b5682b200 msgr2=0x558b56860890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.940+0000 7f3ddcc00640 0 client.26605 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.941+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 shutdown_connections 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.941+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/2494373088 wait complete. 2024-03-21T09:15:53.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.941+0000 7f3dfcdc3740 20 client.26605 trim_cache size 0 max 16384 2024-03-21T09:15:53.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.958+0000 7f3dfcdc3740 1 Processor -- start 2024-03-21T09:15:53.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 -- start start 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b5686c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b5686cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41b2f30 con 0x558b5686d460 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41b0a00 con 0x558b56864dd0 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41aeaa0 con 0x558b568640f0 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3df3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b5686c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3df3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3df3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b5686c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58618/0 (socket says 172.21.15.37:58618) 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3df3400640 1 -- 172.21.15.37:0/1595955761 learned_addr learned my addr 172.21.15.37:0/1595955761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.959+0000 7f3df2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b5686cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3df3400640 1 -- 172.21.15.37:0/1595955761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 msgr2=0x558b5686cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3df3400640 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b5686cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3df3400640 1 -- 172.21.15.37:0/1595955761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 msgr2=0x558b5686e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3df3400640 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3df3400640 1 -- 172.21.15.37:0/1595955761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5687fad0 con 0x558b568640f0 2024-03-21T09:15:53.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3df3400640 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b5686c610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x558b5687ff30 tx=0x7f3de401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d743d477ef5ad08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3ddf400640 1 -- 172.21.15.37:0/1595955761 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4004400 con 0x558b568640f0 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3ddf400640 1 -- 172.21.15.37:0/1595955761 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de4030070 con 0x558b568640f0 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.960+0000 7f3ddf400640 1 -- 172.21.15.37:0/1595955761 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de401f8c0 con 0x558b568640f0 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/1595955761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 msgr2=0x558b5686c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b5686c610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x558b5687ff30 tx=0x7f3de401b000 comp rx=0 tx=0).stop 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/1595955761 shutdown_connections 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b5686d460 0x558b5686e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b56864dd0 0x558b5686cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/1595955761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b5686c610 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/1595955761 >> 172.21.15.37:0/1595955761 conn(0x558b5682b200 msgr2=0x558b568687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/1595955761 shutdown_connections 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/1595955761 wait complete. 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.961+0000 7f3dfcdc3740 1 -- start start 2024-03-21T09:15:53.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 Processor -- start 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b568669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b56864dd0 0x558b56866ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b5686d460 0x558b568d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41af0a0 con 0x558b56864dd0 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41b03a0 con 0x558b5686d460 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41af6e0 con 0x558b568640f0 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3dfcdc3740 10 client.0 _ll_register_callbacks cb 0x7f3de00d36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3df3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b568669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3df3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b568669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58630/0 (socket says 172.21.15.37:58630) 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3df3400640 1 -- 172.21.15.37:0/523597583 learned_addr learned my addr 172.21.15.37:0/523597583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3df2a00640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b56864dd0 0x558b56866ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.962+0000 7f3df3e00640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b5686d460 0x558b568d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3df3400640 1 -- 172.21.15.37:0/523597583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b5686d460 msgr2=0x558b568d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3df3400640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b5686d460 0x558b568d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3df3400640 1 -- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b56864dd0 msgr2=0x558b56866ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3df3400640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b56864dd0 0x558b56866ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3df3400640 1 -- 172.21.15.37:0/523597583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b5649c0f0 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3df3400640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b568669b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3de4012bf0 tx=0x7f3de4005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5421befc167d85bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3ddcc00640 1 -- 172.21.15.37:0/523597583 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4039070 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3ddcc00640 1 -- 172.21.15.37:0/523597583 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3de4005040 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de4030070 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.963+0000 7f3ddcc00640 1 -- 172.21.15.37:0/523597583 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de4045050 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 10 client.26611 fetch_fsmap learned FSMap version 28 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b568d90d0 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3ddcc00640 1 -- 172.21.15.37:0/523597583 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de4030070 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3ddcc00640 1 -- 172.21.15.37:0/523597583 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f3de402b680 con 0x558b568640f0 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 10 client.26611 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:15:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 20 client.26611 populate_metadata read hostname 'smithi037' 2024-03-21T09:15:53.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b568d97d0 0x558b568dbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:15:53.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 4 client.26611 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:15:53.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b567cfae0 con 0x558b568d97d0 2024-03-21T09:15:53.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.964+0000 7f3df2a00640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b568d97d0 0x558b568dbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3df2a00640 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b568d97d0 0x558b568dbbf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5660ea48902267e server_cookie=9e3b60ed9fc63ca6 in_seq=0 out_seq=0 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3ddcc00640 10 client.26611 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3ddcc00640 1 -- 172.21.15.37:0/523597583 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558b567cfae0 con 0x558b568d97d0 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3ddcc00640 10 client.26611 handle_command_reply: tid=0 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 client.26611 shutdown 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 msgr2=0x558b568669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b568669b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3de4012bf0 tx=0x7f3de4005db0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 shutdown_connections 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558b568d97d0 0x558b568dbbf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b5686d460 0x558b568d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b56864dd0 0x558b56866ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 --2- 172.21.15.37:0/523597583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b568640f0 0x558b568669b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 >> 172.21.15.37:0/523597583 conn(0x558b5682b200 msgr2=0x558b5685ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.965+0000 7f3ddcc00640 0 client.26611 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 shutdown_connections 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3dfcdc3740 1 -- 172.21.15.37:0/523597583 wait complete. 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3dfcdc3740 20 client.26611 trim_cache size 0 max 16384 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:15:53.964 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout: "710059c1-974c-41bd-90c2-dc9be99b79f8": { 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "710059c1-974c-41bd-90c2-dc9be99b79f8", 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:15:53.965 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:15:53.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3dd80a3090 msgr2=0x7f3dd80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3df8a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3dd80a3090 0x7f3dd80a5550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x558b56865320 tx=0x7f3de4005f90 comp rx=0 tx=0).stop 2024-03-21T09:15:53.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 msgr2=0x7f3df419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:15:53.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.966+0000 7f3df8a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 0x7f3df419a490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e9c0 tx=0x7f3de000ee80 comp rx=0 tx=0).stop 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 shutdown_connections 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3df4109690 0x7f3df419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3dd80a3090 0x7f3dd80a5550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3df4104040 0x7f3df419a490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 --2- 172.21.15.37:0/2774674452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3df4103680 0x7f3df4199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 >> 172.21.15.37:0/2774674452 conn(0x7f3df40ff110 msgr2=0x7f3df40ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 shutdown_connections 2024-03-21T09:15:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:15:53.967+0000 7f3df8a00640 1 -- 172.21.15.37:0/2774674452 wait complete. 2024-03-21T09:15:53.979 INFO:tasks.cephfs.filesystem:scrub status for tag:710059c1-974c-41bd-90c2-dc9be99b79f8 - {'path': '/', 'tag': '710059c1-974c-41bd-90c2-dc9be99b79f8', 'options': 'recursive,force'} 2024-03-21T09:15:54.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:54 smithi071 ceph-mon[36597]: pgmap v114: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:54.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:54 smithi037 ceph-mon[31830]: pgmap v114: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:54.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:54 smithi053 ceph-mon[36169]: pgmap v114: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:15:55.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:55 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:15:55.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:55 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:55.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:55 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:15:55.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:55 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:55.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:55 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:15:55.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:55 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:55.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:55 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:15:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:55 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:55.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:55 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:15:55.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:55 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:55.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:55 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:15:55.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:55 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:15:56.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:56 smithi071 ceph-mon[36597]: pgmap v115: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:56.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:56 smithi037 ceph-mon[31830]: pgmap v115: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:56.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:56 smithi053 ceph-mon[36169]: pgmap v115: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:15:57.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:57 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:15:57.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:57 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:15:57.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:57 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:15:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:58 smithi071 ceph-mon[36597]: pgmap v116: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:15:58 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:15:58.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:58 smithi037 ceph-mon[31830]: pgmap v116: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:58.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:15:58 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:15:58.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:58 smithi053 ceph-mon[36169]: pgmap v116: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:15:58.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:15:58 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:16:00.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:00 smithi071 ceph-mon[36597]: pgmap v117: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:00.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:00 smithi037 ceph-mon[31830]: pgmap v117: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:00.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:00 smithi053 ceph-mon[36169]: pgmap v117: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:02.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:02 smithi071 ceph-mon[36597]: pgmap v118: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:02.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:02 smithi037 ceph-mon[31830]: pgmap v118: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:02.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:02 smithi053 ceph-mon[36169]: pgmap v118: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:04.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:04 smithi071 ceph-mon[36597]: pgmap v119: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:16:04.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:04 smithi037 ceph-mon[31830]: pgmap v119: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:16:04.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:04 smithi053 ceph-mon[36169]: pgmap v119: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:16:06.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:06 smithi071 ceph-mon[36597]: pgmap v120: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:06.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:06 smithi037 ceph-mon[31830]: pgmap v120: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:06.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:06 smithi053 ceph-mon[36169]: pgmap v120: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:08.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:08 smithi071 ceph-mon[36597]: pgmap v121: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:16:08.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:08 smithi037 ceph-mon[31830]: pgmap v121: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:16:08.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:08 smithi053 ceph-mon[36169]: pgmap v121: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:16:10.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:10 smithi071 ceph-mon[36597]: pgmap v122: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:10.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:10 smithi037 ceph-mon[31830]: pgmap v122: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:10.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:10 smithi053 ceph-mon[36169]: pgmap v122: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:11.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:11 smithi071 ceph-mon[36597]: pgmap v123: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:11.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:11 smithi037 ceph-mon[31830]: pgmap v123: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:11.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:11 smithi053 ceph-mon[36169]: pgmap v123: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:16:14.441 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:14 smithi053 ceph-mon[36169]: pgmap v124: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:16:14.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:14 smithi071 ceph-mon[36597]: pgmap v124: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:16:14.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:14 smithi037 ceph-mon[31830]: pgmap v124: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:16:16.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:16 smithi071 ceph-mon[36597]: pgmap v125: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:16:16.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:16 smithi037 ceph-mon[31830]: pgmap v125: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:16:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:16 smithi053 ceph-mon[36169]: pgmap v125: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:16:18.474 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:18 smithi037 ceph-mon[31830]: pgmap v126: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:16:18.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:18 smithi071 ceph-mon[36597]: pgmap v126: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:16:18.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:18 smithi053 ceph-mon[36169]: pgmap v126: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:16:20.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:20 smithi071 ceph-mon[36597]: pgmap v127: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:16:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:20 smithi037 ceph-mon[31830]: pgmap v127: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:16:20.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:20 smithi053 ceph-mon[36169]: pgmap v127: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:16:22.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:22 smithi071 ceph-mon[36597]: pgmap v128: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:16:22.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:22 smithi037 ceph-mon[31830]: pgmap v128: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:16:22.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:22 smithi053 ceph-mon[36169]: pgmap v128: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:16:23.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:16:23.981 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:16:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 -- 172.21.15.37:0/441547063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6390073a70 msgr2=0x7f6390073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 --2- 172.21.15.37:0/441547063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6390073a70 0x7f6390073e50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6380009900 tx=0x7f638002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 -- 172.21.15.37:0/441547063 shutdown_connections 2024-03-21T09:16:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 --2- 172.21.15.37:0/441547063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639010dce0 0x7f63901100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 --2- 172.21.15.37:0/441547063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 0x7f639010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 --2- 172.21.15.37:0/441547063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6390073a70 0x7f6390073e50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 -- 172.21.15.37:0/441547063 >> 172.21.15.37:0/441547063 conn(0x7f63900fcf50 msgr2=0x7f63900ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 -- 172.21.15.37:0/441547063 shutdown_connections 2024-03-21T09:16:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.260+0000 7f638f200640 1 -- 172.21.15.37:0/441547063 wait complete. 2024-03-21T09:16:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.262+0000 7f638f200640 1 Processor -- start 2024-03-21T09:16:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.262+0000 7f638f200640 1 -- start start 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.262+0000 7f638f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 0x7f639019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 0x7f639019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f639010dce0 0x7f639019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390113a90 con 0x7f6390074390 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390113910 con 0x7f6390073a70 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390113c10 con 0x7f639010dce0 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 0x7f639019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 0x7f639019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:39978/0 (socket says 172.21.15.37:39978) 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 -- 172.21.15.37:0/577144181 learned_addr learned my addr 172.21.15.37:0/577144181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638e800640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f639010dce0 0x7f639019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f6387e00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 0x7f639019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 -- 172.21.15.37:0/577144181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f639010dce0 msgr2=0x7f639019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f639010dce0 0x7f639019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 -- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 msgr2=0x7f639019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 0x7f639019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638de00640 1 -- 172.21.15.37:0/577144181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639019f020 con 0x7f6390073a70 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f6387e00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 0x7f639019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.263+0000 7f638e800640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f639010dce0 0x7f639019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.264+0000 7f638de00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 0x7f639019a030 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6380000c00 tx=0x7f6380004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.264+0000 7f6386a00640 1 -- 172.21.15.37:0/577144181 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380042020 con 0x7f6390073a70 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.264+0000 7f6386a00640 1 -- 172.21.15.37:0/577144181 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63800054e0 con 0x7f6390073a70 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.264+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639019f2b0 con 0x7f6390073a70 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.264+0000 7f6386a00640 1 -- 172.21.15.37:0/577144181 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380003a80 con 0x7f6390073a70 2024-03-21T09:16:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.264+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63901a6ba0 con 0x7f6390073a70 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.267+0000 7f6386a00640 1 -- 172.21.15.37:0/577144181 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6380045020 con 0x7f6390073a70 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.267+0000 7f639787d740 1 Processor -- start 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.267+0000 7f639787d740 1 -- start start 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f639787d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b5fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f639787d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b60570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f639787d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b6b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f639787d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d4ac4250 con 0x5557d4b69460 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f639787d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d4a6ca30 con 0x5557d4b60dd0 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f639787d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d4ac02f0 con 0x5557d4b600f0 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b5fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b60570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b5fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:39990/0 (socket says 172.21.15.37:39990) 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 -- 172.21.15.37:0/832083330 learned_addr learned my addr 172.21.15.37:0/832083330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f6387e00640 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b6b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 -- 172.21.15.37:0/832083330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 msgr2=0x5557d4b60570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b60570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 -- 172.21.15.37:0/832083330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 msgr2=0x5557d4b6b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b6b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.268+0000 7f638e800640 1 -- 172.21.15.37:0/832083330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557d4a57690 con 0x5557d4b60dd0 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f6387e00640 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b6b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f6386a00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63740a3110 0x7f63740a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f638e800640 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b5fb20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6388011790 tx=0x7f6388011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b14acd2889e28e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f6386a00640 1 -- 172.21.15.37:0/577144181 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f63800f16c0 con 0x7f6390073a70 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f638de00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63740a3110 0x7f63740a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f637f000640 1 -- 172.21.15.37:0/832083330 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388004d20 con 0x5557d4b60dd0 2024-03-21T09:16:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f637f000640 1 -- 172.21.15.37:0/832083330 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6388003970 con 0x5557d4b60dd0 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.269+0000 7f637f000640 1 -- 172.21.15.37:0/832083330 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880045c0 con 0x5557d4b60dd0 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 -- 172.21.15.37:0/832083330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 msgr2=0x5557d4b5fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f638de00640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63740a3110 0x7f63740a55d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5557d4b61320 tx=0x7f63800f7530 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b5fb20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6388011790 tx=0x7f6388011c50 comp rx=0 tx=0).stop 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 -- 172.21.15.37:0/832083330 shutdown_connections 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b6b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b60570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 --2- 172.21.15.37:0/832083330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b5fb20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 -- 172.21.15.37:0/832083330 >> 172.21.15.37:0/832083330 conn(0x5557d4b27200 msgr2=0x5557d4b5ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 -- 172.21.15.37:0/832083330 shutdown_connections 2024-03-21T09:16:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.270+0000 7f639787d740 1 -- 172.21.15.37:0/832083330 wait complete. 2024-03-21T09:16:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.271+0000 7f639787d740 1 -- start start 2024-03-21T09:16:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.271+0000 7f639787d740 1 Processor -- start 2024-03-21T09:16:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b65f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b66470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d4b6c2d0 con 0x5557d4b69460 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d4a7fe80 con 0x5557d4b60dd0 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d4b6c450 con 0x5557d4b600f0 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f639787d740 10 client.0 _ll_register_callbacks cb 0x5557d4a92130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f638e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f6387e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b65f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f6387e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b65f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:39994/0 (socket says 172.21.15.37:39994) 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f638e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b68ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44778/0 (socket says 172.21.15.37:44778) 2024-03-21T09:16:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f6387e00640 1 -- 172.21.15.37:0/2414059865 learned_addr learned my addr 172.21.15.37:0/2414059865 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.272+0000 7f638e800640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b66470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6387e00640 1 -- 172.21.15.37:0/2414059865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 msgr2=0x5557d4b68ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6387e00640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b68ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6387e00640 1 -- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 msgr2=0x5557d4b66470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6387e00640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b66470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6387e00640 1 -- 172.21.15.37:0/2414059865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557d48a27d0 con 0x5557d4b60dd0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f638e800640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b66470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6387e00640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b65f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6378013790 tx=0x7f6378013b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b77f38992712e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6373400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6373400640 1 -- 172.21.15.37:0/2414059865 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378020060 con 0x5557d4b60dd0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f6373400640 1 -- 172.21.15.37:0/2414059865 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6378011dd0 con 0x5557d4b60dd0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.273+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6388003970 con 0x5557d4b60dd0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f6373400640 1 -- 172.21.15.37:0/2414059865 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6378027050 con 0x5557d4b60dd0 2024-03-21T09:16:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 10 client.26597 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5557d47d83f0 con 0x5557d4b60dd0 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f6373400640 1 -- 172.21.15.37:0/2414059865 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6388003970 con 0x5557d4b60dd0 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f6373400640 1 -- 172.21.15.37:0/2414059865 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f637801a030 con 0x5557d4b60dd0 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 10 client.26597 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 20 client.26597 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4b79440 0x5557d4b7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 4 client.26597 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.274+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557d4ad1790 con 0x5557d4b79440 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.275+0000 7f638e800640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4b79440 0x5557d4b7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.275+0000 7f638e800640 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4b79440 0x5557d4b7b860 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebe368b18e74324 server_cookie=99a985f72a90c935 in_seq=0 out_seq=0 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.275+0000 7f6373400640 10 client.26597 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f6373400640 1 -- 172.21.15.37:0/2414059865 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5557d4ad1790 con 0x5557d4b79440 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f6373400640 10 client.26597 handle_command_reply: tid=0 2024-03-21T09:16:24.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 client.26597 shutdown 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 msgr2=0x5557d4b65f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b65f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6378013790 tx=0x7f6378013b90 comp rx=0 tx=0).stop 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 shutdown_connections 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b69460 0x5557d4b66470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4b79440 0x5557d4b7b860 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b60dd0 0x5557d4b65f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 --2- 172.21.15.37:0/2414059865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b600f0 0x5557d4b68ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 >> 172.21.15.37:0/2414059865 conn(0x5557d4b27200 msgr2=0x5557d4b5c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.276+0000 7f6373400640 0 client.26597 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.277+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 shutdown_connections 2024-03-21T09:16:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.277+0000 7f639787d740 1 -- 172.21.15.37:0/2414059865 wait complete. 2024-03-21T09:16:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.277+0000 7f639787d740 20 client.26597 trim_cache size 0 max 16384 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.294+0000 7f639787d740 1 Processor -- start 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.294+0000 7f639787d740 1 -- start start 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.294+0000 7f639787d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.294+0000 7f639787d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f639787d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f639787d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901b3040 con 0x5557d4b60dd0 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f639787d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901b0b10 con 0x5557d4b600f0 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f639787d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901aebb0 con 0x5557d4b69460 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b68660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40004/0 (socket says 172.21.15.37:40004) 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 -- 172.21.15.37:0/1245997442 learned_addr learned my addr 172.21.15.37:0/1245997442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f6387e00640 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b6a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 -- 172.21.15.37:0/1245997442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 msgr2=0x5557d4b6a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b6a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 -- 172.21.15.37:0/1245997442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 msgr2=0x5557d4b68ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b68ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 -- 172.21.15.37:0/1245997442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557d47980f0 con 0x5557d4b600f0 2024-03-21T09:16:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.295+0000 7f638e800640 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b68660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6388011730 tx=0x7f6388000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9106ef3f78710729 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.296+0000 7f637e600640 1 -- 172.21.15.37:0/1245997442 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638801d040 con 0x5557d4b600f0 2024-03-21T09:16:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.296+0000 7f637e600640 1 -- 172.21.15.37:0/1245997442 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f638800fe00 con 0x5557d4b600f0 2024-03-21T09:16:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.296+0000 7f637e600640 1 -- 172.21.15.37:0/1245997442 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638802e520 con 0x5557d4b600f0 2024-03-21T09:16:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.296+0000 7f639787d740 1 -- 172.21.15.37:0/1245997442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 msgr2=0x5557d4b68660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.296+0000 7f639787d740 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b68660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6388011730 tx=0x7f6388000f40 comp rx=0 tx=0).stop 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 -- 172.21.15.37:0/1245997442 shutdown_connections 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b6a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b68ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 --2- 172.21.15.37:0/1245997442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b68660 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 -- 172.21.15.37:0/1245997442 >> 172.21.15.37:0/1245997442 conn(0x5557d4b27200 msgr2=0x5557d4b64820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 -- 172.21.15.37:0/1245997442 shutdown_connections 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 -- 172.21.15.37:0/1245997442 wait complete. 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 -- start start 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.297+0000 7f639787d740 1 Processor -- start 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901af1b0 con 0x5557d4b60dd0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901b04b0 con 0x5557d4b600f0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901af7f0 con 0x5557d4b69460 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f639787d740 10 client.0 _ll_register_callbacks cb 0x7f6378009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f638e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f6387e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f638e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b75130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40014/0 (socket says 172.21.15.37:40014) 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f638e800640 1 -- 172.21.15.37:0/2898633190 learned_addr learned my addr 172.21.15.37:0/2898633190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.298+0000 7f638e800640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6387e00640 1 -- 172.21.15.37:0/2898633190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 msgr2=0x5557d4b62700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6387e00640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b62700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6387e00640 1 -- 172.21.15.37:0/2898633190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 msgr2=0x5557d4b75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6387e00640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b75130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6387e00640 1 -- 172.21.15.37:0/2898633190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557d4b62df0 con 0x5557d4b60dd0 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f638e800640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:24.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6387e00640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b621c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6378009d40 tx=0x7f6378005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abae3f124aa72e7e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6373400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6373400640 1 -- 172.21.15.37:0/2898633190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378009d70 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6373400640 1 -- 172.21.15.37:0/2898633190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6378011bd0 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f638800fe00 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f6373400640 1 -- 172.21.15.37:0/2898633190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6378011df0 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f639787d740 10 client.16401 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.299+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5557d4bd5d40 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f6373400640 1 -- 172.21.15.37:0/2898633190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f638800fe00 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f6373400640 1 -- 172.21.15.37:0/2898633190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f637801a040 con 0x5557d4b60dd0 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f639787d740 10 client.16401 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f639787d740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f639787d740 20 client.16401 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f639787d740 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4bd63d0 0x5557d4bd87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f639787d740 4 client.16401 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:16:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5557d4acbae0 con 0x5557d4bd63d0 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.300+0000 7f638e800640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4bd63d0 0x5557d4bd87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f638e800640 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4bd63d0 0x5557d4bd87f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f32224b6c2e30b server_cookie=413fda1b891267fa in_seq=0 out_seq=0 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f6373400640 10 client.16401 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f6373400640 1 -- 172.21.15.37:0/2898633190 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5557d4acbae0 con 0x5557d4bd63d0 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f6373400640 10 client.16401 handle_command_reply: tid=0 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f639787d740 1 client.16401 shutdown 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 msgr2=0x5557d4b621c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.301+0000 7f639787d740 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b621c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6378009d40 tx=0x7f6378005920 comp rx=0 tx=0).stop 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 shutdown_connections 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557d4b69460 0x5557d4b62700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557d4b60dd0 0x5557d4b621c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557d4bd63d0 0x5557d4bd87f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 --2- 172.21.15.37:0/2898633190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557d4b600f0 0x5557d4b75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f6373400640 0 client.16401 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 >> 172.21.15.37:0/2898633190 conn(0x5557d4b27200 msgr2=0x5557d4b5be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 shutdown_connections 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 1 -- 172.21.15.37:0/2898633190 wait complete. 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.302+0000 7f639787d740 20 client.16401 trim_cache size 0 max 16384 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:16:24.302 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:16:24.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63740a3110 msgr2=0x7f63740a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63740a3110 0x7f63740a55d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5557d4b61320 tx=0x7f63800f7530 comp rx=0 tx=0).stop 2024-03-21T09:16:24.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 msgr2=0x7f639019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 0x7f639019a030 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6380000c00 tx=0x7f6380004360 comp rx=0 tx=0).stop 2024-03-21T09:16:24.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 shutdown_connections 2024-03-21T09:16:24.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f639010dce0 0x7f639019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6390074390 0x7f639019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63740a3110 0x7f63740a55d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 --2- 172.21.15.37:0/577144181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6390073a70 0x7f639019a030 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 >> 172.21.15.37:0/577144181 conn(0x7f63900fcf50 msgr2=0x7f63900fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 shutdown_connections 2024-03-21T09:16:24.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.303+0000 7f638f200640 1 -- 172.21.15.37:0/577144181 wait complete. 2024-03-21T09:16:24.314 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:16:24.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:16:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:24 smithi037 ceph-mon[31830]: pgmap v129: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 -- 172.21.15.37:0/1047269931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 msgr2=0x7fd8681079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 --2- 172.21.15.37:0/1047269931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 0x7fd8681079e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd858009900 tx=0x7fd85802bc40 comp rx=0 tx=0).stop 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 -- 172.21.15.37:0/1047269931 shutdown_connections 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 --2- 172.21.15.37:0/1047269931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101f00 0x7fd86810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 --2- 172.21.15.37:0/1047269931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101540 0x7fd8681019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 --2- 172.21.15.37:0/1047269931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 0x7fd8681079e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 -- 172.21.15.37:0/1047269931 >> 172.21.15.37:0/1047269931 conn(0x7fd8680fd070 msgr2=0x7fd8680ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 -- 172.21.15.37:0/1047269931 shutdown_connections 2024-03-21T09:16:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.542+0000 7fd867e00640 1 -- 172.21.15.37:0/1047269931 wait complete. 2024-03-21T09:16:24.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.543+0000 7fd867e00640 1 Processor -- start 2024-03-21T09:16:24.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.543+0000 7fd867e00640 1 -- start start 2024-03-21T09:16:24.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 0x7fd868199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101f00 0x7fd86819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 0x7fd86819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681118c0 con 0x7fd868107600 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868111740 con 0x7fd868101f00 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868111a40 con 0x7fd868101540 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd866a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 0x7fd868199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd85e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101f00 0x7fd86819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd866a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 0x7fd868199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44818/0 (socket says 172.21.15.37:44818) 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd866a00640 1 -- 172.21.15.37:0/2324376160 learned_addr learned my addr 172.21.15.37:0/2324376160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.544+0000 7fd867400640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 0x7fd86819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd866a00640 1 -- 172.21.15.37:0/2324376160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101f00 msgr2=0x7fd86819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd866a00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101f00 0x7fd86819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd866a00640 1 -- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 msgr2=0x7fd86819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd866a00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 0x7fd86819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd866a00640 1 -- 172.21.15.37:0/2324376160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86819efd0 con 0x7fd868101540 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd85e000640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101f00 0x7fd86819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd866a00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 0x7fd868199fe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd858006010 tx=0x7fd8580042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858041070 con 0x7fd868101540 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86819f260 con 0x7fd868101540 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd85802ccd0 con 0x7fd868101540 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858003bd0 con 0x7fd868101540 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.545+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8681a6b50 con 0x7fd868101540 2024-03-21T09:16:24.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.547+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86804f470 con 0x7fd868101540 2024-03-21T09:16:24.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.548+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd858044070 con 0x7fd868101540 2024-03-21T09:16:24.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.550+0000 7fd85fe00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd84c0a3080 0x7fd84c0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:24.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.550+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd8580f14f0 con 0x7fd868101540 2024-03-21T09:16:24.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.550+0000 7fd85e000640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd84c0a3080 0x7fd84c0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:24.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.551+0000 7fd85e000640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd84c0a3080 0x7fd84c0a5540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd86810e0c0 tx=0x7fd850005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:24.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.554+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd8580bdcf0 con 0x7fd868101540 2024-03-21T09:16:24.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:24 smithi071 ceph-mon[36597]: pgmap v129: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:16:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:24 smithi053 ceph-mon[36169]: pgmap v129: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:16:24.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.800+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8681a6d20 con 0x7fd868101540 2024-03-21T09:16:24.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.801+0000 7fd85fe00640 1 -- 172.21.15.37:0/2324376160 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11373 (secure 0 0 0) 0x7fd8580bd5f0 con 0x7fd868101540 2024-03-21T09:16:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:16:24.795 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:15:56.490657+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24575,24589]},"id":1}]} 2024-03-21T09:16:24.795 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.803+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd84c0a3080 msgr2=0x7fd84c0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd84c0a3080 0x7fd84c0a5540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd86810e0c0 tx=0x7fd850005f90 comp rx=0 tx=0).stop 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 msgr2=0x7fd868199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 0x7fd868199fe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd858006010 tx=0x7fd8580042f0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 shutdown_connections 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd868107600 0x7fd86819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd84c0a3080 0x7fd84c0a5540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd868101f00 0x7fd86819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 --2- 172.21.15.37:0/2324376160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd868101540 0x7fd868199fe0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 >> 172.21.15.37:0/2324376160 conn(0x7fd8680fd070 msgr2=0x7fd86810d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 shutdown_connections 2024-03-21T09:16:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:24.804+0000 7fd867e00640 1 -- 172.21.15.37:0/2324376160 wait complete. 2024-03-21T09:16:24.818 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:16:24.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 -- 172.21.15.37:0/567110298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105380 msgr2=0x7f11a81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 --2- 172.21.15.37:0/567110298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105380 0x7f11a81057e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1194009890 tx=0x7f119402bc40 comp rx=0 tx=0).stop 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 -- 172.21.15.37:0/567110298 shutdown_connections 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 --2- 172.21.15.37:0/567110298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8105d20 0x7f11a8110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 --2- 172.21.15.37:0/567110298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105380 0x7f11a81057e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 --2- 172.21.15.37:0/567110298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8103c40 0x7f11a8104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 -- 172.21.15.37:0/567110298 >> 172.21.15.37:0/567110298 conn(0x7f11a80ff350 msgr2=0x7f11a8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 -- 172.21.15.37:0/567110298 shutdown_connections 2024-03-21T09:16:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.064+0000 7f11b0200640 1 -- 172.21.15.37:0/567110298 wait complete. 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.065+0000 7f11b0200640 1 Processor -- start 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.065+0000 7f11b0200640 1 -- start start 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11b0200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 0x7f11a819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11b0200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8105380 0x7f11a819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11b0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105d20 0x7f11a819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11b0200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8113d90 con 0x7f11a8103c40 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11b0200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8113c10 con 0x7f11a8105380 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11b0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8113f10 con 0x7f11a8105d20 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 0x7f11a819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 0x7f11a819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54824/0 (socket says 172.21.15.37:54824) 2024-03-21T09:16:25.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ace00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8105380 0x7f11a819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ad800640 1 -- 172.21.15.37:0/1850995191 learned_addr learned my addr 172.21.15.37:0/1850995191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ae200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105d20 0x7f11a819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ace00640 1 -- 172.21.15.37:0/1850995191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105d20 msgr2=0x7f11a819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ace00640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105d20 0x7f11a819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ace00640 1 -- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 msgr2=0x7f11a819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ace00640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 0x7f11a819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ace00640 1 -- 172.21.15.37:0/1850995191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a819f230 con 0x7f11a8105380 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.066+0000 7f11ad800640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 0x7f11a819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.067+0000 7f11ace00640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8105380 0x7f11a819a7d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f11940057c0 tx=0x7f1194005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.067+0000 7f11a6000640 1 -- 172.21.15.37:0/1850995191 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194041020 con 0x7f11a8105380 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.067+0000 7f11a6000640 1 -- 172.21.15.37:0/1850995191 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119402ca60 con 0x7f11a8105380 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.067+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a819f4c0 con 0x7f11a8105380 2024-03-21T09:16:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.067+0000 7f11a6000640 1 -- 172.21.15.37:0/1850995191 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194039040 con 0x7f11a8105380 2024-03-21T09:16:25.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.067+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a81a6db0 con 0x7f11a8105380 2024-03-21T09:16:25.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.071+0000 7f11a6000640 1 -- 172.21.15.37:0/1850995191 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1194044020 con 0x7f11a8105380 2024-03-21T09:16:25.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.072+0000 7f11b2fe2740 1 Processor -- start 2024-03-21T09:16:25.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.072+0000 7f11b2fe2740 1 -- start start 2024-03-21T09:16:25.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.072+0000 7f11a6000640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f11900a3110 0x7f11900a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.072+0000 7f11a6000640 1 -- 172.21.15.37:0/1850995191 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f11940f1080 con 0x7f11a8105380 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.072+0000 7f11ae200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f11900a3110 0x7f11900a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d3f6d0 0x5637b7d3fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d4b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b7ca32c0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b7c4bc90 con 0x5637b7d48c00 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b7c9f360 con 0x5637b7d40080 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ae200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d4b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44838/0 (socket says 172.21.15.37:44838) 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 -- 172.21.15.37:0/1725730931 learned_addr learned my addr 172.21.15.37:0/1725730931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ad800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d3f6d0 0x5637b7d3fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 -- 172.21.15.37:0/1725730931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 msgr2=0x5637b7d4b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d4b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 -- 172.21.15.37:0/1725730931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d3f6d0 msgr2=0x5637b7d3fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d3f6d0 0x5637b7d3fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.073+0000 7f11ace00640 1 -- 172.21.15.37:0/1725730931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b7c70080 con 0x5637b7d40080 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.074+0000 7f11ae200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f11900a3110 0x7f11900a55d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a0014ab0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.074+0000 7f11ace00640 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11940c5660 tx=0x7f1194003920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ea2d34a0d6bc08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.074+0000 7f119aa00640 1 -- 172.21.15.37:0/1725730931 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11940c12d0 con 0x5637b7d40080 2024-03-21T09:16:25.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.074+0000 7f119aa00640 1 -- 172.21.15.37:0/1725730931 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11940c5b90 con 0x5637b7d40080 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.074+0000 7f119aa00640 1 -- 172.21.15.37:0/1725730931 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11940b97f0 con 0x5637b7d40080 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 -- 172.21.15.37:0/1725730931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 msgr2=0x5637b7d48630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11940c5660 tx=0x7f1194003920 comp rx=0 tx=0).stop 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 -- 172.21.15.37:0/1725730931 shutdown_connections 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d4b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48630 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/1725730931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d3f6d0 0x5637b7d3fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 -- 172.21.15.37:0/1725730931 >> 172.21.15.37:0/1725730931 conn(0x5637b7d39da0 msgr2=0x5637b7d3a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 -- 172.21.15.37:0/1725730931 shutdown_connections 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 -- 172.21.15.37:0/1725730931 wait complete. 2024-03-21T09:16:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 -- start start 2024-03-21T09:16:25.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.075+0000 7f11b2fe2740 1 Processor -- start 2024-03-21T09:16:25.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d40080 0x5637b7d458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d45f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b7d4b930 con 0x5637b7d40080 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b7c5f0d0 con 0x5637b7d48c00 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b7d4bab0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11b2fe2740 10 client.0 _ll_register_callbacks cb 0x5637b7c63e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11ad800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11ad800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44846/0 (socket says 172.21.15.37:44846) 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11ad800640 1 -- 172.21.15.37:0/4072123267 learned_addr learned my addr 172.21.15.37:0/4072123267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11ace00640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d40080 0x5637b7d458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.076+0000 7f11ae200640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d45f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11ad800640 1 -- 172.21.15.37:0/4072123267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 msgr2=0x5637b7d45f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11ad800640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d45f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11ad800640 1 -- 172.21.15.37:0/4072123267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d40080 msgr2=0x5637b7d458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11ad800640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d40080 0x5637b7d458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11ad800640 1 -- 172.21.15.37:0/4072123267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637b79b73f0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11ad800640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48560 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f119c011d10 tx=0x7f119c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765d6d4005eb1628 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f118f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f118f400640 1 -- 172.21.15.37:0/4072123267 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c00fe20 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f118f400640 1 -- 172.21.15.37:0/4072123267 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f119c017050 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.077+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11940c5b90 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f118f400640 1 -- 172.21.15.37:0/4072123267 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f119c027050 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 10 client.26647 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5637b7a817d0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f118f400640 1 -- 172.21.15.37:0/4072123267 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11940c5b90 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f118f400640 1 -- 172.21.15.37:0/4072123267 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f119c022030 con 0x5637b7d3f6d0 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 10 client.26647 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 10 client.26647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:16:25.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 20 client.26647 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7d588a0 0x5637b7d5acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 4 client.26647 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637b7cb0800 con 0x5637b7d588a0 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.078+0000 7f11ace00640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7d588a0 0x5637b7d5acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.079+0000 7f11ace00640 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7d588a0 0x5637b7d5acd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e7310c9aedb9fdf server_cookie=c036570d5e442668 in_seq=0 out_seq=0 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.079+0000 7f118f400640 10 client.26647 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.079+0000 7f118f400640 1 -- 172.21.15.37:0/4072123267 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5637b7cb0800 con 0x5637b7d588a0 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.079+0000 7f118f400640 10 client.26647 handle_command_reply: tid=0 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 client.26647 shutdown 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 msgr2=0x5637b7d48560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48560 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f119c011d10 tx=0x7f119c00f790 comp rx=0 tx=0).stop 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 shutdown_connections 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d48c00 0x5637b7d45f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7d588a0 0x5637b7d5acd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d40080 0x5637b7d458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4072123267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48560 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 >> 172.21.15.37:0/4072123267 conn(0x5637b7d39da0 msgr2=0x5637b7d3baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f118f400640 0 client.26647 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 shutdown_connections 2024-03-21T09:16:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4072123267 wait complete. 2024-03-21T09:16:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.080+0000 7f11b2fe2740 20 client.26647 trim_cache size 0 max 16384 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 Processor -- start 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 -- start start 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d42b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81b3100 con 0x5637b7d48c00 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81b0bd0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81aec70 con 0x5637b7d40080 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11ad800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11ad800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40052/0 (socket says 172.21.15.37:40052) 2024-03-21T09:16:25.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11ace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11ad800640 1 -- 172.21.15.37:0/4226677977 learned_addr learned my addr 172.21.15.37:0/4226677977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.094+0000 7f11ae200640 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d42b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ad800640 1 -- 172.21.15.37:0/4226677977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 msgr2=0x5637b7d48560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ad800640 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ad800640 1 -- 172.21.15.37:0/4226677977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 msgr2=0x5637b7d42b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ad800640 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d42b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ad800640 1 -- 172.21.15.37:0/4226677977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b7d3f290 con 0x5637b7d3f6d0 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ace00640 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f11ad800640 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48020 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f119c000c00 tx=0x7f119c00fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd60d0408890a7b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f119a000640 1 -- 172.21.15.37:0/4226677977 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c01a3f0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f119a000640 1 -- 172.21.15.37:0/4226677977 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119c01a590 con 0x5637b7d3f6d0 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.095+0000 7f119a000640 1 -- 172.21.15.37:0/4226677977 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0048a0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4226677977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 msgr2=0x5637b7d48020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48020 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f119c000c00 tx=0x7f119c00fc90 comp rx=0 tx=0).stop 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4226677977 shutdown_connections 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d42b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d40080 0x5637b7d48560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/4226677977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d3f6d0 0x5637b7d48020 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4226677977 >> 172.21.15.37:0/4226677977 conn(0x5637b7d39da0 msgr2=0x5637b7d38eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4226677977 shutdown_connections 2024-03-21T09:16:25.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.096+0000 7f11b2fe2740 1 -- 172.21.15.37:0/4226677977 wait complete. 2024-03-21T09:16:25.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.097+0000 7f11b2fe2740 1 -- start start 2024-03-21T09:16:25.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.097+0000 7f11b2fe2740 1 Processor -- start 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d40080 0x5637b7d4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d4ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81af270 con 0x5637b7d48c00 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81b0570 con 0x5637b7d40080 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81af8b0 con 0x5637b7d3f6d0 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11b2fe2740 10 client.0 _ll_register_callbacks cb 0x7f119c009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ace00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d40080 0x5637b7d4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d4e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44862/0 (socket says 172.21.15.37:44862) 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 -- 172.21.15.37:0/3346793939 learned_addr learned my addr 172.21.15.37:0/3346793939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ae200640 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d4ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 -- 172.21.15.37:0/3346793939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d40080 msgr2=0x5637b7d4e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d40080 0x5637b7d4e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 -- 172.21.15.37:0/3346793939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 msgr2=0x5637b7d4ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d4ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.098+0000 7f11ad800640 1 -- 172.21.15.37:0/3346793939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637b7db3170 con 0x5637b7d3f6d0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f11ad800640 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d4e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f119c0096d0 tx=0x7f119c004720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d117edc1af982566 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f118f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f118f400640 1 -- 172.21.15.37:0/3346793939 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c03b060 con 0x5637b7d3f6d0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f119c01a590 con 0x5637b7d3f6d0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f118f400640 1 -- 172.21.15.37:0/3346793939 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f119c017040 con 0x5637b7d3f6d0 2024-03-21T09:16:25.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f118f400640 1 -- 172.21.15.37:0/3346793939 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f119c042050 con 0x5637b7d3f6d0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f11b2fe2740 10 client.26653 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5637b7db4440 con 0x5637b7d3f6d0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.099+0000 7f118f400640 1 -- 172.21.15.37:0/3346793939 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119c01a590 con 0x5637b7d3f6d0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f118f400640 1 -- 172.21.15.37:0/3346793939 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f119c022040 con 0x5637b7d3f6d0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11b2fe2740 10 client.26653 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11b2fe2740 10 client.26653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11b2fe2740 20 client.26653 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7db4ab0 0x5637b7db6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11b2fe2740 4 client.26653 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637b7caab50 con 0x5637b7db4ab0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11ace00640 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7db4ab0 0x5637b7db6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f11ace00640 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7db4ab0 0x5637b7db6ed0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51ff99fc7fb01332 server_cookie=3faa78ce23195bcc in_seq=0 out_seq=0 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.100+0000 7f118f400640 10 client.26653 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:25.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f118f400640 1 -- 172.21.15.37:0/3346793939 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637b7caab50 con 0x5637b7db4ab0 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f118f400640 10 client.26653 handle_command_reply: tid=0 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 client.26653 shutdown 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 msgr2=0x5637b7d4e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d4e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f119c0096d0 tx=0x7f119c004720 comp rx=0 tx=0).stop 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 shutdown_connections 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5637b7db4ab0 0x5637b7db6ed0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637b7d48c00 0x5637b7d4ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637b7d40080 0x5637b7d4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f118f400640 0 client.26653 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 --2- 172.21.15.37:0/3346793939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b7d3f6d0 0x5637b7d4e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 >> 172.21.15.37:0/3346793939 conn(0x5637b7d39da0 msgr2=0x5637b7d404b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 shutdown_connections 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 1 -- 172.21.15.37:0/3346793939 wait complete. 2024-03-21T09:16:25.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.101+0000 7f11b2fe2740 20 client.26653 trim_cache size 0 max 16384 2024-03-21T09:16:25.099 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f11900a3110 msgr2=0x7f11900a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f11900a3110 0x7f11900a55d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a0014ab0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8105380 msgr2=0x7f11a819a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8105380 0x7f11a819a7d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f11940057c0 tx=0x7f1194005d00 comp rx=0 tx=0).stop 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 shutdown_connections 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8105d20 0x7f11a819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f11900a3110 0x7f11900a55d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11a8105380 0x7f11a819a7d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 --2- 172.21.15.37:0/1850995191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a8103c40 0x7f11a819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 >> 172.21.15.37:0/1850995191 conn(0x7f11a80ff350 msgr2=0x7f11a8107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 shutdown_connections 2024-03-21T09:16:25.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.102+0000 7f11b0200640 1 -- 172.21.15.37:0/1850995191 wait complete. 2024-03-21T09:16:25.119 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:16:25.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:16:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 -- 172.21.15.37:0/1924336709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c101640 msgr2=0x7fcb6c101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 --2- 172.21.15.37:0/1924336709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c101640 0x7fcb6c101ac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009930 tx=0x7fcb5802bc30 comp rx=0 tx=0).stop 2024-03-21T09:16:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 -- 172.21.15.37:0/1924336709 shutdown_connections 2024-03-21T09:16:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 --2- 172.21.15.37:0/1924336709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c102000 0x7fcb6c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 --2- 172.21.15.37:0/1924336709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c101640 0x7fcb6c101ac0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 --2- 172.21.15.37:0/1924336709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c107650 0x7fcb6c107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 -- 172.21.15.37:0/1924336709 >> 172.21.15.37:0/1924336709 conn(0x7fcb6c0fd130 msgr2=0x7fcb6c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 -- 172.21.15.37:0/1924336709 shutdown_connections 2024-03-21T09:16:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.340+0000 7fcb74200640 1 -- 172.21.15.37:0/1924336709 wait complete. 2024-03-21T09:16:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.342+0000 7fcb74200640 1 Processor -- start 2024-03-21T09:16:25.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 -- start start 2024-03-21T09:16:25.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 0x7fcb6c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c102000 0x7fcb6c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c107650 0x7fcb6c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c111ab0 con 0x7fcb6c102000 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c111930 con 0x7fcb6c101640 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.343+0000 7fcb74200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c111c30 con 0x7fcb6c107650 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 0x7fcb6c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 0x7fcb6c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40080/0 (socket says 172.21.15.37:40080) 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 -- 172.21.15.37:0/3230812386 learned_addr learned my addr 172.21.15.37:0/3230812386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb72200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c107650 0x7fcb6c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb70e00640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c102000 0x7fcb6c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 -- 172.21.15.37:0/3230812386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c107650 msgr2=0x7fcb6c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c107650 0x7fcb6c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 -- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c102000 msgr2=0x7fcb6c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c102000 0x7fcb6c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 -- 172.21.15.37:0/3230812386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c19f190 con 0x7fcb6c101640 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb70e00640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c102000 0x7fcb6c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.344+0000 7fcb71800640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 0x7fcb6c19a1a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.345+0000 7fcb6a000640 1 -- 172.21.15.37:0/3230812386 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000ccc0 con 0x7fcb6c101640 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.345+0000 7fcb6a000640 1 -- 172.21.15.37:0/3230812386 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb60010910 con 0x7fcb6c101640 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.345+0000 7fcb6a000640 1 -- 172.21.15.37:0/3230812386 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600188a0 con 0x7fcb6c101640 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.345+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c19f480 con 0x7fcb6c101640 2024-03-21T09:16:25.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.345+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c1a6d10 con 0x7fcb6c101640 2024-03-21T09:16:25.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.348+0000 7fcb6a000640 1 -- 172.21.15.37:0/3230812386 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcb60018a40 con 0x7fcb6c101640 2024-03-21T09:16:25.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.348+0000 7fcb76fa3740 1 Processor -- start 2024-03-21T09:16:25.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.348+0000 7fcb76fa3740 1 -- start start 2024-03-21T09:16:25.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d69b142c0 con 0x561d69bb06d0 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d69abcc90 con 0x561d69bb1080 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d69b10360 con 0x561d69bb9c00 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb72200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40084/0 (socket says 172.21.15.37:40084) 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 -- 172.21.15.37:0/3721666677 learned_addr learned my addr 172.21.15.37:0/3721666677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb70e00640 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 -- 172.21.15.37:0/3721666677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 msgr2=0x561d69bbc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 -- 172.21.15.37:0/3721666677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 msgr2=0x561d69bb0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.349+0000 7fcb71800640 1 -- 172.21.15.37:0/3721666677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d69ae1080 con 0x561d69bb1080 2024-03-21T09:16:25.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb6a000640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb540a3110 0x7fcb540a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb71800640 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb9630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcb600cd2c0 tx=0x7fcb600d3f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a5c469756d0a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb6a000640 1 -- 172.21.15.37:0/3230812386 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcb600cfff0 con 0x7fcb6c101640 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb72200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb540a3110 0x7fcb540a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb5ea00640 1 -- 172.21.15.37:0/3721666677 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600982c0 con 0x561d69bb1080 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb5ea00640 1 -- 172.21.15.37:0/3721666677 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb6009c720 con 0x561d69bb1080 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.350+0000 7fcb5ea00640 1 -- 172.21.15.37:0/3721666677 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600d3bd0 con 0x561d69bb1080 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 -- 172.21.15.37:0/3721666677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 msgr2=0x561d69bb9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb9630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcb600cd2c0 tx=0x7fcb600d3f90 comp rx=0 tx=0).stop 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb72200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb540a3110 0x7fcb540a55d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005fd0 tx=0x7fcb64009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 -- 172.21.15.37:0/3721666677 shutdown_connections 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb9630 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/3721666677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 -- 172.21.15.37:0/3721666677 >> 172.21.15.37:0/3721666677 conn(0x561d69baada0 msgr2=0x561d69bab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 -- 172.21.15.37:0/3721666677 shutdown_connections 2024-03-21T09:16:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.351+0000 7fcb76fa3740 1 -- 172.21.15.37:0/3721666677 wait complete. 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 -- start start 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 Processor -- start 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bb6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d69bbcb70 con 0x561d69bb06d0 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d69ad00d0 con 0x561d69bb1080 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.352+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d69bbccf0 con 0x561d69bb9c00 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb76fa3740 10 client.0 _ll_register_callbacks cb 0x561d69ad4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bb6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb70e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb71800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb70e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40094/0 (socket says 172.21.15.37:40094) 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb70e00640 1 -- 172.21.15.37:0/2717522691 learned_addr learned my addr 172.21.15.37:0/2717522691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bb6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44890/0 (socket says 172.21.15.37:44890) 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 -- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 msgr2=0x561d69bb67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 -- 172.21.15.37:0/2717522691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 msgr2=0x561d69bb9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 -- 172.21.15.37:0/2717522691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d698283f0 con 0x561d69bb9c00 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.353+0000 7fcb72200640 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bb6d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcb64003f30 tx=0x7fcb64004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb61d527ba5de2f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb53400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb53400640 1 -- 172.21.15.37:0/2717522691 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400f8d0 con 0x561d69bb9c00 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb6009c720 con 0x561d69bb9c00 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb53400640 1 -- 172.21.15.37:0/2717522691 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcb6400fa70 con 0x561d69bb9c00 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb53400640 1 -- 172.21.15.37:0/2717522691 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb6400fc70 con 0x561d69bb9c00 2024-03-21T09:16:25.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb76fa3740 10 client.26665 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561d698f27d0 con 0x561d69bb9c00 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.354+0000 7fcb53400640 1 -- 172.21.15.37:0/2717522691 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb6009c720 con 0x561d69bb9c00 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb53400640 1 -- 172.21.15.37:0/2717522691 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fcb64027070 con 0x561d69bb9c00 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb76fa3740 10 client.26665 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb76fa3740 10 client.26665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb76fa3740 20 client.26665 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69bc9a90 0x561d69bcbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb76fa3740 4 client.26665 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d69b21800 con 0x561d69bc9a90 2024-03-21T09:16:25.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb71800640 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69bc9a90 0x561d69bcbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb71800640 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69bc9a90 0x561d69bcbef0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=867c472ef85bf614 server_cookie=d081b78d266f9c1f in_seq=0 out_seq=0 2024-03-21T09:16:25.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.355+0000 7fcb53400640 10 client.26665 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb53400640 1 -- 172.21.15.37:0/2717522691 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d69b21800 con 0x561d69bc9a90 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb53400640 10 client.26665 handle_command_reply: tid=0 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 client.26665 shutdown 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 msgr2=0x561d69bb6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bb6d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcb64003f30 tx=0x7fcb64004ab0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 shutdown_connections 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69bc9a90 0x561d69bcbef0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bb6d30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb53400640 0 client.26665 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bb67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/2717522691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bb9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 >> 172.21.15.37:0/2717522691 conn(0x561d69baada0 msgr2=0x561d69bacaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 shutdown_connections 2024-03-21T09:16:25.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 1 -- 172.21.15.37:0/2717522691 wait complete. 2024-03-21T09:16:25.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.357+0000 7fcb76fa3740 20 client.26665 trim_cache size 0 max 16384 2024-03-21T09:16:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.373+0000 7fcb76fa3740 1 Processor -- start 2024-03-21T09:16:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 -- start start 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb06d0 0x561d69bb8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 0x561d69bb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb9c00 0x561d69bb3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1b32a0 con 0x561d69bb9c00 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1b0d70 con 0x561d69bb06d0 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1aee10 con 0x561d69bb1080 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb70e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 0x561d69bb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb72200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb9c00 0x561d69bb3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb71800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb06d0 0x561d69bb8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb70e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 0x561d69bb9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44902/0 (socket says 172.21.15.37:44902) 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.374+0000 7fcb70e00640 1 -- 172.21.15.37:0/1057314772 learned_addr learned my addr 172.21.15.37:0/1057314772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb70e00640 1 -- 172.21.15.37:0/1057314772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb06d0 msgr2=0x561d69bb8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb70e00640 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb06d0 0x561d69bb8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb70e00640 1 -- 172.21.15.37:0/1057314772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb9c00 msgr2=0x561d69bb3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb70e00640 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb9c00 0x561d69bb3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb70e00640 1 -- 172.21.15.37:0/1057314772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d69bb7070 con 0x561d69bb1080 2024-03-21T09:16:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb70e00640 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 0x561d69bb9460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009230 tx=0x7fcb5802c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=502bca351522eb27 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb5e000640 1 -- 172.21.15.37:0/1057314772 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004300 con 0x561d69bb1080 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb5e000640 1 -- 172.21.15.37:0/1057314772 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb580044a0 con 0x561d69bb1080 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.375+0000 7fcb5e000640 1 -- 172.21.15.37:0/1057314772 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004920 con 0x561d69bb1080 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 -- 172.21.15.37:0/1057314772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 msgr2=0x561d69bb9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 0x561d69bb9460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009230 tx=0x7fcb5802c710 comp rx=0 tx=0).stop 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 -- 172.21.15.37:0/1057314772 shutdown_connections 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb9c00 0x561d69bb3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb1080 0x561d69bb9460 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/1057314772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb06d0 0x561d69bb8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 -- 172.21.15.37:0/1057314772 >> 172.21.15.37:0/1057314772 conn(0x561d69baada0 msgr2=0x561d69ba9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 -- 172.21.15.37:0/1057314772 shutdown_connections 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 -- 172.21.15.37:0/1057314772 wait complete. 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.376+0000 7fcb76fa3740 1 -- start start 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 Processor -- start 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bc1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bc1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1af410 con 0x561d69bb06d0 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1b0710 con 0x561d69bb1080 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.377+0000 7fcb76fa3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1afa50 con 0x561d69bb9c00 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb76fa3740 10 client.0 _ll_register_callbacks cb 0x7fcb64009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bc1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bc1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40112/0 (socket says 172.21.15.37:40112) 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 -- 172.21.15.37:0/291540047 learned_addr learned my addr 172.21.15.37:0/291540047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb71800640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bc1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb72200640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 -- 172.21.15.37:0/291540047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 msgr2=0x561d69bbdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 -- 172.21.15.37:0/291540047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 msgr2=0x561d69bc1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bc1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 -- 172.21.15.37:0/291540047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d69bbe2e0 con 0x561d69bb1080 2024-03-21T09:16:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb72200640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.378+0000 7fcb70e00640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bc1bc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009850 tx=0x7fcb58009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92e72022bc7bbe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb53400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb53400640 1 -- 172.21.15.37:0/291540047 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58058060 con 0x561d69bb1080 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb580044a0 con 0x561d69bb1080 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb53400640 1 -- 172.21.15.37:0/291540047 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcb58005040 con 0x561d69bb1080 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb53400640 1 -- 172.21.15.37:0/291540047 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb58005240 con 0x561d69bb1080 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb76fa3740 10 client.26645 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561d69c25df0 con 0x561d69bb1080 2024-03-21T09:16:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb53400640 1 -- 172.21.15.37:0/291540047 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb580044a0 con 0x561d69bb1080 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb53400640 1 -- 172.21.15.37:0/291540047 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fcb58003710 con 0x561d69bb1080 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.379+0000 7fcb76fa3740 10 client.26645 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb76fa3740 10 client.26645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb76fa3740 20 client.26645 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69c26470 0x561d69c28860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb76fa3740 4 client.26645 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d69b1bb50 con 0x561d69c26470 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb71800640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69c26470 0x561d69c28860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb71800640 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69c26470 0x561d69c28860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f11b15948dfdaf5 server_cookie=bc352f6f1ea892fa in_seq=0 out_seq=0 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb53400640 10 client.26645 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb53400640 1 -- 172.21.15.37:0/291540047 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d69b1bb50 con 0x561d69c26470 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.380+0000 7fcb53400640 10 client.26645 handle_command_reply: tid=0 2024-03-21T09:16:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 client.26645 shutdown 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 msgr2=0x561d69bc1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bc1bc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009850 tx=0x7fcb58009260 comp rx=0 tx=0).stop 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 shutdown_connections 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561d69c26470 0x561d69c28860 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d69bb9c00 0x561d69bbdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d69bb1080 0x561d69bc1bc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 --2- 172.21.15.37:0/291540047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d69bb06d0 0x561d69bc1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 >> 172.21.15.37:0/291540047 conn(0x561d69baada0 msgr2=0x561d69ba9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb53400640 0 client.26645 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 shutdown_connections 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 1 -- 172.21.15.37:0/291540047 wait complete. 2024-03-21T09:16:25.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.381+0000 7fcb76fa3740 20 client.26645 trim_cache size 0 max 16384 2024-03-21T09:16:25.380 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:16:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb540a3110 msgr2=0x7fcb540a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb540a3110 0x7fcb540a55d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005fd0 tx=0x7fcb64009500 comp rx=0 tx=0).stop 2024-03-21T09:16:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 msgr2=0x7fcb6c19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 0x7fcb6c19a1a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000edf0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 shutdown_connections 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb6c107650 0x7fcb6c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb6c102000 0x7fcb6c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb540a3110 0x7fcb540a55d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 --2- 172.21.15.37:0/3230812386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb6c101640 0x7fcb6c19a1a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 >> 172.21.15.37:0/3230812386 conn(0x7fcb6c0fd130 msgr2=0x7fcb6c10cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 shutdown_connections 2024-03-21T09:16:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.382+0000 7fcb74200640 1 -- 172.21.15.37:0/3230812386 wait complete. 2024-03-21T09:16:25.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:25 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2324376160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:16:25.396 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:16:25.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:16:25.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:25 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2324376160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 -- 172.21.15.37:0/3838487825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 msgr2=0x7f5f68105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 --2- 172.21.15.37:0/3838487825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f68105200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009900 tx=0x7f5f5402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 -- 172.21.15.37:0/3838487825 shutdown_connections 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 --2- 172.21.15.37:0/3838487825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68105820 0x7f5f68105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 --2- 172.21.15.37:0/3838487825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f68105200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 --2- 172.21.15.37:0/3838487825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68100420 0x7f5f68100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 -- 172.21.15.37:0/3838487825 >> 172.21.15.37:0/3838487825 conn(0x7f5f68077130 msgr2=0x7f5f680fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 -- 172.21.15.37:0/3838487825 shutdown_connections 2024-03-21T09:16:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.627+0000 7f5f67200640 1 -- 172.21.15.37:0/3838487825 wait complete. 2024-03-21T09:16:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.629+0000 7f5f67200640 1 Processor -- start 2024-03-21T09:16:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.629+0000 7f5f67200640 1 -- start start 2024-03-21T09:16:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.629+0000 7f5f67200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68100420 0x7f5f6819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.629+0000 7f5f67200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f6819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.629+0000 7f5f67200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68105820 0x7f5f681a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f67200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68117e80 con 0x7f5f68100420 2024-03-21T09:16:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f67200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68117d00 con 0x7f5f68100d40 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f67200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68118000 con 0x7f5f68105820 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68100420 0x7f5f6819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f6819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f6819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40130/0 (socket says 172.21.15.37:40130) 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68100420 0x7f5f6819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54910/0 (socket says 172.21.15.37:54910) 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 -- 172.21.15.37:0/943129236 learned_addr learned my addr 172.21.15.37:0/943129236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f66800640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68105820 0x7f5f681a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 -- 172.21.15.37:0/943129236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68105820 msgr2=0x7f5f681a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68105820 0x7f5f681a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 -- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68100420 msgr2=0x7f5f6819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68100420 0x7f5f6819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f65400640 1 -- 172.21.15.37:0/943129236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f681a3370 con 0x7f5f68100d40 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.630+0000 7f5f66800640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68105820 0x7f5f681a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.631+0000 7f5f65400640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f6819e8c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f54000c00 tx=0x7f5f54004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.631+0000 7f5f5ea00640 1 -- 172.21.15.37:0/943129236 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54042020 con 0x7f5f68100d40 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.631+0000 7f5f5ea00640 1 -- 172.21.15.37:0/943129236 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f54005000 con 0x7f5f68100d40 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.631+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f681a3600 con 0x7f5f68100d40 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.631+0000 7f5f5ea00640 1 -- 172.21.15.37:0/943129236 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54003b20 con 0x7f5f68100d40 2024-03-21T09:16:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.631+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f681aaef0 con 0x7f5f68100d40 2024-03-21T09:16:25.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 Processor -- start 2024-03-21T09:16:25.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f5ea00640 1 -- 172.21.15.37:0/943129236 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5f54045020 con 0x7f5f68100d40 2024-03-21T09:16:25.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 -- start start 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62c98b0 0x55f5b62c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b622d2c0 con 0x55f5b62c98b0 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b61d5c90 con 0x55f5b62ca260 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.636+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b6229360 con 0x55f5b62d2de0 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f66800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62c98b0 0x55f5b62c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62d2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40134/0 (socket says 172.21.15.37:40134) 2024-03-21T09:16:25.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 -- 172.21.15.37:0/611950245 learned_addr learned my addr 172.21.15.37:0/611950245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 -- 172.21.15.37:0/611950245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 msgr2=0x55f5b62d5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 -- 172.21.15.37:0/611950245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62c98b0 msgr2=0x55f5b62c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62c98b0 0x55f5b62c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65e00640 1 -- 172.21.15.37:0/611950245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5b61fa080 con 0x55f5b62ca260 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f65400640 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.637+0000 7f5f66800640 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62c98b0 0x55f5b62c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f65e00640 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62d2810 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009d20 tx=0x7f5f58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3abc591560373 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f53000640 1 -- 172.21.15.37:0/611950245 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5801e070 con 0x55f5b62ca260 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f53000640 1 -- 172.21.15.37:0/611950245 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f58019b10 con 0x55f5b62ca260 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f53000640 1 -- 172.21.15.37:0/611950245 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800ebf0 con 0x55f5b62ca260 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f5ea00640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5f4c0a3110 0x7f5f4c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f5ea00640 1 -- 172.21.15.37:0/943129236 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5f540f1570 con 0x7f5f68100d40 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.638+0000 7f5f66800640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5f4c0a3110 0x7f5f4c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 -- 172.21.15.37:0/611950245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 msgr2=0x55f5b62d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62d2810 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009d20 tx=0x7f5f58011810 comp rx=0 tx=0).stop 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 -- 172.21.15.37:0/611950245 shutdown_connections 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62d2810 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/611950245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62c98b0 0x55f5b62c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 -- 172.21.15.37:0/611950245 >> 172.21.15.37:0/611950245 conn(0x55f5b62c3f80 msgr2=0x55f5b62c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 -- 172.21.15.37:0/611950245 shutdown_connections 2024-03-21T09:16:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f66800640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5f4c0a3110 0x7f5f4c0a55d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009fd0 tx=0x7f5f60009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.639+0000 7f5f6fa35740 1 -- 172.21.15.37:0/611950245 wait complete. 2024-03-21T09:16:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.640+0000 7f5f6fa35740 1 -- start start 2024-03-21T09:16:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.640+0000 7f5f6fa35740 1 Processor -- start 2024-03-21T09:16:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62d2de0 0x55f5b62cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b62d5d50 con 0x55f5b62d2de0 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b61e90d0 con 0x55f5b62ca260 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b62d5ed0 con 0x55f5b62c98b0 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f6fa35740 10 client.0 _ll_register_callbacks cb 0x55f5b61ede80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44940/0 (socket says 172.21.15.37:44940) 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.641+0000 7f5f65e00640 1 -- 172.21.15.37:0/992309277 learned_addr learned my addr 172.21.15.37:0/992309277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65400640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f66800640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62d2de0 0x55f5b62cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65e00640 1 -- 172.21.15.37:0/992309277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 msgr2=0x55f5b62cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65e00640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65e00640 1 -- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62d2de0 msgr2=0x55f5b62cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65e00640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62d2de0 0x55f5b62cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65e00640 1 -- 172.21.15.37:0/992309277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5b5f413f0 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65400640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f65e00640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62d26a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5f5801b000 tx=0x7f5f5800e5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63adea924f5032b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.642+0000 7f5f47400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f47400640 1 -- 172.21.15.37:0/992309277 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58026020 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f58019b10 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f47400640 1 -- 172.21.15.37:0/992309277 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5f58019840 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f47400640 1 -- 172.21.15.37:0/992309277 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f58031050 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f6fa35740 10 client.26683 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f5b600b7d0 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.643+0000 7f5f47400640 1 -- 172.21.15.37:0/992309277 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f58019b10 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f47400640 1 -- 172.21.15.37:0/992309277 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f5f58005420 con 0x55f5b62c98b0 2024-03-21T09:16:25.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f6fa35740 10 client.26683 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:25.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f6fa35740 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:25.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f6fa35740 20 client.26683 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:25.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b62e2ae0 0x55f5b62e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f6fa35740 4 client.26683 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:25.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5b623a800 con 0x55f5b62e2ae0 2024-03-21T09:16:25.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.644+0000 7f5f65400640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b62e2ae0 0x55f5b62e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.645+0000 7f5f65400640 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b62e2ae0 0x55f5b62e4ed0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de08e1d4ba24bad0 server_cookie=a20fb9f964b8e066 in_seq=0 out_seq=0 2024-03-21T09:16:25.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.645+0000 7f5f47400640 10 client.26683 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:25.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.645+0000 7f5f47400640 1 -- 172.21.15.37:0/992309277 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f5b623a800 con 0x55f5b62e2ae0 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.645+0000 7f5f47400640 10 client.26683 handle_command_reply: tid=0 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.645+0000 7f5f6fa35740 1 client.26683 shutdown 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 msgr2=0x55f5b62d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62d26a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5f5801b000 tx=0x7f5f5800e5e0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 shutdown_connections 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62d2de0 0x55f5b62cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b62e2ae0 0x55f5b62e4ed0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62ca260 0x55f5b62cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f47400640 0 client.26683 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/992309277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62d26a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 >> 172.21.15.37:0/992309277 conn(0x55f5b62c3f80 msgr2=0x55f5b62c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 shutdown_connections 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 1 -- 172.21.15.37:0/992309277 wait complete. 2024-03-21T09:16:25.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.646+0000 7f5f6fa35740 20 client.26683 trim_cache size 0 max 16384 2024-03-21T09:16:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.662+0000 7f5f6fa35740 1 Processor -- start 2024-03-21T09:16:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 -- start start 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62d2de0 0x55f5b62cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b7380 con 0x55f5b62ca260 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b4e50 con 0x55f5b62d2de0 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b2ef0 con 0x55f5b62c98b0 2024-03-21T09:16:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44954/0 (socket says 172.21.15.37:44954) 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f65e00640 1 -- 172.21.15.37:0/3549891266 learned_addr learned my addr 172.21.15.37:0/3549891266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f65400640 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.663+0000 7f5f66800640 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62d2de0 0x55f5b62cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f65e00640 1 -- 172.21.15.37:0/3549891266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62d2de0 msgr2=0x55f5b62cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f65e00640 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62d2de0 0x55f5b62cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f65e00640 1 -- 172.21.15.37:0/3549891266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 msgr2=0x55f5b62c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f65e00640 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f65e00640 1 -- 172.21.15.37:0/3549891266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5b62d51d0 con 0x55f5b62c98b0 2024-03-21T09:16:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f65e00640 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62c33b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5f5801c040 tx=0x7f5f5800e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c14fae5aa7440c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f52600640 1 -- 172.21.15.37:0/3549891266 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58015070 con 0x55f5b62c98b0 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f52600640 1 -- 172.21.15.37:0/3549891266 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f58005b90 con 0x55f5b62c98b0 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.664+0000 7f5f52600640 1 -- 172.21.15.37:0/3549891266 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58026040 con 0x55f5b62c98b0 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 -- 172.21.15.37:0/3549891266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 msgr2=0x55f5b62c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62c33b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5f5801c040 tx=0x7f5f5800e6d0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 -- 172.21.15.37:0/3549891266 shutdown_connections 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62d2de0 0x55f5b62cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/3549891266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62c98b0 0x55f5b62c33b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.665+0000 7f5f6fa35740 1 -- 172.21.15.37:0/3549891266 >> 172.21.15.37:0/3549891266 conn(0x55f5b62c3f80 msgr2=0x55f5b62ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.666+0000 7f5f6fa35740 1 -- 172.21.15.37:0/3549891266 shutdown_connections 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.666+0000 7f5f6fa35740 1 -- 172.21.15.37:0/3549891266 wait complete. 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.666+0000 7f5f6fa35740 1 -- start start 2024-03-21T09:16:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.666+0000 7f5f6fa35740 1 Processor -- start 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 0x55f5b62dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b34f0 con 0x55f5b62ca260 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b47f0 con 0x55f5b62c98b0 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681b3b30 con 0x55f5b62d2de0 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f6fa35740 10 client.0 _ll_register_callbacks cb 0x7f5f5801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 0x55f5b62dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f65e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 0x55f5b62dacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40146/0 (socket says 172.21.15.37:40146) 2024-03-21T09:16:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f66800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f65e00640 1 -- 172.21.15.37:0/1007566191 learned_addr learned my addr 172.21.15.37:0/1007566191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.667+0000 7f5f65400640 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f65e00640 1 -- 172.21.15.37:0/1007566191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 msgr2=0x55f5b62d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f65e00640 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f65e00640 1 -- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 msgr2=0x55f5b62d6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f65e00640 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62d6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f65e00640 1 -- 172.21.15.37:0/1007566191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5b62d6c40 con 0x55f5b62c98b0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f65e00640 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 0x55f5b62dacf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005fd0 tx=0x7f5f58004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=134e55e6aa9a1d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f47400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f47400640 1 -- 172.21.15.37:0/1007566191 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58015040 con 0x55f5b62c98b0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f47400640 1 -- 172.21.15.37:0/1007566191 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5f5800ecc0 con 0x55f5b62c98b0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.668+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f58005b90 con 0x55f5b62c98b0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f47400640 1 -- 172.21.15.37:0/1007566191 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f58045050 con 0x55f5b62c98b0 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 10 client.26663 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f5b633dfd0 con 0x55f5b62c98b0 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f47400640 1 -- 172.21.15.37:0/1007566191 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f58005b90 con 0x55f5b62c98b0 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f47400640 1 -- 172.21.15.37:0/1007566191 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f5f580199e0 con 0x55f5b62c98b0 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 10 client.26663 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 20 client.26663 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b633e630 0x55f5b6340a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 4 client.26663 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.669+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5b6234b50 con 0x55f5b633e630 2024-03-21T09:16:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.670+0000 7f5f65400640 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b633e630 0x55f5b6340a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.670+0000 7f5f65400640 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b633e630 0x55f5b6340a20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafdeb9dbcfb66cb server_cookie=edb49a9e215751a2 in_seq=0 out_seq=0 2024-03-21T09:16:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.670+0000 7f5f47400640 10 client.26663 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f47400640 1 -- 172.21.15.37:0/1007566191 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5b6234b50 con 0x55f5b633e630 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f47400640 10 client.26663 handle_command_reply: tid=0 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 client.26663 shutdown 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 msgr2=0x55f5b62dacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 0x55f5b62dacf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005fd0 tx=0x7f5f58004b10 comp rx=0 tx=0).stop 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 shutdown_connections 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5b62d2de0 0x55f5b62d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5b62ca260 0x55f5b62d6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f5b633e630 0x55f5b6340a20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 --2- 172.21.15.37:0/1007566191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5b62c98b0 0x55f5b62dacf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f47400640 0 client.26663 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 >> 172.21.15.37:0/1007566191 conn(0x55f5b62c3f80 msgr2=0x55f5b62c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 shutdown_connections 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 1 -- 172.21.15.37:0/1007566191 wait complete. 2024-03-21T09:16:25.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.671+0000 7f5f6fa35740 20 client.26663 trim_cache size 0 max 16384 2024-03-21T09:16:25.668 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:16:25.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5f4c0a3110 msgr2=0x7f5f4c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5f4c0a3110 0x7f5f4c0a55d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009fd0 tx=0x7f5f60009500 comp rx=0 tx=0).stop 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 msgr2=0x7f5f6819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f6819e8c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f54000c00 tx=0x7f5f54004360 comp rx=0 tx=0).stop 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 shutdown_connections 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f68105820 0x7f5f681a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5f4c0a3110 0x7f5f4c0a55d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f68100d40 0x7f5f6819e8c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 --2- 172.21.15.37:0/943129236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f68100420 0x7f5f6819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.672+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 >> 172.21.15.37:0/943129236 conn(0x7f5f68077130 msgr2=0x7f5f681031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.673+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 shutdown_connections 2024-03-21T09:16:25.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:25.673+0000 7f5f67200640 1 -- 172.21.15.37:0/943129236 wait complete. 2024-03-21T09:16:25.690 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:25 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2324376160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:16:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:26 smithi071 ceph-mon[36597]: pgmap v130: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:26.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:26 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:16:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:26 smithi037 ceph-mon[31830]: pgmap v130: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:26.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:26 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:16:26.686 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:16:26.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:16:26.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:16:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:26 smithi053 ceph-mon[36169]: pgmap v130: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:26 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:16:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 -- 172.21.15.37:0/896385712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 msgr2=0x7f213c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 --2- 172.21.15.37:0/896385712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c1009b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f212c009900 tx=0x7f212c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:16:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 -- 172.21.15.37:0/896385712 shutdown_connections 2024-03-21T09:16:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 --2- 172.21.15.37:0/896385712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c1056a0 0x7f213c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 --2- 172.21.15.37:0/896385712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f213c100f80 0x7f213c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 --2- 172.21.15.37:0/896385712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c1009b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 -- 172.21.15.37:0/896385712 >> 172.21.15.37:0/896385712 conn(0x7f213c0fbe70 msgr2=0x7f213c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 -- 172.21.15.37:0/896385712 shutdown_connections 2024-03-21T09:16:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.918+0000 7f213be00640 1 -- 172.21.15.37:0/896385712 wait complete. 2024-03-21T09:16:26.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.919+0000 7f213be00640 1 Processor -- start 2024-03-21T09:16:26.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.919+0000 7f213be00640 1 -- start start 2024-03-21T09:16:26.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c10dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c1056a0 0x7f213c10e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f213c10e680 0x7f213c1a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1118a0 con 0x7f213c1056a0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c111720 con 0x7f213c10e680 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c111a20 con 0x7f213c1005d0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c10dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c10dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44970/0 (socket says 172.21.15.37:44970) 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213aa00640 1 -- 172.21.15.37:0/462940832 learned_addr learned my addr 172.21.15.37:0/462940832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213a000640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c1056a0 0x7f213c10e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.920+0000 7f213b400640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f213c10e680 0x7f213c1a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213aa00640 1 -- 172.21.15.37:0/462940832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f213c10e680 msgr2=0x7f213c1a5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213aa00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f213c10e680 0x7f213c1a5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213aa00640 1 -- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c1056a0 msgr2=0x7f213c10e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213aa00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c1056a0 0x7f213c10e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213aa00640 1 -- 172.21.15.37:0/462940832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c1a6650 con 0x7f213c1005d0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213aa00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c10dc00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f212c02c240 tx=0x7f212c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f2133400640 1 -- 172.21.15.37:0/462940832 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c043020 con 0x7f213c1005d0 2024-03-21T09:16:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f2133400640 1 -- 172.21.15.37:0/462940832 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f212c0054e0 con 0x7f213c1005d0 2024-03-21T09:16:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c1a68e0 con 0x7f213c1005d0 2024-03-21T09:16:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.921+0000 7f2133400640 1 -- 172.21.15.37:0/462940832 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c003b50 con 0x7f213c1005d0 2024-03-21T09:16:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.922+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c1a6e30 con 0x7f213c1005d0 2024-03-21T09:16:26.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.925+0000 7f2133400640 1 -- 172.21.15.37:0/462940832 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f212c046020 con 0x7f213c1005d0 2024-03-21T09:16:26.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.925+0000 7f21447d3740 1 Processor -- start 2024-03-21T09:16:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.925+0000 7f21447d3740 1 -- start start 2024-03-21T09:16:26.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f21447d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a18670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f21447d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f21447d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f21447d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22697ceb0 con 0x55b226a18bb0 2024-03-21T09:16:26.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f21447d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226925a60 con 0x55b226a21790 2024-03-21T09:16:26.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f21447d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226978f50 con 0x55b226a18290 2024-03-21T09:16:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f213b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a18670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f213b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a18670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44974/0 (socket says 172.21.15.37:44974) 2024-03-21T09:16:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f213a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f213b400640 1 -- 172.21.15.37:0/4218997758 learned_addr learned my addr 172.21.15.37:0/4218997758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f213aa00640 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.926+0000 7f2133400640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f21200a3210 0x7f21200a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213a000640 1 -- 172.21.15.37:0/4218997758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 msgr2=0x55b226a18670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213a000640 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a18670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213a000640 1 -- 172.21.15.37:0/4218997758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 msgr2=0x55b226a21250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213a000640 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a21250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213a000640 1 -- 172.21.15.37:0/4218997758 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2267597e0 con 0x55b226a21790 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f2133400640 1 -- 172.21.15.37:0/462940832 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f212c0f19b0 con 0x7f213c1005d0 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213b400640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f21200a3210 0x7f21200a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f213a000640 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a23b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f213c06ac50 tx=0x7f2128011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a23f7abd640a70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f2127e00640 1 -- 172.21.15.37:0/4218997758 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128004230 con 0x55b226a21790 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.927+0000 7f2127e00640 1 -- 172.21.15.37:0/4218997758 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2128009e10 con 0x55b226a21790 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f2127e00640 1 -- 172.21.15.37:0/4218997758 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128004d70 con 0x55b226a21790 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 -- 172.21.15.37:0/4218997758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 msgr2=0x55b226a23b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f213b400640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f21200a3210 0x7f21200a56d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2134005f40 tx=0x7f2134004a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a23b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f213c06ac50 tx=0x7f2128011810 comp rx=0 tx=0).stop 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 -- 172.21.15.37:0/4218997758 shutdown_connections 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a23b80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a21250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 --2- 172.21.15.37:0/4218997758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a18670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.928+0000 7f21447d3740 1 -- 172.21.15.37:0/4218997758 >> 172.21.15.37:0/4218997758 conn(0x55b226a129c0 msgr2=0x55b226a12da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.929+0000 7f21447d3740 1 -- 172.21.15.37:0/4218997758 shutdown_connections 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.929+0000 7f21447d3740 1 -- 172.21.15.37:0/4218997758 wait complete. 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.929+0000 7f21447d3740 1 -- start start 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.929+0000 7f21447d3740 1 Processor -- start 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a21150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a1e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a1eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226a24580 con 0x55b226a18bb0 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226938eb0 con 0x55b226a21790 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b226a24700 con 0x55b226a18290 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f21447d3740 10 client.0 _ll_register_callbacks cb 0x55b22692d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a21150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a1e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a21150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44980/0 (socket says 172.21.15.37:44980) 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213aa00640 1 -- 172.21.15.37:0/2829000582 learned_addr learned my addr 172.21.15.37:0/2829000582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213b400640 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a1eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213aa00640 1 -- 172.21.15.37:0/2829000582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 msgr2=0x55b226a1eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213aa00640 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a1eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.930+0000 7f213aa00640 1 -- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 msgr2=0x55b226a1e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f213aa00640 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a1e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f213aa00640 1 -- 172.21.15.37:0/2829000582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2267b9f70 con 0x55b226a18290 2024-03-21T09:16:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f213aa00640 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a21150 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f212c0ba4e0 tx=0x7f212c0bd9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ccbc35bd7ec20b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:26.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f2124c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:26.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f2124c00640 1 -- 172.21.15.37:0/2829000582 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0f1e60 con 0x55b226a18290 2024-03-21T09:16:26.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f2124c00640 1 -- 172.21.15.37:0/2829000582 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f212c0ba180 con 0x55b226a18290 2024-03-21T09:16:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2128009e10 con 0x55b226a18290 2024-03-21T09:16:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f2124c00640 1 -- 172.21.15.37:0/2829000582 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f212c004c80 con 0x55b226a18290 2024-03-21T09:16:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f21447d3740 10 client.26701 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.931+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b226651170 con 0x55b226a18290 2024-03-21T09:16:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f2124c00640 1 -- 172.21.15.37:0/2829000582 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2128009e10 con 0x55b226a18290 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f2124c00640 1 -- 172.21.15.37:0/2829000582 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f212c0c6090 con 0x55b226a18290 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f21447d3740 10 client.26701 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f21447d3740 10 client.26701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f21447d3740 20 client.26701 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f21447d3740 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a31770 0x55b226a33ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f21447d3740 4 client.26701 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b22698a3f0 con 0x55b226a31770 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.932+0000 7f213a000640 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a31770 0x55b226a33ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.933+0000 7f213a000640 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a31770 0x55b226a33ba0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94d35de52e5dd34 server_cookie=6479fbc2f7193ead in_seq=0 out_seq=0 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.933+0000 7f2124c00640 10 client.26701 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f2124c00640 1 -- 172.21.15.37:0/2829000582 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b22698a3f0 con 0x55b226a31770 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f2124c00640 10 client.26701 handle_command_reply: tid=0 2024-03-21T09:16:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 client.26701 shutdown 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 msgr2=0x55b226a21150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a21150 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f212c0ba4e0 tx=0x7f212c0bd9a0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 shutdown_connections 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a31770 0x55b226a33ba0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a1eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a1e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 --2- 172.21.15.37:0/2829000582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a21150 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f2124c00640 0 client.26701 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 >> 172.21.15.37:0/2829000582 conn(0x55b226a129c0 msgr2=0x55b226a14710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.934+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 shutdown_connections 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.935+0000 7f21447d3740 1 -- 172.21.15.37:0/2829000582 wait complete. 2024-03-21T09:16:26.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.935+0000 7f21447d3740 20 client.26701 trim_cache size 0 max 16384 2024-03-21T09:16:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 Processor -- start 2024-03-21T09:16:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 -- start start 2024-03-21T09:16:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18290 0x55b226a11cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a18bb0 0x55b226a20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 0x55b226a21150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1b2e80 con 0x55b226a18290 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1b0950 con 0x55b226a18bb0 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.953+0000 7f21447d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1ae9f0 con 0x55b226a21790 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 0x55b226a21150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 0x55b226a21150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44982/0 (socket says 172.21.15.37:44982) 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 -- 172.21.15.37:0/2115766498 learned_addr learned my addr 172.21.15.37:0/2115766498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213aa00640 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18290 0x55b226a11cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213a000640 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a18bb0 0x55b226a20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 -- 172.21.15.37:0/2115766498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a18bb0 msgr2=0x55b226a20c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a18bb0 0x55b226a20c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 -- 172.21.15.37:0/2115766498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18290 msgr2=0x55b226a11cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18290 0x55b226a11cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 -- 172.21.15.37:0/2115766498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b226a1a0e0 con 0x55b226a21790 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213aa00640 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18290 0x55b226a11cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.954+0000 7f213b400640 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 0x55b226a21150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2134010830 tx=0x7f2134016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc10411a6e06d361 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.955+0000 7f2127400640 1 -- 172.21.15.37:0/2115766498 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134028420 con 0x55b226a21790 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.955+0000 7f2127400640 1 -- 172.21.15.37:0/2115766498 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f213401d8f0 con 0x55b226a21790 2024-03-21T09:16:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.955+0000 7f2127400640 1 -- 172.21.15.37:0/2115766498 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213401b6c0 con 0x55b226a21790 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 -- 172.21.15.37:0/2115766498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 msgr2=0x55b226a21150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 0x55b226a21150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2134010830 tx=0x7f2134016000 comp rx=0 tx=0).stop 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 -- 172.21.15.37:0/2115766498 shutdown_connections 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a21790 0x55b226a21150 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a18bb0 0x55b226a20c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 --2- 172.21.15.37:0/2115766498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18290 0x55b226a11cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 -- 172.21.15.37:0/2115766498 >> 172.21.15.37:0/2115766498 conn(0x55b226a129c0 msgr2=0x55b226a14630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 -- 172.21.15.37:0/2115766498 shutdown_connections 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.956+0000 7f21447d3740 1 -- 172.21.15.37:0/2115766498 wait complete. 2024-03-21T09:16:26.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 -- start start 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 Processor -- start 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a26920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1aeff0 con 0x55b226a18bb0 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1b02f0 con 0x55b226a21790 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1af630 con 0x55b226a18290 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.957+0000 7f21447d3740 10 client.0 _ll_register_callbacks cb 0x7f212c0d40b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a26920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44990/0 (socket says 172.21.15.37:44990) 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 -- 172.21.15.37:0/1931739887 learned_addr learned my addr 172.21.15.37:0/1931739887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213a000640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 -- 172.21.15.37:0/1931739887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 msgr2=0x55b226a273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a273a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 -- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 msgr2=0x55b226a26e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a26e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 -- 172.21.15.37:0/1931739887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b226a8d240 con 0x55b226a18290 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213a000640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213b400640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f213aa00640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a26920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f212c0bdaa0 tx=0x7f212c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8db704059e29104 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.958+0000 7f2124c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f2124c00640 1 -- 172.21.15.37:0/1931739887 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0cb060 con 0x55b226a18290 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f2124c00640 1 -- 172.21.15.37:0/1931739887 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f212c0bd160 con 0x55b226a18290 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f213401d8f0 con 0x55b226a18290 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f2124c00640 1 -- 172.21.15.37:0/1931739887 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f212c0bd470 con 0x55b226a18290 2024-03-21T09:16:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 10 client.26707 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b226a8e500 con 0x55b226a18290 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f2124c00640 1 -- 172.21.15.37:0/1931739887 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f213401d8f0 con 0x55b226a18290 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f2124c00640 1 -- 172.21.15.37:0/1931739887 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f212c0bc9d0 con 0x55b226a18290 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 10 client.26707 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 20 client.26707 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a8e9e0 0x55b226a90dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 4 client.26707 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.959+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b226984740 con 0x55b226a8e9e0 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.960+0000 7f213a000640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a8e9e0 0x55b226a90dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.960+0000 7f213a000640 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a8e9e0 0x55b226a90dd0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8686ac83cd377e0b server_cookie=76a9474eec6dcbcc in_seq=0 out_seq=0 2024-03-21T09:16:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.960+0000 7f2124c00640 10 client.26707 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:26.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f2124c00640 1 -- 172.21.15.37:0/1931739887 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b226984740 con 0x55b226a8e9e0 2024-03-21T09:16:26.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f2124c00640 10 client.26707 handle_command_reply: tid=0 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 client.26707 shutdown 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 msgr2=0x55b226a26920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a26920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f212c0bdaa0 tx=0x7f212c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 shutdown_connections 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b226a8e9e0 0x55b226a90dd0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b226a21790 0x55b226a273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b226a18bb0 0x55b226a26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 --2- 172.21.15.37:0/1931739887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b226a18290 0x55b226a26920 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f2124c00640 0 client.26707 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.962+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 >> 172.21.15.37:0/1931739887 conn(0x55b226a129c0 msgr2=0x55b226a13b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.963+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 shutdown_connections 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.963+0000 7f21447d3740 1 -- 172.21.15.37:0/1931739887 wait complete. 2024-03-21T09:16:26.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.963+0000 7f21447d3740 20 client.26707 trim_cache size 0 max 16384 2024-03-21T09:16:26.961 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:16:26.961 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:16:26.961 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "6ee475db-dc79-4004-9c9f-d7bb8bfbb435", 2024-03-21T09:16:26.961 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:16:26.961 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f21200a3210 msgr2=0x7f21200a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f21200a3210 0x7f21200a56d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2134005f40 tx=0x7f2134004a40 comp rx=0 tx=0).stop 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 msgr2=0x7f213c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c10dc00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f212c02c240 tx=0x7f212c0042f0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 shutdown_connections 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f21200a3210 0x7f21200a56d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f213c10e680 0x7f213c1a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c1056a0 0x7f213c10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 --2- 172.21.15.37:0/462940832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f213c1005d0 0x7f213c10dc00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 >> 172.21.15.37:0/462940832 conn(0x7f213c0fbe70 msgr2=0x7f213c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:26.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 shutdown_connections 2024-03-21T09:16:26.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:26.964+0000 7f213be00640 1 -- 172.21.15.37:0/462940832 wait complete. 2024-03-21T09:16:26.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:16:26.978 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:16:27.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.197+0000 7ff0a1c00640 1 -- 172.21.15.37:0/3022820979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c103660 msgr2=0x7ff09c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.197+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/3022820979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c103660 0x7ff09c103ae0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff084009890 tx=0x7ff08402bb70 comp rx=0 tx=0).stop 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 -- 172.21.15.37:0/3022820979 shutdown_connections 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/3022820979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/3022820979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c103660 0x7ff09c103ae0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/3022820979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c109670 0x7ff09c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 -- 172.21.15.37:0/3022820979 >> 172.21.15.37:0/3022820979 conn(0x7ff09c0ff110 msgr2=0x7ff09c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 -- 172.21.15.37:0/3022820979 shutdown_connections 2024-03-21T09:16:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.198+0000 7ff0a1c00640 1 -- 172.21.15.37:0/3022820979 wait complete. 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.199+0000 7ff0a1c00640 1 Processor -- start 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.199+0000 7ff0a1c00640 1 -- start start 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.199+0000 7ff0a1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c103660 0x7ff09c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.199+0000 7ff0a1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff0a1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c109670 0x7ff09c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff0a1c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c113a80 con 0x7ff09c103660 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff0a1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c113900 con 0x7ff09c104020 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff0a1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c113c00 con 0x7ff09c109670 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40240/0 (socket says 172.21.15.37:40240) 2024-03-21T09:16:27.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 -- 172.21.15.37:0/2569950076 learned_addr learned my addr 172.21.15.37:0/2569950076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff0a1200640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c109670 0x7ff09c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff09be00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c103660 0x7ff09c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 -- 172.21.15.37:0/2569950076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c109670 msgr2=0x7ff09c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c109670 0x7ff09c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 -- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c103660 msgr2=0x7ff09c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c103660 0x7ff09c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff093e00640 1 -- 172.21.15.37:0/2569950076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c19efb0 con 0x7ff09c104020 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.200+0000 7ff09be00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c103660 0x7ff09c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:16:27.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.201+0000 7ff093e00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c19a500 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff0840057c0 tx=0x7ff0840045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:27.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.201+0000 7ff099600640 1 -- 172.21.15.37:0/2569950076 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084041020 con 0x7ff09c104020 2024-03-21T09:16:27.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.201+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c19f240 con 0x7ff09c104020 2024-03-21T09:16:27.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.201+0000 7ff099600640 1 -- 172.21.15.37:0/2569950076 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff08402c9f0 con 0x7ff09c104020 2024-03-21T09:16:27.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.201+0000 7ff099600640 1 -- 172.21.15.37:0/2569950076 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084039040 con 0x7ff09c104020 2024-03-21T09:16:27.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.201+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c1a6b30 con 0x7ff09c104020 2024-03-21T09:16:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.204+0000 7ff099600640 1 -- 172.21.15.37:0/2569950076 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff084044020 con 0x7ff09c104020 2024-03-21T09:16:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.204+0000 7ff0a5fd8740 1 Processor -- start 2024-03-21T09:16:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 -- start start 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad4db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad59850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6acb2250 con 0x55be6ad57460 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6ac5aa30 con 0x55be6ad4edd0 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6acae2f0 con 0x55be6ad4e0f0 2024-03-21T09:16:27.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad4db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad4db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40256/0 (socket says 172.21.15.37:40256) 2024-03-21T09:16:27.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff0a1200640 1 -- 172.21.15.37:0/1564759239 learned_addr learned my addr 172.21.15.37:0/1564759239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:27.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.205+0000 7ff09be00640 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff093e00640 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad59850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff0a1200640 1 -- 172.21.15.37:0/1564759239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 msgr2=0x55be6ad4e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff0a1200640 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad4e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff0a1200640 1 -- 172.21.15.37:0/1564759239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 msgr2=0x55be6ad59850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff0a1200640 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad59850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff0a1200640 1 -- 172.21.15.37:0/1564759239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6ac45690 con 0x55be6ad4edd0 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff0a1200640 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad4db20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff09400e450 tx=0x7ff0940095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab102d4339d91b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.206+0000 7ff091600640 1 -- 172.21.15.37:0/1564759239 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09400d910 con 0x55be6ad4edd0 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff091600640 1 -- 172.21.15.37:0/1564759239 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff09401c070 con 0x55be6ad4edd0 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff091600640 1 -- 172.21.15.37:0/1564759239 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094016380 con 0x55be6ad4edd0 2024-03-21T09:16:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff099600640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff0800a3090 0x7ff0800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff099600640 1 -- 172.21.15.37:0/2569950076 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff0840f13b0 con 0x7ff09c104020 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1564759239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 msgr2=0x55be6ad4db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad4db20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff09400e450 tx=0x7ff0940095c0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff09be00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff0800a3090 0x7ff0800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1564759239 shutdown_connections 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad59850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1564759239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad4db20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1564759239 >> 172.21.15.37:0/1564759239 conn(0x55be6ad15200 msgr2=0x55be6ad48b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.207+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1564759239 shutdown_connections 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.208+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1564759239 wait complete. 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.208+0000 7ff09be00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff0800a3090 0x7ff0800a5550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55be6ad4f320 tx=0x7ff08c005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.208+0000 7ff0a5fd8740 1 -- start start 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.208+0000 7ff0a5fd8740 1 Processor -- start 2024-03-21T09:16:27.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad56ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad53e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad54390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6ad5a2d0 con 0x55be6ad57460 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6ac6de80 con 0x55be6ad4edd0 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be6ad5a450 con 0x55be6ad4e0f0 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a5fd8740 10 client.0 _ll_register_callbacks cb 0x55be6ac80130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff09be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad56ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff09be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad56ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45020/0 (socket says 172.21.15.37:45020) 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff09be00640 1 -- 172.21.15.37:0/3823798250 learned_addr learned my addr 172.21.15.37:0/3823798250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff093e00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad53e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.209+0000 7ff0a1200640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad54390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff09be00640 1 -- 172.21.15.37:0/3823798250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 msgr2=0x55be6ad53e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff09be00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad53e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff09be00640 1 -- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 msgr2=0x55be6ad54390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff09be00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad54390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff09be00640 1 -- 172.21.15.37:0/3823798250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6aa907d0 con 0x55be6ad4e0f0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff093e00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad53e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff0a1200640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad54390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff09be00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad56ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff08c01e6e0 tx=0x7ff08c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd1212d6e703a9d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff08a000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff08a000640 1 -- 172.21.15.37:0/3823798250 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02b060 con 0x55be6ad4e0f0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff09401c070 con 0x55be6ad4e0f0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff08a000640 1 -- 172.21.15.37:0/3823798250 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff08c011730 con 0x55be6ad4e0f0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff08a000640 1 -- 172.21.15.37:0/3823798250 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff08c032050 con 0x55be6ad4e0f0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.210+0000 7ff0a5fd8740 10 client.26719 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55be6a9c63f0 con 0x55be6ad4e0f0 2024-03-21T09:16:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff08a000640 1 -- 172.21.15.37:0/3823798250 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff09401c070 con 0x55be6ad4e0f0 2024-03-21T09:16:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff08a000640 1 -- 172.21.15.37:0/3823798250 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7ff08c01a040 con 0x55be6ad4e0f0 2024-03-21T09:16:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 10 client.26719 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 10 client.26719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 20 client.26719 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6ad67380 0x55be6ad69790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 4 client.26719 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.211+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be6acbf790 con 0x55be6ad67380 2024-03-21T09:16:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.212+0000 7ff093e00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6ad67380 0x55be6ad69790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.212+0000 7ff093e00640 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6ad67380 0x55be6ad69790 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83806bdd3a539750 server_cookie=7653f1d029035964 in_seq=0 out_seq=0 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.212+0000 7ff08a000640 10 client.26719 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff08a000640 1 -- 172.21.15.37:0/3823798250 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be6acbf790 con 0x55be6ad67380 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff08a000640 10 client.26719 handle_command_reply: tid=0 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 client.26719 shutdown 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 msgr2=0x55be6ad56ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad56ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff08c01e6e0 tx=0x7ff08c0046c0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 shutdown_connections 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad54390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6ad67380 0x55be6ad69790 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad53e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/3823798250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad56ac0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff08a000640 0 client.26719 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 >> 172.21.15.37:0/3823798250 conn(0x55be6ad15200 msgr2=0x55be6ad4a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.213+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 shutdown_connections 2024-03-21T09:16:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.214+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/3823798250 wait complete. 2024-03-21T09:16:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.214+0000 7ff0a5fd8740 20 client.26719 trim_cache size 0 max 16384 2024-03-21T09:16:27.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 Processor -- start 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 -- start start 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad52e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad56580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad56ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1b2f20 con 0x55be6ad57460 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1b09f0 con 0x55be6ad4edd0 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.236+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1aea90 con 0x55be6ad4e0f0 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad56580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff0a1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad56ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff09be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad52e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad56580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:40264/0 (socket says 172.21.15.37:40264) 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 -- 172.21.15.37:0/4172385170 learned_addr learned my addr 172.21.15.37:0/4172385170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:27.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 -- 172.21.15.37:0/4172385170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 msgr2=0x55be6ad52e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad52e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 -- 172.21.15.37:0/4172385170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 msgr2=0x55be6ad56ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad56ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff093e00640 1 -- 172.21.15.37:0/4172385170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be6a9860f0 con 0x55be6ad4edd0 2024-03-21T09:16:27.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.237+0000 7ff09be00640 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.238+0000 7ff093e00640 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad56580 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff084035fd0 tx=0x7ff084005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a3f96c20089336e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.238+0000 7ff090c00640 1 -- 172.21.15.37:0/4172385170 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840bcff0 con 0x55be6ad4edd0 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.238+0000 7ff090c00640 1 -- 172.21.15.37:0/4172385170 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0840eee50 con 0x55be6ad4edd0 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.238+0000 7ff090c00640 1 -- 172.21.15.37:0/4172385170 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840bce10 con 0x55be6ad4edd0 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/4172385170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 msgr2=0x55be6ad56580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad56580 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff084035fd0 tx=0x7ff084005f90 comp rx=0 tx=0).stop 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/4172385170 shutdown_connections 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad57460 0x55be6ad56ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad4edd0 0x55be6ad56580 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/4172385170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4e0f0 0x55be6ad52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/4172385170 >> 172.21.15.37:0/4172385170 conn(0x55be6ad15200 msgr2=0x55be6ad52720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/4172385170 shutdown_connections 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/4172385170 wait complete. 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 -- start start 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.239+0000 7ff0a5fd8740 1 Processor -- start 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad4e0f0 0x55be6ad504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 0x55be6ad50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad57460 0x55be6ad50f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1af090 con 0x55be6ad4e0f0 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1b0390 con 0x55be6ad57460 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1af6d0 con 0x55be6ad4edd0 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a5fd8740 10 client.0 _ll_register_callbacks cb 0x7ff08c013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff093e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 0x55be6ad50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff09be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad4e0f0 0x55be6ad504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff093e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 0x55be6ad50a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45026/0 (socket says 172.21.15.37:45026) 2024-03-21T09:16:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff093e00640 1 -- 172.21.15.37:0/1357992691 learned_addr learned my addr 172.21.15.37:0/1357992691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff0a1200640 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad57460 0x55be6ad50f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff093e00640 1 -- 172.21.15.37:0/1357992691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad57460 msgr2=0x55be6ad50f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.240+0000 7ff093e00640 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad57460 0x55be6ad50f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff093e00640 1 -- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad4e0f0 msgr2=0x55be6ad504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff093e00640 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad4e0f0 0x55be6ad504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff093e00640 1 -- 172.21.15.37:0/1357992691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be6adc1c20 con 0x55be6ad4edd0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff093e00640 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 0x55be6ad50a10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff08402c180 tx=0x7ff0840bd280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f8498d656958ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff08a000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff08a000640 1 -- 172.21.15.37:0/1357992691 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840bce10 con 0x55be6ad4edd0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff08a000640 1 -- 172.21.15.37:0/1357992691 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff084034640 con 0x55be6ad4edd0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0840eee50 con 0x55be6ad4edd0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff08a000640 1 -- 172.21.15.37:0/1357992691 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0840bcff0 con 0x55be6ad4edd0 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff0a5fd8740 10 client.26725 fetch_fsmap learned FSMap version 29 2024-03-21T09:16:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.241+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55be6adc2e90 con 0x55be6ad4edd0 2024-03-21T09:16:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff08a000640 1 -- 172.21.15.37:0/1357992691 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0840eee50 con 0x55be6ad4edd0 2024-03-21T09:16:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff08a000640 1 -- 172.21.15.37:0/1357992691 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7930+0+0 (secure 0 0 0) 0x7ff08403c030 con 0x55be6ad4edd0 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff0a5fd8740 10 client.26725 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff0a5fd8740 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff0a5fd8740 20 client.26725 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6adc3430 0x55be6adc5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff0a5fd8740 4 client.26725 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be6acb9ae0 con 0x55be6adc3430 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.242+0000 7ff09be00640 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6adc3430 0x55be6adc5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff09be00640 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6adc3430 0x55be6adc5850 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5350d3c2f1bb5b6d server_cookie=4adf810483a1ec33 in_seq=0 out_seq=0 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff08a000640 10 client.26725 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff08a000640 1 -- 172.21.15.37:0/1357992691 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55be6acb9ae0 con 0x55be6adc3430 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff08a000640 10 client.26725 handle_command_reply: tid=0 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 client.26725 shutdown 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 msgr2=0x55be6ad50a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 0x55be6ad50a10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff08402c180 tx=0x7ff0840bd280 comp rx=0 tx=0).stop 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 shutdown_connections 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55be6adc3430 0x55be6adc5850 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be6ad57460 0x55be6ad50f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be6ad4edd0 0x55be6ad50a10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 --2- 172.21.15.37:0/1357992691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be6ad4e0f0 0x55be6ad504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff08a000640 0 client.26725 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.243+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 >> 172.21.15.37:0/1357992691 conn(0x55be6ad15200 msgr2=0x55be6ad52fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.244+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 shutdown_connections 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.244+0000 7ff0a5fd8740 1 -- 172.21.15.37:0/1357992691 wait complete. 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.244+0000 7ff0a5fd8740 20 client.26725 trim_cache size 0 max 16384 2024-03-21T09:16:27.241 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "6ee475db-dc79-4004-9c9f-d7bb8bfbb435": { 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "6ee475db-dc79-4004-9c9f-d7bb8bfbb435", 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:16:27.242 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff0800a3090 msgr2=0x7ff0800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff0800a3090 0x7ff0800a5550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55be6ad4f320 tx=0x7ff08c005f90 comp rx=0 tx=0).stop 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 msgr2=0x7ff09c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c19a500 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff0840057c0 tx=0x7ff0840045d0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 shutdown_connections 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff09c109670 0x7ff09c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff0800a3090 0x7ff0800a5550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff09c104020 0x7ff09c19a500 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 --2- 172.21.15.37:0/2569950076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff09c103660 0x7ff09c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 >> 172.21.15.37:0/2569950076 conn(0x7ff09c0ff110 msgr2=0x7ff09c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 shutdown_connections 2024-03-21T09:16:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:27.245+0000 7ff0a1c00640 1 -- 172.21.15.37:0/2569950076 wait complete. 2024-03-21T09:16:27.258 INFO:tasks.cephfs.filesystem:scrub status for tag:6ee475db-dc79-4004-9c9f-d7bb8bfbb435 - {'path': '/', 'tag': '6ee475db-dc79-4004-9c9f-d7bb8bfbb435', 'options': 'recursive,force'} 2024-03-21T09:16:27.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:27 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:16:27.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:27 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:16:27.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:27 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:16:27.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:27 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:16:27.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:27 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:16:27.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:27 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:16:27.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:27 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:16:27.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:27 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:16:27.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:27 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:16:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:28 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:16:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:28 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:16:28.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:28 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:16:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:28 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:16:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:28 smithi071 ceph-mon[36597]: pgmap v131: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:28.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:28 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:16:28.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:28 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:16:28.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:28 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:16:28.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:28 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:16:28.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:28 smithi037 ceph-mon[31830]: pgmap v131: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:28.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:28 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:16:28.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:28 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:16:28.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:28 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:16:28.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:28 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:16:28.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:28 smithi053 ceph-mon[36169]: pgmap v131: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:30.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:30 smithi071 ceph-mon[36597]: pgmap v132: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:30.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:30 smithi037 ceph-mon[31830]: pgmap v132: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:30.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:30 smithi053 ceph-mon[36169]: pgmap v132: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:32.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:32 smithi071 ceph-mon[36597]: pgmap v133: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:32.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:32 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:16:32.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:32 smithi037 ceph-mon[31830]: pgmap v133: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:32.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:32 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:16:32.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:32 smithi053 ceph-mon[36169]: pgmap v133: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:32.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:32 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:16:34.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:34 smithi071 ceph-mon[36597]: pgmap v134: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:34.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:34 smithi037 ceph-mon[31830]: pgmap v134: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:34.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:34 smithi053 ceph-mon[36169]: pgmap v134: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:36.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:36 smithi071 ceph-mon[36597]: pgmap v135: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:36.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:36 smithi037 ceph-mon[31830]: pgmap v135: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:36.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:36 smithi053 ceph-mon[36169]: pgmap v135: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:16:38.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:38 smithi071 ceph-mon[36597]: pgmap v136: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:38.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:38 smithi037 ceph-mon[31830]: pgmap v136: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:38.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:38 smithi053 ceph-mon[36169]: pgmap v136: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:16:40.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:40 smithi071 ceph-mon[36597]: pgmap v137: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:40.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:40 smithi037 ceph-mon[31830]: pgmap v137: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:40.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:40 smithi053 ceph-mon[36169]: pgmap v137: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:42.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:42 smithi071 ceph-mon[36597]: pgmap v138: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:42.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:42 smithi037 ceph-mon[31830]: pgmap v138: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:42.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:42 smithi053 ceph-mon[36169]: pgmap v138: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:44.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:44 smithi071 ceph-mon[36597]: pgmap v139: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:44.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:44 smithi037 ceph-mon[31830]: pgmap v139: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:44.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:44 smithi053 ceph-mon[36169]: pgmap v139: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:46.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:46 smithi071 ceph-mon[36597]: pgmap v140: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:46.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:46 smithi037 ceph-mon[31830]: pgmap v140: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:46.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:46 smithi053 ceph-mon[36169]: pgmap v140: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:48 smithi071 ceph-mon[36597]: pgmap v141: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:48.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:48 smithi037 ceph-mon[31830]: pgmap v141: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:48.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:48 smithi053 ceph-mon[36169]: pgmap v141: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:50.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:50 smithi071 ceph-mon[36597]: pgmap v142: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:50.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:50 smithi037 ceph-mon[31830]: pgmap v142: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:50.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:50 smithi053 ceph-mon[36169]: pgmap v142: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:52.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:52 smithi071 ceph-mon[36597]: pgmap v143: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:52 smithi037 ceph-mon[31830]: pgmap v143: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:52.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:52 smithi053 ceph-mon[36169]: pgmap v143: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:54.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:54 smithi071 ceph-mon[36597]: pgmap v144: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:54.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:54 smithi071 ceph-mon[36597]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:16:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:54 smithi037 ceph-mon[31830]: pgmap v144: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:54 smithi037 ceph-mon[31830]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:16:54.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:54 smithi053 ceph-mon[36169]: pgmap v144: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:54.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:54 smithi053 ceph-mon[36169]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-21T09:16:55.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:55 smithi071 ceph-mon[36597]: pgmap v145: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:55.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:55 smithi037 ceph-mon[31830]: pgmap v145: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:55.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:55 smithi053 ceph-mon[36169]: pgmap v145: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:16:57.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:16:57.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:16:57.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 -- 172.21.15.37:0/3531236817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e810ba50 msgr2=0x7f42e810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 --2- 172.21.15.37:0/3531236817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e810ba50 0x7f42e810ded0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 -- 172.21.15.37:0/3531236817 shutdown_connections 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 --2- 172.21.15.37:0/3531236817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e810ba50 0x7f42e810ded0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 --2- 172.21.15.37:0/3531236817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e8102f50 0x7f42e810b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 --2- 172.21.15.37:0/3531236817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e8102630 0x7f42e8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 -- 172.21.15.37:0/3531236817 >> 172.21.15.37:0/3531236817 conn(0x7f42e8077130 msgr2=0x7f42e80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 -- 172.21.15.37:0/3531236817 shutdown_connections 2024-03-21T09:16:57.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.507+0000 7f42eda00640 1 -- 172.21.15.37:0/3531236817 wait complete. 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.508+0000 7f42eda00640 1 Processor -- start 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.508+0000 7f42eda00640 1 -- start start 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 0x7f42e8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42eda00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e8102f50 0x7f42e819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42eda00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e810ba50 0x7f42e819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42eda00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8111860 con 0x7f42e8102f50 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42eda00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81116e0 con 0x7f42e810ba50 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42eda00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81119e0 con 0x7f42e8102630 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 0x7f42e8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 0x7f42e8199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45868/0 (socket says 172.21.15.37:45868) 2024-03-21T09:16:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 -- 172.21.15.37:0/3841815443 learned_addr learned my addr 172.21.15.37:0/3841815443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7400640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e8102f50 0x7f42e819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42ed000640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e810ba50 0x7f42e819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 -- 172.21.15.37:0/3841815443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e810ba50 msgr2=0x7f42e819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e810ba50 0x7f42e819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 -- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e8102f50 msgr2=0x7f42e819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e8102f50 0x7f42e819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.509+0000 7f42e7e00640 1 -- 172.21.15.37:0/3841815443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e819eef0 con 0x7f42e8102630 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.510+0000 7f42e7e00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 0x7f42e8199f00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f42d800eb90 tx=0x7f42d800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.510+0000 7f42e4c00640 1 -- 172.21.15.37:0/3841815443 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8004d20 con 0x7f42e8102630 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.510+0000 7f42e4c00640 1 -- 172.21.15.37:0/3841815443 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d800ee90 con 0x7f42e8102630 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.510+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e819f1e0 con 0x7f42e8102630 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.510+0000 7f42e4c00640 1 -- 172.21.15.37:0/3841815443 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8010830 con 0x7f42e8102630 2024-03-21T09:16:57.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.510+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e81a6a70 con 0x7f42e8102630 2024-03-21T09:16:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.513+0000 7f42f1c29740 1 Processor -- start 2024-03-21T09:16:57.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 -- start start 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42e4c00640 1 -- 172.21.15.37:0/3841815443 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f42d8005020 con 0x7f42e8102630 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e84db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e84e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e859850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e7b2250 con 0x55f63e857460 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e75aa30 con 0x55f63e84e0f0 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42f1c29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e7ae2f0 con 0x55f63e84edd0 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42ed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e84db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e84e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42ed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e84db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45870/0 (socket says 172.21.15.37:45870) 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.514+0000 7f42ed000640 1 -- 172.21.15.37:0/2838793247 learned_addr learned my addr 172.21.15.37:0/2838793247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:57.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42e7400640 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e859850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42ed000640 1 -- 172.21.15.37:0/2838793247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 msgr2=0x55f63e84e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42ed000640 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e84e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42ed000640 1 -- 172.21.15.37:0/2838793247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 msgr2=0x55f63e859850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42ed000640 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e859850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42ed000640 1 -- 172.21.15.37:0/2838793247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63e745690 con 0x55f63e84edd0 2024-03-21T09:16:57.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42ed000640 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e84db20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f42e002c2d0 tx=0x7f42e0009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a366d64adb70d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42dd600640 1 -- 172.21.15.37:0/2838793247 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0043070 con 0x55f63e84edd0 2024-03-21T09:16:57.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42dd600640 1 -- 172.21.15.37:0/2838793247 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e002c910 con 0x55f63e84edd0 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42e4c00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42cc0a3090 0x7f42cc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.515+0000 7f42dd600640 1 -- 172.21.15.37:0/2838793247 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0004ae0 con 0x55f63e84edd0 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42e4c00640 1 -- 172.21.15.37:0/3841815443 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f42d80d01c0 con 0x7f42e8102630 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42e7e00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42cc0a3090 0x7f42cc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 -- 172.21.15.37:0/2838793247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 msgr2=0x55f63e84db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e84db20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f42e002c2d0 tx=0x7f42e0009610 comp rx=0 tx=0).stop 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 -- 172.21.15.37:0/2838793247 shutdown_connections 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e859850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e84e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 --2- 172.21.15.37:0/2838793247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e84db20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 -- 172.21.15.37:0/2838793247 >> 172.21.15.37:0/2838793247 conn(0x55f63e815200 msgr2=0x55f63e848b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42f1c29740 1 -- 172.21.15.37:0/2838793247 shutdown_connections 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.516+0000 7f42e7e00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42cc0a3090 0x7f42cc0a5550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42d80cd620 tx=0x7f42d80d3d10 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.517+0000 7f42f1c29740 1 -- 172.21.15.37:0/2838793247 wait complete. 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.517+0000 7f42f1c29740 1 -- start start 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.517+0000 7f42f1c29740 1 Processor -- start 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e853f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e854460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e85a270 con 0x55f63e857460 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e76de80 con 0x55f63e84e0f0 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63e85a3f0 con 0x55f63e84edd0 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42f1c29740 10 client.0 _ll_register_callbacks cb 0x55f63e780130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e853f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e853f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45874/0 (socket says 172.21.15.37:45874) 2024-03-21T09:16:57.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 -- 172.21.15.37:0/1815206059 learned_addr learned my addr 172.21.15.37:0/1815206059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42ed000640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42ed000640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e854460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 -- 172.21.15.37:0/1815206059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 msgr2=0x55f63e856b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 -- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 msgr2=0x55f63e854460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e854460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42e7400640 1 -- 172.21.15.37:0/1815206059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f63e5907d0 con 0x55f63e84edd0 2024-03-21T09:16:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.518+0000 7f42ed000640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e854460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42e7400640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e853f20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f42d000b940 tx=0x7f42d000be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35e0bbe1934f7cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42d6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42d6000640 1 -- 172.21.15.37:0/1815206059 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d001b640 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42d6000640 1 -- 172.21.15.37:0/1815206059 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f42d001b7e0 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42e002c910 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42d6000640 1 -- 172.21.15.37:0/1815206059 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42d0024050 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42f1c29740 10 client.26737 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f63e4c63f0 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.519+0000 7f42d6000640 1 -- 172.21.15.37:0/1815206059 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e002c910 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42d6000640 1 -- 172.21.15.37:0/1815206059 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f42d0019020 con 0x55f63e84edd0 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42f1c29740 10 client.26737 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42f1c29740 10 client.26737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42f1c29740 20 client.26737 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8673d0 0x55f63e8697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42f1c29740 4 client.26737 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f63e7bf790 con 0x55f63e8673d0 2024-03-21T09:16:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.520+0000 7f42ed000640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8673d0 0x55f63e8697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.521+0000 7f42ed000640 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8673d0 0x55f63e8697c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739bdc9cacf39991 server_cookie=c65d4de715d915bc in_seq=0 out_seq=0 2024-03-21T09:16:57.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.521+0000 7f42d6000640 10 client.26737 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.521+0000 7f42d6000640 1 -- 172.21.15.37:0/1815206059 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f63e7bf790 con 0x55f63e8673d0 2024-03-21T09:16:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.521+0000 7f42d6000640 10 client.26737 handle_command_reply: tid=0 2024-03-21T09:16:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.521+0000 7f42f1c29740 1 client.26737 shutdown 2024-03-21T09:16:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 msgr2=0x55f63e853f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e853f20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f42d000b940 tx=0x7f42d000be00 comp rx=0 tx=0).stop 2024-03-21T09:16:57.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 shutdown_connections 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e857460 0x55f63e854460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e84edd0 0x55f63e853f20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8673d0 0x55f63e8697c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1815206059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 >> 172.21.15.37:0/1815206059 conn(0x55f63e815200 msgr2=0x55f63e84a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42d6000640 0 client.26737 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 shutdown_connections 2024-03-21T09:16:57.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 1 -- 172.21.15.37:0/1815206059 wait complete. 2024-03-21T09:16:57.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.522+0000 7f42f1c29740 20 client.26737 trim_cache size 0 max 16384 2024-03-21T09:16:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 Processor -- start 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 -- start start 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84edd0 0x55f63e856b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e862b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81b2e60 con 0x55f63e84edd0 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81b0960 con 0x55f63e84e0f0 2024-03-21T09:16:57.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42f1c29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81aea00 con 0x55f63e857460 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42e7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84edd0 0x55f63e856b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35942/0 (socket says 172.21.15.37:35942) 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42ed000640 1 -- 172.21.15.37:0/1657501415 learned_addr learned my addr 172.21.15.37:0/1657501415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.540+0000 7f42ed000640 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e862b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42ed000640 1 -- 172.21.15.37:0/1657501415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 msgr2=0x55f63e862b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42ed000640 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e862b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42ed000640 1 -- 172.21.15.37:0/1657501415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84edd0 msgr2=0x55f63e856b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42ed000640 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84edd0 0x55f63e856b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42ed000640 1 -- 172.21.15.37:0/1657501415 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f63e4860f0 con 0x55f63e84e0f0 2024-03-21T09:16:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42ed000640 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006010 tx=0x7f42e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c14b6cb5789757 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42dcc00640 1 -- 172.21.15.37:0/1657501415 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0043040 con 0x55f63e84e0f0 2024-03-21T09:16:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42dcc00640 1 -- 172.21.15.37:0/1657501415 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e002c910 con 0x55f63e84e0f0 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.541+0000 7f42dcc00640 1 -- 172.21.15.37:0/1657501415 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0005140 con 0x55f63e84e0f0 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 -- 172.21.15.37:0/1657501415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 msgr2=0x55f63e856650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006010 tx=0x7f42e00046c0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 -- 172.21.15.37:0/1657501415 shutdown_connections 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e862b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84edd0 0x55f63e856b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 --2- 172.21.15.37:0/1657501415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84e0f0 0x55f63e856650 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 -- 172.21.15.37:0/1657501415 >> 172.21.15.37:0/1657501415 conn(0x55f63e815200 msgr2=0x55f63e853ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 -- 172.21.15.37:0/1657501415 shutdown_connections 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 -- 172.21.15.37:0/1657501415 wait complete. 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.542+0000 7f42f1c29740 1 -- start start 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 Processor -- start 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84e0f0 0x55f63e851420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 0x55f63e851960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e857e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81af000 con 0x55f63e84e0f0 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81b0300 con 0x55f63e84edd0 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81af640 con 0x55f63e857460 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42f1c29740 10 client.0 _ll_register_callbacks cb 0x7f42d0004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 0x55f63e851960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42e7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 0x55f63e851960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35950/0 (socket says 172.21.15.37:35950) 2024-03-21T09:16:57.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42e7400640 1 -- 172.21.15.37:0/351640274 learned_addr learned my addr 172.21.15.37:0/351640274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.543+0000 7f42ed000640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84e0f0 0x55f63e851420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42ed000640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e857e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42e7400640 1 -- 172.21.15.37:0/351640274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 msgr2=0x55f63e857e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42e7400640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e857e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42e7400640 1 -- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84e0f0 msgr2=0x55f63e851420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:57.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42e7400640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84e0f0 0x55f63e851420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42e7400640 1 -- 172.21.15.37:0/351640274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f63e8583a0 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42e7400640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 0x55f63e851960 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55f63e853780 tx=0x7f42d0004070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6953d9765abc0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42d6000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42d6000640 1 -- 172.21.15.37:0/351640274 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004450 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42e002c910 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.544+0000 7f42d6000640 1 -- 172.21.15.37:0/351640274 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f42d0011440 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42d6000640 1 -- 172.21.15.37:0/351640274 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42d0011640 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 10 client.26717 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f63e8c3410 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42d6000640 1 -- 172.21.15.37:0/351640274 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e002c910 con 0x55f63e84edd0 2024-03-21T09:16:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42d6000640 1 -- 172.21.15.37:0/351640274 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f42d0019050 con 0x55f63e84edd0 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 10 client.26717 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 20 client.26717 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8c3b10 0x55f63e8c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 4 client.26717 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f63e7b9ae0 con 0x55f63e8c3b10 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.545+0000 7f42ed000640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8c3b10 0x55f63e8c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.546+0000 7f42ed000640 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8c3b10 0x55f63e8c5f30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcf77d85c4dd6f5 server_cookie=9f083cbc2f7872a9 in_seq=0 out_seq=0 2024-03-21T09:16:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.546+0000 7f42d6000640 10 client.26717 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.546+0000 7f42d6000640 1 -- 172.21.15.37:0/351640274 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f63e7b9ae0 con 0x55f63e8c3b10 2024-03-21T09:16:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.546+0000 7f42d6000640 10 client.26717 handle_command_reply: tid=0 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.546+0000 7f42f1c29740 1 client.26717 shutdown 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 msgr2=0x55f63e851960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 0x55f63e851960 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55f63e853780 tx=0x7f42d0004070 comp rx=0 tx=0).stop 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 shutdown_connections 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f63e857460 0x55f63e857e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f63e8c3b10 0x55f63e8c5f30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f63e84edd0 0x55f63e851960 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 --2- 172.21.15.37:0/351640274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f63e84e0f0 0x55f63e851420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 >> 172.21.15.37:0/351640274 conn(0x55f63e815200 msgr2=0x55f63e852810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42d6000640 0 client.26717 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 shutdown_connections 2024-03-21T09:16:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 1 -- 172.21.15.37:0/351640274 wait complete. 2024-03-21T09:16:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.547+0000 7f42f1c29740 20 client.26717 trim_cache size 0 max 16384 2024-03-21T09:16:57.556 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:16:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:16:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:16:57.556 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:16:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42cc0a3090 msgr2=0x7f42cc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42cc0a3090 0x7f42cc0a5550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42d80cd620 tx=0x7f42d80d3d10 comp rx=0 tx=0).stop 2024-03-21T09:16:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 msgr2=0x7f42e8199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 0x7f42e8199f00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f42d800eb90 tx=0x7f42d800c6a0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 shutdown_connections 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f42cc0a3090 0x7f42cc0a5550 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42e810ba50 0x7f42e819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42e8102f50 0x7f42e819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 --2- 172.21.15.37:0/3841815443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e8102630 0x7f42e8199f00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 >> 172.21.15.37:0/3841815443 conn(0x7f42e8077130 msgr2=0x7f42e80fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 shutdown_connections 2024-03-21T09:16:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.548+0000 7f42eda00640 1 -- 172.21.15.37:0/3841815443 wait complete. 2024-03-21T09:16:57.561 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:16:57.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:16:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 -- 172.21.15.37:0/3421715777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128104100 msgr2=0x7f0128110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 --2- 172.21.15.37:0/3421715777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128104100 0x7f0128110610 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0120009c10 tx=0x7f012002bef0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 -- 172.21.15.37:0/3421715777 shutdown_connections 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 --2- 172.21.15.37:0/3421715777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128104100 0x7f0128110610 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 --2- 172.21.15.37:0/3421715777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f0128103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 --2- 172.21.15.37:0/3421715777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0128109750 0x7f0128109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 -- 172.21.15.37:0/3421715777 >> 172.21.15.37:0/3421715777 conn(0x7f01280ff210 msgr2=0x7f0128101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 -- 172.21.15.37:0/3421715777 shutdown_connections 2024-03-21T09:16:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.775+0000 7f012dc00640 1 -- 172.21.15.37:0/3421715777 wait complete. 2024-03-21T09:16:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.776+0000 7f012dc00640 1 Processor -- start 2024-03-21T09:16:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.776+0000 7f012dc00640 1 -- start start 2024-03-21T09:16:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f012819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0128104100 0x7f012819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128109750 0x7f012819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0128113bb0 con 0x7f0128104100 2024-03-21T09:16:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0128113a30 con 0x7f0128103740 2024-03-21T09:16:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0128113d30 con 0x7f0128109750 2024-03-21T09:16:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f0127e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f012819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f0127e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f012819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35960/0 (socket says 172.21.15.37:35960) 2024-03-21T09:16:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f0127e00640 1 -- 172.21.15.37:0/4022446253 learned_addr learned my addr 172.21.15.37:0/4022446253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012d200640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128109750 0x7f012819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f0127400640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0128104100 0x7f012819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012d200640 1 -- 172.21.15.37:0/4022446253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 msgr2=0x7f012819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012d200640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f012819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012d200640 1 -- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0128104100 msgr2=0x7f012819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012d200640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0128104100 0x7f012819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f012d200640 1 -- 172.21.15.37:0/4022446253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012819f020 con 0x7f0128109750 2024-03-21T09:16:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.777+0000 7f0127e00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f012819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.778+0000 7f012d200640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128109750 0x7f012819e8f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0120036040 tx=0x7f01200336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.778+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120043020 con 0x7f0128109750 2024-03-21T09:16:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.778+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0120003e50 con 0x7f0128109750 2024-03-21T09:16:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.778+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012003e6b0 con 0x7f0128109750 2024-03-21T09:16:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.778+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012819f2b0 con 0x7f0128109750 2024-03-21T09:16:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.778+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01281a6ba0 con 0x7f0128109750 2024-03-21T09:16:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.780+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128104e80 con 0x7f0128109750 2024-03-21T09:16:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.781+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f0120046020 con 0x7f0128109750 2024-03-21T09:16:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.782+0000 7f0124c00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f010c0a3080 0x7f010c0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.783+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f01200f1ff0 con 0x7f0128109750 2024-03-21T09:16:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.783+0000 7f0127e00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f010c0a3080 0x7f010c0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.784+0000 7f0127e00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f010c0a3080 0x7f010c0a5540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01180058c0 tx=0x7f0118009260 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:57.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:57.786+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f012004d020 con 0x7f0128109750 2024-03-21T09:16:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.030+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f012810bcb0 con 0x7f0128109750 2024-03-21T09:16:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.031+0000 7f0124c00640 1 -- 172.21.15.37:0/4022446253 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f01200be7f0 con 0x7f0128109750 2024-03-21T09:16:58.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:16:58.026 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:16:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.033+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f010c0a3080 msgr2=0x7f010c0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.033+0000 7f012dc00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f010c0a3080 0x7f010c0a5540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01180058c0 tx=0x7f0118009260 comp rx=0 tx=0).stop 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.033+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128109750 msgr2=0x7f012819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.033+0000 7f012dc00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128109750 0x7f012819e8f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0120036040 tx=0x7f01200336b0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 shutdown_connections 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0128109750 0x7f012819e8f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0128104100 0x7f012819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f010c0a3080 0x7f010c0a5540 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 --2- 172.21.15.37:0/4022446253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0128103740 0x7f012819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 >> 172.21.15.37:0/4022446253 conn(0x7f01280ff210 msgr2=0x7f01280ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 shutdown_connections 2024-03-21T09:16:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.034+0000 7f012dc00640 1 -- 172.21.15.37:0/4022446253 wait complete. 2024-03-21T09:16:58.049 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:16:58.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:16:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 -- 172.21.15.37:0/2674473479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c073ad0 msgr2=0x7f145c073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 --2- 172.21.15.37:0/2674473479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c073ad0 0x7f145c073eb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1450009900 tx=0x7f145002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 -- 172.21.15.37:0/2674473479 shutdown_connections 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 --2- 172.21.15.37:0/2674473479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c10dca0 0x7f145c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 --2- 172.21.15.37:0/2674473479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c0743f0 0x7f145c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 --2- 172.21.15.37:0/2674473479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c073ad0 0x7f145c073eb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 -- 172.21.15.37:0/2674473479 >> 172.21.15.37:0/2674473479 conn(0x7f145c0fcff0 msgr2=0x7f145c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 -- 172.21.15.37:0/2674473479 shutdown_connections 2024-03-21T09:16:58.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.269+0000 7f1462c00640 1 -- 172.21.15.37:0/2674473479 wait complete. 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 Processor -- start 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 -- start start 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 0x7f145c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c0743f0 0x7f145c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c10dca0 0x7f145c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c113a50 con 0x7f145c10dca0 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1138d0 con 0x7f145c073ad0 2024-03-21T09:16:58.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.271+0000 7f1462c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c113bd0 con 0x7f145c0743f0 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 0x7f145c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1460e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c0743f0 0x7f145c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 0x7f145c19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35968/0 (socket says 172.21.15.37:35968) 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 -- 172.21.15.37:0/1149012127 learned_addr learned my addr 172.21.15.37:0/1149012127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1462200640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c10dca0 0x7f145c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 -- 172.21.15.37:0/1149012127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c0743f0 msgr2=0x7f145c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c0743f0 0x7f145c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 -- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c10dca0 msgr2=0x7f145c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c10dca0 0x7f145c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 -- 172.21.15.37:0/1149012127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c19eff0 con 0x7f145c073ad0 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1462200640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c10dca0 0x7f145c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:58.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.272+0000 7f1461800640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 0x7f145c19a050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1450000c00 tx=0x7f1450004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.273+0000 7f145a000640 1 -- 172.21.15.37:0/1149012127 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450042020 con 0x7f145c073ad0 2024-03-21T09:16:58.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.273+0000 7f145a000640 1 -- 172.21.15.37:0/1149012127 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14500054e0 con 0x7f145c073ad0 2024-03-21T09:16:58.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.273+0000 7f145a000640 1 -- 172.21.15.37:0/1149012127 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450003af0 con 0x7f145c073ad0 2024-03-21T09:16:58.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.273+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c19f280 con 0x7f145c073ad0 2024-03-21T09:16:58.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.273+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c1a6b70 con 0x7f145c073ad0 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.276+0000 7f1466fdb740 1 Processor -- start 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.276+0000 7f1466fdb740 1 -- start start 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf32c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf33260 0x55a8bcf3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf3e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1466fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce962c0 con 0x55a8bcf33260 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1466fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce3ec90 con 0x55a8bcf328b0 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1466fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce92360 con 0x55a8bcf3bde0 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf32c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf32c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35982/0 (socket says 172.21.15.37:35982) 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 -- 172.21.15.37:0/1895600181 learned_addr learned my addr 172.21.15.37:0/1895600181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1460e00640 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf3e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 -- 172.21.15.37:0/1895600181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 msgr2=0x55a8bcf3e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf3e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 -- 172.21.15.37:0/1895600181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf33260 msgr2=0x55a8bcf3b810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf33260 0x55a8bcf3b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f1462200640 1 -- 172.21.15.37:0/1895600181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bce63080 con 0x55a8bcf328b0 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.277+0000 7f145a000640 1 -- 172.21.15.37:0/1149012127 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1450045020 con 0x7f145c073ad0 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.278+0000 7f1462200640 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf32c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1454009fd0 tx=0x7f1454010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30aa1eb99ee68356 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.278+0000 7f144ea00640 1 -- 172.21.15.37:0/1895600181 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454011700 con 0x55a8bcf328b0 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.278+0000 7f144ea00640 1 -- 172.21.15.37:0/1895600181 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1454011d80 con 0x55a8bcf328b0 2024-03-21T09:16:58.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.278+0000 7f144ea00640 1 -- 172.21.15.37:0/1895600181 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400e3c0 con 0x55a8bcf328b0 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 -- 172.21.15.37:0/1895600181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 msgr2=0x55a8bcf32c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf32c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1454009fd0 tx=0x7f1454010860 comp rx=0 tx=0).stop 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 -- 172.21.15.37:0/1895600181 shutdown_connections 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf3e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf33260 0x55a8bcf3b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1895600181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf32c90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 -- 172.21.15.37:0/1895600181 >> 172.21.15.37:0/1895600181 conn(0x55a8bcf2cf80 msgr2=0x55a8bcf2d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 -- 172.21.15.37:0/1895600181 shutdown_connections 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 -- 172.21.15.37:0/1895600181 wait complete. 2024-03-21T09:16:58.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.279+0000 7f1466fdb740 1 -- start start 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f145a000640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f14440a2fa0 0x7f14440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 Processor -- start 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f145a000640 1 -- 172.21.15.37:0/1149012127 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f14500f1760 con 0x7f145c073ad0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1462200640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f14440a2fa0 0x7f14440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf3b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf38b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcf3ed50 con 0x55a8bcf328b0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bce520d0 con 0x55a8bcf33260 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.280+0000 7f1466fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bcf3eed0 con 0x55a8bcf3bde0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1466fdb740 10 client.0 _ll_register_callbacks cb 0x55a8bce56e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf38b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1461800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf3b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf38b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35984/0 (socket says 172.21.15.37:35984) 2024-03-21T09:16:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 -- 172.21.15.37:0/4260256178 learned_addr learned my addr 172.21.15.37:0/4260256178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1462200640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f14440a2fa0 0x7f14440a5460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f145401bc90 tx=0x7f1454013040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1462200640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 -- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 msgr2=0x55a8bcf392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 -- 172.21.15.37:0/4260256178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 msgr2=0x55a8bcf3b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf3b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 -- 172.21.15.37:0/4260256178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8bcbaa3f0 con 0x55a8bcf33260 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1462200640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.281+0000 7f1460e00640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf38b70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f14480126e0 tx=0x7f1448012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20004480e32232c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1443400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1443400640 1 -- 172.21.15.37:0/4260256178 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480120d0 con 0x55a8bcf33260 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1454011d80 con 0x55a8bcf33260 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1443400640 1 -- 172.21.15.37:0/4260256178 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f144800f920 con 0x55a8bcf33260 2024-03-21T09:16:58.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1443400640 1 -- 172.21.15.37:0/4260256178 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1448012270 con 0x55a8bcf33260 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1466fdb740 10 client.26735 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a8bcc747d0 con 0x55a8bcf33260 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.282+0000 7f1443400640 1 -- 172.21.15.37:0/4260256178 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1454011d80 con 0x55a8bcf33260 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1443400640 1 -- 172.21.15.37:0/4260256178 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f144801c420 con 0x55a8bcf33260 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1466fdb740 10 client.26735 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1466fdb740 10 client.26735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1466fdb740 20 client.26735 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1466fdb740 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcf4bd10 0x55a8bcf4e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1466fdb740 4 client.26735 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8bcea3800 con 0x55a8bcf4bd10 2024-03-21T09:16:58.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.283+0000 7f1461800640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcf4bd10 0x55a8bcf4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1461800640 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcf4bd10 0x55a8bcf4e100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8201857ddbc21fac server_cookie=1d1b883080a68f07 in_seq=0 out_seq=0 2024-03-21T09:16:58.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1443400640 10 client.26735 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1443400640 1 -- 172.21.15.37:0/4260256178 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a8bcea3800 con 0x55a8bcf4bd10 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1443400640 10 client.26735 handle_command_reply: tid=0 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1466fdb740 1 client.26735 shutdown 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 msgr2=0x55a8bcf38b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.284+0000 7f1466fdb740 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf38b70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f14480126e0 tx=0x7f1448012ba0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 shutdown_connections 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf38b70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcf4bd10 0x55a8bcf4e100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 --2- 172.21.15.37:0/4260256178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf3b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1443400640 0 client.26735 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 >> 172.21.15.37:0/4260256178 conn(0x55a8bcf2cf80 msgr2=0x55a8bcf2ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 shutdown_connections 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 1 -- 172.21.15.37:0/4260256178 wait complete. 2024-03-21T09:16:58.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.285+0000 7f1466fdb740 20 client.26735 trim_cache size 0 max 16384 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 Processor -- start 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 -- start start 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf2edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf33260 0x55a8bcf38430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1b2fb0 con 0x55a8bcf3bde0 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1b0ab0 con 0x55a8bcf328b0 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1466fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1aeb50 con 0x55a8bcf33260 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1461800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf2edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1461800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf2edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35988/0 (socket says 172.21.15.37:35988) 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1461800640 1 -- 172.21.15.37:0/1978660678 learned_addr learned my addr 172.21.15.37:0/1978660678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.299+0000 7f1460e00640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf33260 0x55a8bcf38430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1462200640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1461800640 1 -- 172.21.15.37:0/1978660678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf33260 msgr2=0x55a8bcf38430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1461800640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf33260 0x55a8bcf38430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1461800640 1 -- 172.21.15.37:0/1978660678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf3bde0 msgr2=0x55a8bcf35db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1461800640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf35db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1461800640 1 -- 172.21.15.37:0/1978660678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bcf32490 con 0x55a8bcf328b0 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1462200640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1460e00640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf33260 0x55a8bcf38430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f1461800640 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf2edd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1450005ab0 tx=0x7f14500a8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5daa58953ecb282 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f144e000640 1 -- 172.21.15.37:0/1978660678 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500bd860 con 0x55a8bcf328b0 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f144e000640 1 -- 172.21.15.37:0/1978660678 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14500c19b0 con 0x55a8bcf328b0 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.300+0000 7f144e000640 1 -- 172.21.15.37:0/1978660678 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500a80b0 con 0x55a8bcf328b0 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 -- 172.21.15.37:0/1978660678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 msgr2=0x55a8bcf2edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf2edd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1450005ab0 tx=0x7f14500a8040 comp rx=0 tx=0).stop 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 -- 172.21.15.37:0/1978660678 shutdown_connections 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf33260 0x55a8bcf38430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1978660678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf328b0 0x55a8bcf2edd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 -- 172.21.15.37:0/1978660678 >> 172.21.15.37:0/1978660678 conn(0x55a8bcf2cf80 msgr2=0x55a8bcf3ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 -- 172.21.15.37:0/1978660678 shutdown_connections 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 -- 172.21.15.37:0/1978660678 wait complete. 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.301+0000 7f1466fdb740 1 -- start start 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 Processor -- start 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf43ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf42270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1af150 con 0x55a8bcf328b0 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1b0450 con 0x55a8bcf33260 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1af790 con 0x55a8bcf3bde0 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1466fdb740 10 client.0 _ll_register_callbacks cb 0x7f1448009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1461800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf43ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1462200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1461800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf43ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50174/0 (socket says 172.21.15.37:50174) 2024-03-21T09:16:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1461800640 1 -- 172.21.15.37:0/1676654445 learned_addr learned my addr 172.21.15.37:0/1676654445 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.302+0000 7f1460e00640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf42270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1462200640 1 -- 172.21.15.37:0/1676654445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 msgr2=0x55a8bcf42270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1462200640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf42270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1462200640 1 -- 172.21.15.37:0/1676654445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 msgr2=0x55a8bcf43ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1462200640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf43ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1462200640 1 -- 172.21.15.37:0/1676654445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8bcf42fe0 con 0x55a8bcf3bde0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1461800640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf43ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1460e00640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1462200640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf427b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1454005cc0 tx=0x7f1454005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bcec51b38d6274 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1443400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1443400640 1 -- 172.21.15.37:0/1676654445 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454038060 con 0x55a8bcf3bde0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14500c19b0 con 0x55a8bcf3bde0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.303+0000 7f1443400640 1 -- 172.21.15.37:0/1676654445 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f1454011910 con 0x55a8bcf3bde0 2024-03-21T09:16:58.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1443400640 1 -- 172.21.15.37:0/1676654445 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145403f050 con 0x55a8bcf3bde0 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 10 client.26767 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a8bcfa8610 con 0x55a8bcf3bde0 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1443400640 1 -- 172.21.15.37:0/1676654445 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14500c19b0 con 0x55a8bcf3bde0 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1443400640 1 -- 172.21.15.37:0/1676654445 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1454005730 con 0x55a8bcf3bde0 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 10 client.26767 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 10 client.26767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 20 client.26767 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcfa89b0 0x55a8bcfaada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 4 client.26767 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a8bce9db50 con 0x55a8bcfa89b0 2024-03-21T09:16:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.304+0000 7f1461800640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcfa89b0 0x55a8bcfaada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1461800640 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcfa89b0 0x55a8bcfaada0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d77d3a975f39d6f server_cookie=7cdcc8551257718a in_seq=0 out_seq=0 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1443400640 10 client.26767 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1443400640 1 -- 172.21.15.37:0/1676654445 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a8bce9db50 con 0x55a8bcfa89b0 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1443400640 10 client.26767 handle_command_reply: tid=0 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 client.26767 shutdown 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 msgr2=0x55a8bcf427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf427b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1454005cc0 tx=0x7f1454005fd0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 shutdown_connections 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a8bcf3bde0 0x55a8bcf427b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bcf33260 0x55a8bcf42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55a8bcfa89b0 0x55a8bcfaada0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 --2- 172.21.15.37:0/1676654445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8bcf328b0 0x55a8bcf43ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1443400640 0 client.26767 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 >> 172.21.15.37:0/1676654445 conn(0x55a8bcf2cf80 msgr2=0x55a8bcf3a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.305+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 shutdown_connections 2024-03-21T09:16:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1466fdb740 1 -- 172.21.15.37:0/1676654445 wait complete. 2024-03-21T09:16:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1466fdb740 20 client.26767 trim_cache size 0 max 16384 2024-03-21T09:16:58.306 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:16:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f14440a2fa0 msgr2=0x7f14440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f14440a2fa0 0x7f14440a5460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f145401bc90 tx=0x7f1454013040 comp rx=0 tx=0).stop 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 msgr2=0x7f145c19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 0x7f145c19a050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1450000c00 tx=0x7f1450004360 comp rx=0 tx=0).stop 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 shutdown_connections 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f145c10dca0 0x7f145c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c0743f0 0x7f145c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f14440a2fa0 0x7f14440a5460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 --2- 172.21.15.37:0/1149012127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f145c073ad0 0x7f145c19a050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 >> 172.21.15.37:0/1149012127 conn(0x7f145c0fcff0 msgr2=0x7f145c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.306+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 shutdown_connections 2024-03-21T09:16:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.307+0000 7f1462c00640 1 -- 172.21.15.37:0/1149012127 wait complete. 2024-03-21T09:16:58.319 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:58 smithi037 ceph-mon[31830]: pgmap v146: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:58.319 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:16:58 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4022446253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:16:58.322 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:16:58.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:16:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 -- 172.21.15.37:0/2608079276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c109870 msgr2=0x7fcf7c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 --2- 172.21.15.37:0/2608079276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c109870 0x7fcf7c109c50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009890 tx=0x7fcf6c02bb70 comp rx=0 tx=0).stop 2024-03-21T09:16:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 -- 172.21.15.37:0/2608079276 shutdown_connections 2024-03-21T09:16:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 --2- 172.21.15.37:0/2608079276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf7c104220 0x7fcf7c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 --2- 172.21.15.37:0/2608079276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf7c103860 0x7fcf7c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 --2- 172.21.15.37:0/2608079276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c109870 0x7fcf7c109c50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 -- 172.21.15.37:0/2608079276 >> 172.21.15.37:0/2608079276 conn(0x7fcf7c0ff310 msgr2=0x7fcf7c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 -- 172.21.15.37:0/2608079276 shutdown_connections 2024-03-21T09:16:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.539+0000 7fcf81600640 1 -- 172.21.15.37:0/2608079276 wait complete. 2024-03-21T09:16:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.540+0000 7fcf81600640 1 Processor -- start 2024-03-21T09:16:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.540+0000 7fcf81600640 1 -- start start 2024-03-21T09:16:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf81600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 0x7fcf7c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf81600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf7c104220 0x7fcf7c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf81600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf7c109870 0x7fcf7c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf81600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c113c40 con 0x7fcf7c109870 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf81600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c113ac0 con 0x7fcf7c104220 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf81600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c113dc0 con 0x7fcf7c103860 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 0x7fcf7c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 0x7fcf7c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45954/0 (socket says 172.21.15.37:45954) 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 -- 172.21.15.37:0/683091265 learned_addr learned my addr 172.21.15.37:0/683091265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7a000640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf7c104220 0x7fcf7c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7b400640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf7c109870 0x7fcf7c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 -- 172.21.15.37:0/683091265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf7c104220 msgr2=0x7fcf7c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf7c104220 0x7fcf7c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 -- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf7c109870 msgr2=0x7fcf7c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf7c109870 0x7fcf7c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.541+0000 7fcf7aa00640 1 -- 172.21.15.37:0/683091265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c19f170 con 0x7fcf7c103860 2024-03-21T09:16:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.542+0000 7fcf7aa00640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 0x7fcf7c19a180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0057c0 tx=0x7fcf6c0045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.542+0000 7fcf73400640 1 -- 172.21.15.37:0/683091265 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c041020 con 0x7fcf7c103860 2024-03-21T09:16:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.542+0000 7fcf73400640 1 -- 172.21.15.37:0/683091265 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c02c9f0 con 0x7fcf7c103860 2024-03-21T09:16:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.542+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c19f400 con 0x7fcf7c103860 2024-03-21T09:16:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.542+0000 7fcf73400640 1 -- 172.21.15.37:0/683091265 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c039040 con 0x7fcf7c103860 2024-03-21T09:16:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.542+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c1a6cf0 con 0x7fcf7c103860 2024-03-21T09:16:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.545+0000 7fcf73400640 1 -- 172.21.15.37:0/683091265 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcf6c044020 con 0x7fcf7c103860 2024-03-21T09:16:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.545+0000 7fcf843ea740 1 Processor -- start 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 -- start start 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c67bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c684630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c687050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5c5df2c0 con 0x558d5c684c00 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5c587c90 con 0x558d5c67c080 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf843ea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5c5db360 con 0x558d5c67b6d0 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c67bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c684630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c67bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45966/0 (socket says 172.21.15.37:45966) 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf7b400640 1 -- 172.21.15.37:0/3339434341 learned_addr learned my addr 172.21.15.37:0/3339434341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.546+0000 7fcf7a000640 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c687050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf7b400640 1 -- 172.21.15.37:0/3339434341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 msgr2=0x558d5c684630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c684630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf7b400640 1 -- 172.21.15.37:0/3339434341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 msgr2=0x558d5c687050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c687050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf7b400640 1 -- 172.21.15.37:0/3339434341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5c5ac080 con 0x558d5c67b6d0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c67bab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009f20 tx=0x7fcf74010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca9efb65ecb65f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf73400640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf600a3090 0x7fcf600a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf67400640 1 -- 172.21.15.37:0/3339434341 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74011700 con 0x558d5c67b6d0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf67400640 1 -- 172.21.15.37:0/3339434341 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf74011d80 con 0x558d5c67b6d0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf67400640 1 -- 172.21.15.37:0/3339434341 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7401a380 con 0x558d5c67b6d0 2024-03-21T09:16:58.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.547+0000 7fcf73400640 1 -- 172.21.15.37:0/683091265 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcf6c0f13b0 con 0x7fcf7c103860 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf7aa00640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf600a3090 0x7fcf600a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 -- 172.21.15.37:0/3339434341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 msgr2=0x558d5c67bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c67bab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009f20 tx=0x7fcf74010860 comp rx=0 tx=0).stop 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 -- 172.21.15.37:0/3339434341 shutdown_connections 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c687050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c684630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3339434341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c67bab0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 -- 172.21.15.37:0/3339434341 >> 172.21.15.37:0/3339434341 conn(0x558d5c675da0 msgr2=0x558d5c676180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 -- 172.21.15.37:0/3339434341 shutdown_connections 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf7aa00640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf600a3090 0x7fcf600a5550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0f4fd0 tx=0x7fcf6c0f4e60 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.548+0000 7fcf843ea740 1 -- 172.21.15.37:0/3339434341 wait complete. 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 -- start start 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 Processor -- start 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c6844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c681880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c681e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5c687b70 con 0x558d5c684c00 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5c59b0d0 con 0x558d5c67c080 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.549+0000 7fcf843ea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5c687cf0 con 0x558d5c67b6d0 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf843ea740 10 client.0 _ll_register_callbacks cb 0x558d5c59fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c681880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c681880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36022/0 (socket says 172.21.15.37:36022) 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 -- 172.21.15.37:0/3488394423 learned_addr learned my addr 172.21.15.37:0/3488394423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c6844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c681e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 -- 172.21.15.37:0/3488394423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 msgr2=0x558d5c6844f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c6844f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 -- 172.21.15.37:0/3488394423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 msgr2=0x558d5c681e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c681e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 -- 172.21.15.37:0/3488394423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d5c2f33f0 con 0x558d5c67c080 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c681e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf7a000640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c681880 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf68011670 tx=0x7fcf68011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c90dced02eedb1d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.550+0000 7fcf5be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf5be00640 1 -- 172.21.15.37:0/3488394423 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6801e020 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf5be00640 1 -- 172.21.15.37:0/3488394423 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcf6800fb30 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf74011d80 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf5be00640 1 -- 172.21.15.37:0/3488394423 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf68026050 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf843ea740 10 client.26753 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d5c3bd7d0 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf5be00640 1 -- 172.21.15.37:0/3488394423 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf74011d80 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.551+0000 7fcf5be00640 1 -- 172.21.15.37:0/3488394423 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcf68019070 con 0x558d5c67c080 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf843ea740 10 client.26753 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf843ea740 10 client.26753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf843ea740 20 client.26753 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c694ac0 0x558d5c696f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf843ea740 4 client.26753 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d5c5ec800 con 0x558d5c694ac0 2024-03-21T09:16:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c694ac0 0x558d5c696f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf7b400640 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c694ac0 0x558d5c696f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5876e14ae7a7e205 server_cookie=ff4e8484fa863576 in_seq=0 out_seq=0 2024-03-21T09:16:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.552+0000 7fcf5be00640 10 client.26753 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.553+0000 7fcf5be00640 1 -- 172.21.15.37:0/3488394423 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558d5c5ec800 con 0x558d5c694ac0 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.553+0000 7fcf5be00640 10 client.26753 handle_command_reply: tid=0 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.553+0000 7fcf843ea740 1 client.26753 shutdown 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.553+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 msgr2=0x558d5c681880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.553+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c681880 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf68011670 tx=0x7fcf68011b30 comp rx=0 tx=0).stop 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 shutdown_connections 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c694ac0 0x558d5c696f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c681e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c681880 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 --2- 172.21.15.37:0/3488394423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67b6d0 0x558d5c6844f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 >> 172.21.15.37:0/3488394423 conn(0x558d5c675da0 msgr2=0x558d5c677af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf5be00640 0 client.26753 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 shutdown_connections 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 1 -- 172.21.15.37:0/3488394423 wait complete. 2024-03-21T09:16:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.554+0000 7fcf843ea740 20 client.26753 trim_cache size 0 max 16384 2024-03-21T09:16:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.567+0000 7fcf843ea740 1 Processor -- start 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 -- start start 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c67b6d0 0x558d5c670d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c683fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c684c00 0x558d5c6844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1b30f0 con 0x558d5c67b6d0 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1b0bf0 con 0x558d5c67c080 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf843ea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1aec90 con 0x558d5c684c00 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c683fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c683fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36024/0 (socket says 172.21.15.37:36024) 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf7a000640 1 -- 172.21.15.37:0/1958184561 learned_addr learned my addr 172.21.15.37:0/1958184561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf7b400640 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c67b6d0 0x558d5c670d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.568+0000 7fcf7b400640 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c684c00 0x558d5c6844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7a000640 1 -- 172.21.15.37:0/1958184561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c684c00 msgr2=0x558d5c6844f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7a000640 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c684c00 0x558d5c6844f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7a000640 1 -- 172.21.15.37:0/1958184561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c67b6d0 msgr2=0x558d5c670d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7a000640 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c67b6d0 0x558d5c670d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7a000640 1 -- 172.21.15.37:0/1958184561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5c6819c0 con 0x558d5c67c080 2024-03-21T09:16:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7b400640 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c67b6d0 0x558d5c670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:16:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf7a000640 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c683fb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcf680194e0 tx=0x7fcf68005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f070b9302f5ce0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf66a00640 1 -- 172.21.15.37:0/1958184561 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6802f070 con 0x558d5c67c080 2024-03-21T09:16:58.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf66a00640 1 -- 172.21.15.37:0/1958184561 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf680199b0 con 0x558d5c67c080 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.569+0000 7fcf66a00640 1 -- 172.21.15.37:0/1958184561 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6801e040 con 0x558d5c67c080 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 -- 172.21.15.37:0/1958184561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 msgr2=0x558d5c683fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c683fb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcf680194e0 tx=0x7fcf68005a00 comp rx=0 tx=0).stop 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 -- 172.21.15.37:0/1958184561 shutdown_connections 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c684c00 0x558d5c6844f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67c080 0x558d5c683fb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1958184561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c67b6d0 0x558d5c670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 -- 172.21.15.37:0/1958184561 >> 172.21.15.37:0/1958184561 conn(0x558d5c675da0 msgr2=0x558d5c674620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 -- 172.21.15.37:0/1958184561 shutdown_connections 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 -- 172.21.15.37:0/1958184561 wait complete. 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.570+0000 7fcf843ea740 1 -- start start 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 Processor -- start 2024-03-21T09:16:58.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67b6d0 0x558d5c68a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 0x558d5c68ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c68b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1af290 con 0x558d5c684c00 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1b0590 con 0x558d5c67b6d0 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1af8d0 con 0x558d5c67c080 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf843ea740 10 client.0 _ll_register_callbacks cb 0x7fcf68009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 0x558d5c68ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf7b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67b6d0 0x558d5c68a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 0x558d5c68ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45996/0 (socket says 172.21.15.37:45996) 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf7a000640 1 -- 172.21.15.37:0/1244378700 learned_addr learned my addr 172.21.15.37:0/1244378700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.571+0000 7fcf7b400640 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c68b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf7a000640 1 -- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67b6d0 msgr2=0x558d5c68a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf7a000640 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67b6d0 0x558d5c68a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf7a000640 1 -- 172.21.15.37:0/1244378700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 msgr2=0x558d5c68b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf7a000640 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c68b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf7a000640 1 -- 172.21.15.37:0/1244378700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d5c6ef640 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf7a000640 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 0x558d5c68ab10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcf68011b70 tx=0x7fcf68019d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7baec98cb1ab30df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf5be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf5be00640 1 -- 172.21.15.37:0/1244378700 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6802f040 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf680199b0 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf5be00640 1 -- 172.21.15.37:0/1244378700 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcf6800fb30 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf5be00640 1 -- 172.21.15.37:0/1244378700 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6801e040 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf843ea740 10 client.26785 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.572+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558d5c6f0940 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf5be00640 1 -- 172.21.15.37:0/1244378700 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf680199b0 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf5be00640 1 -- 172.21.15.37:0/1244378700 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcf68004960 con 0x558d5c67c080 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf843ea740 10 client.26785 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf843ea740 10 client.26785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:16:58.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf843ea740 20 client.26785 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c6f0ef0 0x558d5c6f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf843ea740 4 client.26785 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d5c5e6b50 con 0x558d5c6f0ef0 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.573+0000 7fcf7b400640 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c6f0ef0 0x558d5c6f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf5be00640 10 client.26785 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf7b400640 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c6f0ef0 0x558d5c6f3310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd07d0a4c409dfd9 server_cookie=b725b04e3a1feb3f in_seq=0 out_seq=0 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf5be00640 1 -- 172.21.15.37:0/1244378700 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d5c5e6b50 con 0x558d5c6f0ef0 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf5be00640 10 client.26785 handle_command_reply: tid=0 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 client.26785 shutdown 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 msgr2=0x558d5c68ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 0x558d5c68ab10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcf68011b70 tx=0x7fcf68019d90 comp rx=0 tx=0).stop 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 shutdown_connections 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558d5c6f0ef0 0x558d5c6f3310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d5c684c00 0x558d5c68b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d5c67c080 0x558d5c68ab10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf5be00640 0 client.26785 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 --2- 172.21.15.37:0/1244378700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d5c67b6d0 0x558d5c68a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 >> 172.21.15.37:0/1244378700 conn(0x558d5c675da0 msgr2=0x558d5c677910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 shutdown_connections 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.574+0000 7fcf843ea740 1 -- 172.21.15.37:0/1244378700 wait complete. 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf843ea740 20 client.26785 trim_cache size 0 max 16384 2024-03-21T09:16:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf600a3090 msgr2=0x7fcf600a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf600a3090 0x7fcf600a5550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0f4fd0 tx=0x7fcf6c0f4e60 comp rx=0 tx=0).stop 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 msgr2=0x7fcf7c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 0x7fcf7c19a180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0057c0 tx=0x7fcf6c0045d0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 shutdown_connections 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf7c109870 0x7fcf7c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcf600a3090 0x7fcf600a5550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf7c104220 0x7fcf7c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 --2- 172.21.15.37:0/683091265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7c103860 0x7fcf7c19a180 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.575+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 >> 172.21.15.37:0/683091265 conn(0x7fcf7c0ff310 msgr2=0x7fcf7c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.576+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 shutdown_connections 2024-03-21T09:16:58.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.576+0000 7fcf81600640 1 -- 172.21.15.37:0/683091265 wait complete. 2024-03-21T09:16:58.588 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:16:58.589 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:16:58.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:58 smithi071 ceph-mon[36597]: pgmap v146: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:58.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:16:58 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4022446253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:16:58.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:58 smithi053 ceph-mon[36169]: pgmap v146: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:16:58.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:16:58 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4022446253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.804+0000 7f4a18c00640 1 -- 172.21.15.37:0/3220030283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a141028b0 msgr2=0x7f4a14102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.804+0000 7f4a18c00640 1 --2- 172.21.15.37:0/3220030283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a141028b0 0x7f4a14102c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009ca0 tx=0x7f4a0402c100 comp rx=0 tx=0).stop 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 -- 172.21.15.37:0/3220030283 shutdown_connections 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 --2- 172.21.15.37:0/3220030283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a14107da0 0x7f4a1410e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 --2- 172.21.15.37:0/3220030283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14103260 0x7f4a14107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 --2- 172.21.15.37:0/3220030283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a141028b0 0x7f4a14102c90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 -- 172.21.15.37:0/3220030283 >> 172.21.15.37:0/3220030283 conn(0x7f4a14077780 msgr2=0x7f4a14077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 -- 172.21.15.37:0/3220030283 shutdown_connections 2024-03-21T09:16:58.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.805+0000 7f4a18c00640 1 -- 172.21.15.37:0/3220030283 wait complete. 2024-03-21T09:16:58.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.806+0000 7f4a18c00640 1 Processor -- start 2024-03-21T09:16:58.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.806+0000 7f4a18c00640 1 -- start start 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a18c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 0x7f4a141980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a18c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14107da0 0x7f4a141985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a18c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1419c960 0x7f4a1419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a18c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14111ba0 con 0x7f4a141028b0 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a18c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14111a20 con 0x7f4a14107da0 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a18c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14111d20 con 0x7f4a1419c960 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 0x7f4a141980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a12800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1419c960 0x7f4a1419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a11400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14107da0 0x7f4a141985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 0x7f4a141980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50238/0 (socket says 172.21.15.37:50238) 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a11e00640 1 -- 172.21.15.37:0/803789975 learned_addr learned my addr 172.21.15.37:0/803789975 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a12800640 1 -- 172.21.15.37:0/803789975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14107da0 msgr2=0x7f4a141985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.807+0000 7f4a12800640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14107da0 0x7f4a141985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a12800640 1 -- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 msgr2=0x7f4a141980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a12800640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 0x7f4a141980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a12800640 1 -- 172.21.15.37:0/803789975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1419d450 con 0x7f4a1419c960 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a11e00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 0x7f4a141980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a12800640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1419c960 0x7f4a1419ce10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00d590 tx=0x7f4a0c00da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a0aa00640 1 -- 172.21.15.37:0/803789975 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c017070 con 0x7f4a1419c960 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a0aa00640 1 -- 172.21.15.37:0/803789975 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c005500 con 0x7f4a1419c960 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1419d740 con 0x7f4a1419c960 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a0aa00640 1 -- 172.21.15.37:0/803789975 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0096d0 con 0x7f4a1419c960 2024-03-21T09:16:58.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.808+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a141a4ee0 con 0x7f4a1419c960 2024-03-21T09:16:58.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 Processor -- start 2024-03-21T09:16:58.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a0aa00640 1 -- 172.21.15.37:0/803789975 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4a0c005020 con 0x7f4a1419c960 2024-03-21T09:16:58.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 -- start start 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb10950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb1e010 0x55999bb20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999ba742c0 con 0x55999bb10570 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999ba1cc90 con 0x55999bb1e010 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.813+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999ba70360 con 0x55999bb10f20 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb10950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46032/0 (socket says 172.21.15.37:46032) 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 -- 172.21.15.37:0/3404620445 learned_addr learned my addr 172.21.15.37:0/3404620445 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11400640 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb1e010 0x55999bb20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 -- 172.21.15.37:0/3404620445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb1e010 msgr2=0x55999bb20420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb1e010 0x55999bb20420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 -- 172.21.15.37:0/3404620445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 msgr2=0x55999bb10950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb10950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.814+0000 7f4a11e00640 1 -- 172.21.15.37:0/3404620445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999ba41080 con 0x55999bb10f20 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a0aa00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f49f80a3090 0x7f49f80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1da40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a04037040 tx=0x7f4a040098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f49c5ae6ba880f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a0aa00640 1 -- 172.21.15.37:0/803789975 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4a0c0cf640 con 0x7f4a1419c960 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a03000640 1 -- 172.21.15.37:0/3404620445 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04045020 con 0x55999bb10f20 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a03000640 1 -- 172.21.15.37:0/3404620445 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a0403a040 con 0x55999bb10f20 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a03000640 1 -- 172.21.15.37:0/3404620445 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04039040 con 0x55999bb10f20 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.815+0000 7f4a11400640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f49f80a3090 0x7f49f80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3404620445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 msgr2=0x55999bb1da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1da40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a04037040 tx=0x7f4a040098a0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a11400640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f49f80a3090 0x7f49f80a5550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009730 tx=0x7f49fc015040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3404620445 shutdown_connections 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb1e010 0x55999bb20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1da40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3404620445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb10950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3404620445 >> 172.21.15.37:0/3404620445 conn(0x55999bb0ada0 msgr2=0x55999bb0b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3404620445 shutdown_connections 2024-03-21T09:16:58.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.816+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3404620445 wait complete. 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.817+0000 7f4a1b85d740 1 -- start start 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.817+0000 7f4a1b85d740 1 Processor -- start 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 0x55999bb1ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb1e010 0x55999bb1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999bb20ee0 con 0x55999bb10570 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999ba300d0 con 0x55999bb10f20 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999bb21060 con 0x55999bb1e010 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a1b85d740 10 client.0 _ll_register_callbacks cb 0x55999ba34e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a11400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 0x55999bb1ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a11400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 0x55999bb1ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36080/0 (socket says 172.21.15.37:36080) 2024-03-21T09:16:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a11400640 1 -- 172.21.15.37:0/62720350 learned_addr learned my addr 172.21.15.37:0/62720350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.818+0000 7f4a12800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb1e010 0x55999bb1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a12800640 1 -- 172.21.15.37:0/62720350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 msgr2=0x55999bb1ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a12800640 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 0x55999bb1ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a12800640 1 -- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 msgr2=0x55999bb1d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a12800640 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb1d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a12800640 1 -- 172.21.15.37:0/62720350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55999b7883f0 con 0x55999bb1e010 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a11400640 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 0x55999bb1ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a12800640 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb1e010 0x55999bb1b2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0cc990 tx=0x7f4a0c09af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69539d603bdf1ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f49f7400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f49f7400640 1 -- 172.21.15.37:0/62720350 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c09b630 con 0x55999bb1e010 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a0403a040 con 0x55999bb1e010 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f49f7400640 1 -- 172.21.15.37:0/62720350 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4a0c0b4650 con 0x55999bb1e010 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f49f7400640 1 -- 172.21.15.37:0/62720350 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a0c09f780 con 0x55999bb1e010 2024-03-21T09:16:58.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a1b85d740 10 client.26797 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.819+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55999b8527d0 con 0x55999bb1e010 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f49f7400640 1 -- 172.21.15.37:0/62720350 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a0403a040 con 0x55999bb1e010 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f49f7400640 1 -- 172.21.15.37:0/62720350 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4a0c097ca0 con 0x55999bb1e010 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a1b85d740 10 client.26797 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a1b85d740 10 client.26797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a1b85d740 20 client.26797 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb2ddf0 0x55999bb30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a1b85d740 4 client.26797 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55999ba81800 con 0x55999bb2ddf0 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.820+0000 7f4a11e00640 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb2ddf0 0x55999bb30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.821+0000 7f4a11e00640 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb2ddf0 0x55999bb30220 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ac27c4341d1e6f server_cookie=fc36e0d0fcebc442 in_seq=0 out_seq=0 2024-03-21T09:16:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.821+0000 7f49f7400640 10 client.26797 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f49f7400640 1 -- 172.21.15.37:0/62720350 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55999ba81800 con 0x55999bb2ddf0 2024-03-21T09:16:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f49f7400640 10 client.26797 handle_command_reply: tid=0 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 client.26797 shutdown 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb1e010 msgr2=0x55999bb1b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb1e010 0x55999bb1b2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0cc990 tx=0x7f4a0c09af90 comp rx=0 tx=0).stop 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 shutdown_connections 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb1e010 0x55999bb1b2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb2ddf0 0x55999bb30220 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10f20 0x55999bb1ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/62720350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb10570 0x55999bb1d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f49f7400640 0 client.26797 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 >> 172.21.15.37:0/62720350 conn(0x55999bb0ada0 msgr2=0x55999bb0caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 shutdown_connections 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.822+0000 7f4a1b85d740 1 -- 172.21.15.37:0/62720350 wait complete. 2024-03-21T09:16:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.823+0000 7f4a1b85d740 20 client.26797 trim_cache size 0 max 16384 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.836+0000 7f4a1b85d740 1 Processor -- start 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.836+0000 7f4a1b85d740 1 -- start start 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb1d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141aed00 con 0x55999bb1e010 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141ac800 con 0x55999bb10570 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141aa8a0 con 0x55999bb10f20 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb1d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb1d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36086/0 (socket says 172.21.15.37:36086) 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 -- 172.21.15.37:0/3011676014 learned_addr learned my addr 172.21.15.37:0/3011676014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11400640 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a12800640 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 -- 172.21.15.37:0/3011676014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 msgr2=0x55999bb1d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 -- 172.21.15.37:0/3011676014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 msgr2=0x55999bb17f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb17f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.837+0000 7f4a11e00640 1 -- 172.21.15.37:0/3011676014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999bb0bfd0 con 0x55999bb10570 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb1d410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4a04006010 tx=0x7f4a04005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d60178b09fc04b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a02600640 1 -- 172.21.15.37:0/3011676014 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04039040 con 0x55999bb10570 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a02600640 1 -- 172.21.15.37:0/3011676014 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a040347d0 con 0x55999bb10570 2024-03-21T09:16:58.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a02600640 1 -- 172.21.15.37:0/3011676014 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04045040 con 0x55999bb10570 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3011676014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 msgr2=0x55999bb1d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb1d410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4a04006010 tx=0x7f4a04005d80 comp rx=0 tx=0).stop 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3011676014 shutdown_connections 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.838+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb17f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb1d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3011676014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb1d410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3011676014 >> 172.21.15.37:0/3011676014 conn(0x55999bb0ada0 msgr2=0x55999bb0ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3011676014 shutdown_connections 2024-03-21T09:16:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3011676014 wait complete. 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 -- start start 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.839+0000 7f4a1b85d740 1 Processor -- start 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb26060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb24400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb24940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141aaea0 con 0x55999bb1e010 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141ac1a0 con 0x55999bb10570 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141ab4e0 con 0x55999bb10f20 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a1b85d740 10 client.0 _ll_register_callbacks cb 0x7f4a0c097210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb26060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a11400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb24400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb26060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36102/0 (socket says 172.21.15.37:36102) 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a11e00640 1 -- 172.21.15.37:0/3374649370 learned_addr learned my addr 172.21.15.37:0/3374649370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.840+0000 7f4a12800640 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb24940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a11e00640 1 -- 172.21.15.37:0/3374649370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 msgr2=0x55999bb24400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb24400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a11e00640 1 -- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 msgr2=0x55999bb24940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb24940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a11e00640 1 -- 172.21.15.37:0/3374649370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55999bb25170 con 0x55999bb10570 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a11e00640 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb26060 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a04053040 tx=0x7f4a040346d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9829dcef6855650 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f49f7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f49f7400640 1 -- 172.21.15.37:0/3374649370 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04045030 con 0x55999bb10570 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f49f7400640 1 -- 172.21.15.37:0/3374649370 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4a04032c90 con 0x55999bb10570 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a040347d0 con 0x55999bb10570 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f49f7400640 1 -- 172.21.15.37:0/3374649370 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a04064050 con 0x55999bb10570 2024-03-21T09:16:58.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a1b85d740 10 client.26777 fetch_fsmap learned FSMap version 30 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.841+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55999bb8a6b0 con 0x55999bb10570 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f49f7400640 1 -- 172.21.15.37:0/3374649370 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a040347d0 con 0x55999bb10570 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f49f7400640 1 -- 172.21.15.37:0/3374649370 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4a04004040 con 0x55999bb10570 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a1b85d740 10 client.26777 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a1b85d740 10 client.26777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a1b85d740 20 client.26777 populate_metadata read hostname 'smithi037' 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb8a930 0x55999bb8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a1b85d740 4 client.26777 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55999ba7bb50 con 0x55999bb8a930 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.842+0000 7f4a11400640 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb8a930 0x55999bb8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a11400640 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb8a930 0x55999bb8cd20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7231195c222d25f8 server_cookie=917d477b29ccfb55 in_seq=0 out_seq=0 2024-03-21T09:16:58.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f49f7400640 10 client.26777 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f49f7400640 1 -- 172.21.15.37:0/3374649370 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55999ba7bb50 con 0x55999bb8a930 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f49f7400640 10 client.26777 handle_command_reply: tid=0 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 client.26777 shutdown 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 msgr2=0x55999bb26060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb26060 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a04053040 tx=0x7f4a040346d0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 shutdown_connections 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bb1e010 0x55999bb24940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55999bb10f20 0x55999bb24400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55999bb8a930 0x55999bb8cd20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 --2- 172.21.15.37:0/3374649370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55999bb10570 0x55999bb26060 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f49f7400640 0 client.26777 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:16:58.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.843+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 >> 172.21.15.37:0/3374649370 conn(0x55999bb0ada0 msgr2=0x55999bb0c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 shutdown_connections 2024-03-21T09:16:58.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a1b85d740 1 -- 172.21.15.37:0/3374649370 wait complete. 2024-03-21T09:16:58.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a1b85d740 20 client.26777 trim_cache size 0 max 16384 2024-03-21T09:16:58.844 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f49f80a3090 msgr2=0x7f49f80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f49f80a3090 0x7f49f80a5550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009730 tx=0x7f49fc015040 comp rx=0 tx=0).stop 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1419c960 msgr2=0x7f4a1419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1419c960 0x7f4a1419ce10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00d590 tx=0x7f4a0c00da50 comp rx=0 tx=0).stop 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 shutdown_connections 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1419c960 0x7f4a1419ce10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f49f80a3090 0x7f49f80a5550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14107da0 0x7f4a141985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 --2- 172.21.15.37:0/803789975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a141028b0 0x7f4a141980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.844+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 >> 172.21.15.37:0/803789975 conn(0x7f4a14077780 msgr2=0x7f4a140ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:16:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.845+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 shutdown_connections 2024-03-21T09:16:58.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:16:58.845+0000 7f4a18c00640 1 -- 172.21.15.37:0/803789975 wait complete. 2024-03-21T09:16:59.858 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:16:59.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:16:59.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:17:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 -- 172.21.15.37:0/906454056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7000743f0 msgr2=0x7fa70010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 --2- 172.21.15.37:0/906454056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7000743f0 0x7fa70010d7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009860 tx=0x7fa6ec02bc00 comp rx=0 tx=0).stop 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 -- 172.21.15.37:0/906454056 shutdown_connections 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 --2- 172.21.15.37:0/906454056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa70010dce0 0x7fa700110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 --2- 172.21.15.37:0/906454056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7000743f0 0x7fa70010d7a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 --2- 172.21.15.37:0/906454056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 0x7fa700073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.119+0000 7fa707000640 1 -- 172.21.15.37:0/906454056 >> 172.21.15.37:0/906454056 conn(0x7fa7000fcfb0 msgr2=0x7fa7000ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.120+0000 7fa707000640 1 -- 172.21.15.37:0/906454056 shutdown_connections 2024-03-21T09:17:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.120+0000 7fa707000640 1 -- 172.21.15.37:0/906454056 wait complete. 2024-03-21T09:17:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 Processor -- start 2024-03-21T09:17:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 -- start start 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 0x7fa700199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7000743f0 0x7fa70019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70010dce0 0x7fa70019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700113a80 con 0x7fa700073ad0 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700113900 con 0x7fa70010dce0 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.121+0000 7fa707000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700113c00 con 0x7fa7000743f0 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 0x7fa700199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa706600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70010dce0 0x7fa70019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 0x7fa700199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50300/0 (socket says 172.21.15.37:50300) 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7000743f0 0x7fa70019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705c00640 1 -- 172.21.15.37:0/3184334873 learned_addr learned my addr 172.21.15.37:0/3184334873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 -- 172.21.15.37:0/3184334873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70010dce0 msgr2=0x7fa70019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70010dce0 0x7fa70019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 -- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 msgr2=0x7fa700199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 0x7fa700199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 -- 172.21.15.37:0/3184334873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70019ef80 con 0x7fa7000743f0 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa706600640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70010dce0 0x7fa70019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.122+0000 7fa705200640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7000743f0 0x7fa70019a520 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec02c140 tx=0x7fa6ec0344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.123+0000 7fa6fe000640 1 -- 172.21.15.37:0/3184334873 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec041020 con 0x7fa7000743f0 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.123+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70019f210 con 0x7fa7000743f0 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.123+0000 7fa6fe000640 1 -- 172.21.15.37:0/3184334873 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec02ca80 con 0x7fa7000743f0 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.123+0000 7fa6fe000640 1 -- 172.21.15.37:0/3184334873 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec039040 con 0x7fa7000743f0 2024-03-21T09:17:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.123+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7001a6b00 con 0x7fa7000743f0 2024-03-21T09:17:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa6fe000640 1 -- 172.21.15.37:0/3184334873 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa6ec044020 con 0x7fa7000743f0 2024-03-21T09:17:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 Processor -- start 2024-03-21T09:17:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 -- start start 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2feb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e64d257eb0 con 0x55e64d2f3bb0 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e64d200a60 con 0x55e64d2f3290 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.127+0000 7fa70b39b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e64d253f50 con 0x55e64d2fc790 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2f3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36126/0 (socket says 172.21.15.37:36126) 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 -- 172.21.15.37:0/726568819 learned_addr learned my addr 172.21.15.37:0/726568819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa705200640 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa705c00640 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 -- 172.21.15.37:0/726568819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 msgr2=0x55e64d2feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2feb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 -- 172.21.15.37:0/726568819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 msgr2=0x55e64d2fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 -- 172.21.15.37:0/726568819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e64d0347e0 con 0x55e64d2f3290 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa705200640 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.128+0000 7fa706600640 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2f3670 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8011790 tx=0x7fa6f8011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e2bb9db78f84a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa6fe000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6e80a3090 0x7fa6e80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa6f2a00640 1 -- 172.21.15.37:0/726568819 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8003c30 con 0x55e64d2f3290 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa6f2a00640 1 -- 172.21.15.37:0/726568819 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6f800fe80 con 0x55e64d2f3290 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa6f2a00640 1 -- 172.21.15.37:0/726568819 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80193e0 con 0x55e64d2f3290 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa6fe000640 1 -- 172.21.15.37:0/3184334873 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa6ec0f1090 con 0x7fa7000743f0 2024-03-21T09:17:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa705c00640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6e80a3090 0x7fa6e80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa70b39b740 1 -- 172.21.15.37:0/726568819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 msgr2=0x55e64d2f3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.129+0000 7fa70b39b740 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2f3670 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8011790 tx=0x7fa6f8011c50 comp rx=0 tx=0).stop 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 -- 172.21.15.37:0/726568819 shutdown_connections 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 --2- 172.21.15.37:0/726568819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2f3670 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 -- 172.21.15.37:0/726568819 >> 172.21.15.37:0/726568819 conn(0x55e64d2ed9c0 msgr2=0x55e64d2edda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 -- 172.21.15.37:0/726568819 shutdown_connections 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa705c00640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6e80a3090 0x7fa6e80a5550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55e64d2f2d60 tx=0x7fa6f4009260 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 -- 172.21.15.37:0/726568819 wait complete. 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 -- start start 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.130+0000 7fa70b39b740 1 Processor -- start 2024-03-21T09:17:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e64d2ff580 con 0x55e64d2f3bb0 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e64d213eb0 con 0x55e64d2f3290 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e64d2ff700 con 0x55e64d2fc790 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa70b39b740 10 client.0 _ll_register_callbacks cb 0x55e64d208150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa705c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa705c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2fc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36128/0 (socket says 172.21.15.37:36128) 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa705c00640 1 -- 172.21.15.37:0/2241388485 learned_addr learned my addr 172.21.15.37:0/2241388485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa706600640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.131+0000 7fa705200640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705c00640 1 -- 172.21.15.37:0/2241388485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 msgr2=0x55e64d2f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705c00640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705c00640 1 -- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 msgr2=0x55e64d2f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705c00640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705c00640 1 -- 172.21.15.37:0/2241388485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e64d094f70 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa706600640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705200640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa705c00640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2fc120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa6f401d9c0 tx=0x7fa6f4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4178d2d3c0d1bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa6e7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa6e7400640 1 -- 172.21.15.37:0/2241388485 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f402d060 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa6e7400640 1 -- 172.21.15.37:0/2241388485 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa6f4018030 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e64d2653f0 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.132+0000 7fa6e7400640 1 -- 172.21.15.37:0/2241388485 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f4034050 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 10 client.26789 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e64cf2c170 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa6e7400640 1 -- 172.21.15.37:0/2241388485 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e64d2653f0 con 0x55e64d2f3290 2024-03-21T09:17:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa6e7400640 1 -- 172.21.15.37:0/2241388485 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa6f401c4d0 con 0x55e64d2f3290 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 10 client.26789 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 20 client.26789 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d30c8b0 0x55e64d30eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 4 client.26789 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e64d25f740 con 0x55e64d30c8b0 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.133+0000 7fa705200640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d30c8b0 0x55e64d30eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.134+0000 7fa705200640 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d30c8b0 0x55e64d30eca0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b58cbacf81447 server_cookie=7896d36a09ca1e94 in_seq=0 out_seq=0 2024-03-21T09:17:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.134+0000 7fa6e7400640 10 client.26789 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa6e7400640 1 -- 172.21.15.37:0/2241388485 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e64d25f740 con 0x55e64d30c8b0 2024-03-21T09:17:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa6e7400640 10 client.26789 handle_command_reply: tid=0 2024-03-21T09:17:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 client.26789 shutdown 2024-03-21T09:17:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 msgr2=0x55e64d2fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2fc120 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa6f401d9c0 tx=0x7fa6f4004730 comp rx=0 tx=0).stop 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 shutdown_connections 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3bb0 0x55e64d2f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d30c8b0 0x55e64d30eca0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 --2- 172.21.15.37:0/2241388485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d2fc120 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa6e7400640 0 client.26789 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 >> 172.21.15.37:0/2241388485 conn(0x55e64d2ed9c0 msgr2=0x55e64d2ef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 shutdown_connections 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 1 -- 172.21.15.37:0/2241388485 wait complete. 2024-03-21T09:17:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.135+0000 7fa70b39b740 20 client.26789 trim_cache size 0 max 16384 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.153+0000 7fa70b39b740 1 Processor -- start 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.153+0000 7fa70b39b740 1 -- start start 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3290 0x55e64d2fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa70b39b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001b2ea0 con 0x55e64d2f3290 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa70b39b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001b09a0 con 0x55e64d2f3bb0 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa70b39b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001aea40 con 0x55e64d2fc790 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3290 0x55e64d2fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36130/0 (socket says 172.21.15.37:36130) 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa706600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 -- 172.21.15.37:0/227303485 learned_addr learned my addr 172.21.15.37:0/227303485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 -- 172.21.15.37:0/227303485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 msgr2=0x55e64d2f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 -- 172.21.15.37:0/227303485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3290 msgr2=0x55e64d2fbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3290 0x55e64d2fbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.154+0000 7fa705200640 1 -- 172.21.15.37:0/227303485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e64d2f9da0 con 0x55e64d2f3bb0 2024-03-21T09:17:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa705200640 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec02c210 tx=0x7fa6ec0dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5964d9a7f28944db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa6f2000640 1 -- 172.21.15.37:0/227303485 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0bcb50 con 0x55e64d2f3bb0 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa6f2000640 1 -- 172.21.15.37:0/227303485 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0bd8f0 con 0x55e64d2f3bb0 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa6f2000640 1 -- 172.21.15.37:0/227303485 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0ee6c0 con 0x55e64d2f3bb0 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 -- 172.21.15.37:0/227303485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 msgr2=0x55e64d2fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec02c210 tx=0x7fa6ec0dd040 comp rx=0 tx=0).stop 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 -- 172.21.15.37:0/227303485 shutdown_connections 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2fc790 0x55e64d2f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3bb0 0x55e64d2fc120 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 --2- 172.21.15.37:0/227303485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2f3290 0x55e64d2fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.155+0000 7fa70b39b740 1 -- 172.21.15.37:0/227303485 >> 172.21.15.37:0/227303485 conn(0x55e64d2ed9c0 msgr2=0x55e64d2f40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.156+0000 7fa70b39b740 1 -- 172.21.15.37:0/227303485 shutdown_connections 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.156+0000 7fa70b39b740 1 -- 172.21.15.37:0/227303485 wait complete. 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.156+0000 7fa70b39b740 1 -- start start 2024-03-21T09:17:00.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.156+0000 7fa70b39b740 1 Processor -- start 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d301920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 0x55e64d301e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2fc790 0x55e64d3023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001af040 con 0x55e64d2fc790 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001b0340 con 0x55e64d2f3290 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001af680 con 0x55e64d2f3bb0 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa70b39b740 10 client.0 _ll_register_callbacks cb 0x7fa6f401d880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 0x55e64d301e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d301920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 0x55e64d301e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46096/0 (socket says 172.21.15.37:46096) 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705200640 1 -- 172.21.15.37:0/566664208 learned_addr learned my addr 172.21.15.37:0/566664208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa706600640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2fc790 0x55e64d3023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705c00640 1 -- 172.21.15.37:0/566664208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 msgr2=0x55e64d301e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705c00640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 0x55e64d301e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705c00640 1 -- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2fc790 msgr2=0x55e64d3023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705c00640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2fc790 0x55e64d3023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705c00640 1 -- 172.21.15.37:0/566664208 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e64d2feef0 con 0x55e64d2f3290 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa705200640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 0x55e64d301e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.157+0000 7fa706600640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2fc790 0x55e64d3023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:17:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa705c00640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d301920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4019040 tx=0x7fa6f4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc627f6cdc01f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa6e7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa6e7400640 1 -- 172.21.15.37:0/566664208 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f403c070 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa6e7400640 1 -- 172.21.15.37:0/566664208 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa6f401c5b0 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6ec0bd8f0 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa6e7400640 1 -- 172.21.15.37:0/566664208 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f4048050 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa70b39b740 10 client.26795 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.158+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e64d3690e0 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa6e7400640 1 -- 172.21.15.37:0/566664208 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0bd8f0 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa6e7400640 1 -- 172.21.15.37:0/566664208 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa6f4041c20 con 0x55e64d2f3290 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa70b39b740 10 client.26795 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa70b39b740 10 client.26795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa70b39b740 20 client.26795 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa70b39b740 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d369750 0x55e64d36bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa70b39b740 4 client.26795 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:17:00.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e64d1dda40 con 0x55e64d369750 2024-03-21T09:17:00.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.159+0000 7fa705200640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d369750 0x55e64d36bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.160+0000 7fa705200640 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d369750 0x55e64d36bb40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b78a84e0f02eb27 server_cookie=3b93c602570b44a2 in_seq=0 out_seq=0 2024-03-21T09:17:00.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.160+0000 7fa6e7400640 10 client.26795 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa6e7400640 1 -- 172.21.15.37:0/566664208 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e64d1dda40 con 0x55e64d369750 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa6e7400640 10 client.26795 handle_command_reply: tid=0 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 client.26795 shutdown 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 msgr2=0x55e64d301920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d301920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4019040 tx=0x7fa6f4005e80 comp rx=0 tx=0).stop 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 shutdown_connections 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e64d2fc790 0x55e64d3023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e64d2f3bb0 0x55e64d301e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55e64d369750 0x55e64d36bb40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 --2- 172.21.15.37:0/566664208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e64d2f3290 0x55e64d301920 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 >> 172.21.15.37:0/566664208 conn(0x55e64d2ed9c0 msgr2=0x55e64d2f4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.161+0000 7fa6e7400640 0 client.26795 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.162+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 shutdown_connections 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.162+0000 7fa70b39b740 1 -- 172.21.15.37:0/566664208 wait complete. 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.162+0000 7fa70b39b740 20 client.26795 trim_cache size 0 max 16384 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:17:00.160 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:17:00.161 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "8bf8ca6f-1fd3-491d-84ea-57b857f4d8e3", 2024-03-21T09:17:00.161 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:17:00.161 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:17:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6e80a3090 msgr2=0x7fa6e80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6e80a3090 0x7fa6e80a5550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55e64d2f2d60 tx=0x7fa6f4009260 comp rx=0 tx=0).stop 2024-03-21T09:17:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7000743f0 msgr2=0x7fa70019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7000743f0 0x7fa70019a520 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec02c140 tx=0x7fa6ec0344c0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 shutdown_connections 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6e80a3090 0x7fa6e80a5550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70010dce0 0x7fa70019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7000743f0 0x7fa70019a520 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 --2- 172.21.15.37:0/3184334873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa700073ad0 0x7fa700199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.163+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 >> 172.21.15.37:0/3184334873 conn(0x7fa7000fcfb0 msgr2=0x7fa7000fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.164+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 shutdown_connections 2024-03-21T09:17:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.164+0000 7fa707000640 1 -- 172.21.15.37:0/3184334873 wait complete. 2024-03-21T09:17:00.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:17:00.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:17:00.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 -- 172.21.15.37:0/3567554548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 msgr2=0x7f359c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 --2- 172.21.15.37:0/3567554548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 0x7f359c106a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f358c009930 tx=0x7f358c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:17:00.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 -- 172.21.15.37:0/3567554548 shutdown_connections 2024-03-21T09:17:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 --2- 172.21.15.37:0/3567554548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c10d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 --2- 172.21.15.37:0/3567554548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c100560 0x7f359c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 --2- 172.21.15.37:0/3567554548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 0x7f359c106a00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 -- 172.21.15.37:0/3567554548 >> 172.21.15.37:0/3567554548 conn(0x7f359c077750 msgr2=0x7f359c077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 -- 172.21.15.37:0/3567554548 shutdown_connections 2024-03-21T09:17:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.401+0000 7f35a1a00640 1 -- 172.21.15.37:0/3567554548 wait complete. 2024-03-21T09:17:00.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 Processor -- start 2024-03-21T09:17:00.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 -- start start 2024-03-21T09:17:00.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c100560 0x7f359c1021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 0x7f359c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c110930 con 0x7f359c100560 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1107b0 con 0x7f359c100f20 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.402+0000 7f35a1a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c110ab0 con 0x7f359c106620 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c100560 0x7f359c1021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36144/0 (socket says 172.21.15.37:36144) 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 -- 172.21.15.37:0/1385241968 learned_addr learned my addr 172.21.15.37:0/1385241968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 0x7f359c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 -- 172.21.15.37:0/1385241968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 msgr2=0x7f359c102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 0x7f359c102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 -- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c100560 msgr2=0x7f359c1021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c100560 0x7f359c1021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 -- 172.21.15.37:0/1385241968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c1a6410 con 0x7f359c100f20 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.403+0000 7f359a000640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c102710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f358800e9f0 tx=0x7f358800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.404+0000 7f3593400640 1 -- 172.21.15.37:0/1385241968 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358800cd80 con 0x7f359c100f20 2024-03-21T09:17:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.404+0000 7f3593400640 1 -- 172.21.15.37:0/1385241968 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3588010910 con 0x7f359c100f20 2024-03-21T09:17:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.404+0000 7f3593400640 1 -- 172.21.15.37:0/1385241968 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588018990 con 0x7f359c100f20 2024-03-21T09:17:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.404+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c1a6700 con 0x7f359c100f20 2024-03-21T09:17:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.404+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c1a6ca0 con 0x7f359c100f20 2024-03-21T09:17:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.407+0000 7f35a479a740 1 Processor -- start 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.407+0000 7f35a479a740 1 -- start start 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.407+0000 7f3593400640 1 -- 172.21.15.37:0/1385241968 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3588010ac0 con 0x7f359c100f20 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f35a479a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cd3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f35a479a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f35a479a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f35a479a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea06c38250 con 0x55ea06cd4c00 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f35a479a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea06be0a30 con 0x55ea06cd3f20 2024-03-21T09:17:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f35a479a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea06c342f0 con 0x55ea06cdd290 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f359b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cd3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f359a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f359aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f359aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36148/0 (socket says 172.21.15.37:36148) 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f359aa00640 1 -- 172.21.15.37:0/57376053 learned_addr learned my addr 172.21.15.37:0/57376053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.408+0000 7f359a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46132/0 (socket says 172.21.15.37:46132) 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359b400640 1 -- 172.21.15.37:0/57376053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 msgr2=0x55ea06cdf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359b400640 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359b400640 1 -- 172.21.15.37:0/57376053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 msgr2=0x55ea06cd43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359b400640 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359b400640 1 -- 172.21.15.37:0/57376053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea06bcb690 con 0x55ea06cd4c00 2024-03-21T09:17:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359aa00640 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f359b400640 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cd3950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3594009ef0 tx=0x7f3594011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7144397ca7250fea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f3593400640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f35800a3110 0x7f35800a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f3587e00640 1 -- 172.21.15.37:0/57376053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594004b20 con 0x55ea06cd4c00 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.409+0000 7f3587e00640 1 -- 172.21.15.37:0/57376053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3594004cc0 con 0x55ea06cd4c00 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f3587e00640 1 -- 172.21.15.37:0/57376053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359400e7e0 con 0x55ea06cd4c00 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f3593400640 1 -- 172.21.15.37:0/1385241968 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f35880cfe70 con 0x7f359c100f20 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f359aa00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f35800a3110 0x7f35800a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 -- 172.21.15.37:0/57376053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 msgr2=0x55ea06cd3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cd3950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3594009ef0 tx=0x7f3594011810 comp rx=0 tx=0).stop 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 -- 172.21.15.37:0/57376053 shutdown_connections 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 --2- 172.21.15.37:0/57376053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cd3950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.410+0000 7f35a479a740 1 -- 172.21.15.37:0/57376053 >> 172.21.15.37:0/57376053 conn(0x55ea06c9b200 msgr2=0x55ea06cce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.411+0000 7f35a479a740 1 -- 172.21.15.37:0/57376053 shutdown_connections 2024-03-21T09:17:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.411+0000 7f35a479a740 1 -- 172.21.15.37:0/57376053 wait complete. 2024-03-21T09:17:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.411+0000 7f359aa00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f35800a3110 0x7f35800a55d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f358c0096c0 tx=0x7f358c02df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.411+0000 7f35a479a740 1 -- start start 2024-03-21T09:17:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 Processor -- start 2024-03-21T09:17:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd3f20 0x55ea06cdc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 0x55ea06cd9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06cda1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea06ce0100 con 0x55ea06cd3f20 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea06bf3e80 con 0x55ea06cdd290 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea06ce0280 con 0x55ea06cd4c00 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.412+0000 7f35a479a740 10 client.0 _ll_register_callbacks cb 0x55ea06c06130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 0x55ea06cd9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 0x55ea06cd9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46146/0 (socket says 172.21.15.37:46146) 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06cda1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359a000640 1 -- 172.21.15.37:0/505340900 learned_addr learned my addr 172.21.15.37:0/505340900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359aa00640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd3f20 0x55ea06cdc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 -- 172.21.15.37:0/505340900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 msgr2=0x55ea06cd9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 0x55ea06cd9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 -- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd3f20 msgr2=0x55ea06cdc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd3f20 0x55ea06cdc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 -- 172.21.15.37:0/505340900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea06a167d0 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359a000640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 0x55ea06cd9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f359b400640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06cda1f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3594011d10 tx=0x7f3594005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0a0e96605feb7a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.413+0000 7f3584c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f3584c00640 1 -- 172.21.15.37:0/505340900 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594028070 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3594004cc0 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f3584c00640 1 -- 172.21.15.37:0/505340900 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f359400e4e0 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f3584c00640 1 -- 172.21.15.37:0/505340900 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3594034050 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f35a479a740 10 client.26807 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ea0694c3f0 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.414+0000 7f3584c00640 1 -- 172.21.15.37:0/505340900 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3594004cc0 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f3584c00640 1 -- 172.21.15.37:0/505340900 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3594024560 con 0x55ea06cdd290 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f35a479a740 10 client.26807 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f35a479a740 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f35a479a740 20 client.26807 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f35a479a740 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06ced0d0 0x55ea06cef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f35a479a740 4 client.26807 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea06c45790 con 0x55ea06ced0d0 2024-03-21T09:17:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.415+0000 7f359aa00640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06ced0d0 0x55ea06cef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.416+0000 7f359aa00640 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06ced0d0 0x55ea06cef4c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae00ffad4f227054 server_cookie=52d20530695ead12 in_seq=0 out_seq=0 2024-03-21T09:17:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.416+0000 7f3584c00640 10 client.26807 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.416+0000 7f3584c00640 1 -- 172.21.15.37:0/505340900 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ea06c45790 con 0x55ea06ced0d0 2024-03-21T09:17:00.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.416+0000 7f3584c00640 10 client.26807 handle_command_reply: tid=0 2024-03-21T09:17:00.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.416+0000 7f35a479a740 1 client.26807 shutdown 2024-03-21T09:17:00.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 msgr2=0x55ea06cda1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06cda1f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3594011d10 tx=0x7f3594005620 comp rx=0 tx=0).stop 2024-03-21T09:17:00.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 shutdown_connections 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06ced0d0 0x55ea06cef4c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06cda1f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd4c00 0x55ea06cd9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 --2- 172.21.15.37:0/505340900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd3f20 0x55ea06cdc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f3584c00640 0 client.26807 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 >> 172.21.15.37:0/505340900 conn(0x55ea06c9b200 msgr2=0x55ea06cd06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 shutdown_connections 2024-03-21T09:17:00.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 1 -- 172.21.15.37:0/505340900 wait complete. 2024-03-21T09:17:00.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.417+0000 7f35a479a740 20 client.26807 trim_cache size 0 max 16384 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 Processor -- start 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 -- start start 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cdc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1b3180 con 0x55ea06cd4c00 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1b0c80 con 0x55ea06cd3f20 2024-03-21T09:17:00.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.435+0000 7f35a479a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1aed20 con 0x55ea06cdd290 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36172/0 (socket says 172.21.15.37:36172) 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 -- 172.21.15.37:0/3832831946 learned_addr learned my addr 172.21.15.37:0/3832831946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359b400640 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359a000640 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cdc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 -- 172.21.15.37:0/3832831946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 msgr2=0x55ea06cdc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 -- 172.21.15.37:0/3832831946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 msgr2=0x55ea06cdc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cdc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 -- 172.21.15.37:0/3832831946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea0690c0f0 con 0x55ea06cd3f20 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.436+0000 7f359aa00640 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd8b60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f358c036040 tx=0x7f358c046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee276c4103eacb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f3587400640 1 -- 172.21.15.37:0/3832831946 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c046070 con 0x55ea06cd3f20 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f3587400640 1 -- 172.21.15.37:0/3832831946 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f358c044070 con 0x55ea06cd3f20 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f3587400640 1 -- 172.21.15.37:0/3832831946 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c005210 con 0x55ea06cd3f20 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 -- 172.21.15.37:0/3832831946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 msgr2=0x55ea06cd8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd8b60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f358c036040 tx=0x7f358c046000 comp rx=0 tx=0).stop 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 -- 172.21.15.37:0/3832831946 shutdown_connections 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cdd290 0x55ea06cdc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06cdc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 --2- 172.21.15.37:0/3832831946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cd3f20 0x55ea06cd8b60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 -- 172.21.15.37:0/3832831946 >> 172.21.15.37:0/3832831946 conn(0x55ea06c9b200 msgr2=0x55ea06cd8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 -- 172.21.15.37:0/3832831946 shutdown_connections 2024-03-21T09:17:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.437+0000 7f35a479a740 1 -- 172.21.15.37:0/3832831946 wait complete. 2024-03-21T09:17:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.438+0000 7f35a479a740 1 -- start start 2024-03-21T09:17:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.438+0000 7f35a479a740 1 Processor -- start 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.438+0000 7f35a479a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 0x55ea06ce0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.438+0000 7f35a479a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06ce10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.438+0000 7f35a479a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06ce1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f35a479a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1af320 con 0x55ea06cd4c00 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f35a479a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1b0620 con 0x55ea06cdd290 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f35a479a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1af960 con 0x55ea06cd3f20 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f35a479a740 10 client.0 _ll_register_callbacks cb 0x7f35940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 0x55ea06ce0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 0x55ea06ce0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46172/0 (socket says 172.21.15.37:46172) 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 -- 172.21.15.37:0/2550156044 learned_addr learned my addr 172.21.15.37:0/2550156044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359a000640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06ce10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359b400640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06ce1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 -- 172.21.15.37:0/2550156044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 msgr2=0x55ea06ce1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06ce1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 -- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 msgr2=0x55ea06ce10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06ce10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 -- 172.21.15.37:0/2550156044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea06d47980 con 0x55ea06cd3f20 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f359aa00640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 0x55ea06ce0b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f358c031040 tx=0x7f358c0050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28496ea53911820 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.439+0000 7f3584c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f3584c00640 1 -- 172.21.15.37:0/2550156044 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c064030 con 0x55ea06cd3f20 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f358c044070 con 0x55ea06cd3f20 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f3584c00640 1 -- 172.21.15.37:0/2550156044 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f358c05b040 con 0x55ea06cd3f20 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f3584c00640 1 -- 172.21.15.37:0/2550156044 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f358c06e050 con 0x55ea06cd3f20 2024-03-21T09:17:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f35a479a740 10 client.26839 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ea06d48ca0 con 0x55ea06cd3f20 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f3584c00640 1 -- 172.21.15.37:0/2550156044 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f358c044070 con 0x55ea06cd3f20 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f3584c00640 1 -- 172.21.15.37:0/2550156044 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f358c003660 con 0x55ea06cd3f20 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f35a479a740 10 client.26839 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f35a479a740 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.440+0000 7f35a479a740 20 client.26839 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.441+0000 7f35a479a740 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06d49370 0x55ea06d4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.441+0000 7f35a479a740 4 client.26839 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.441+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea06c3fae0 con 0x55ea06d49370 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.441+0000 7f359a000640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06d49370 0x55ea06d4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.441+0000 7f359a000640 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06d49370 0x55ea06d4b760 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9974f6b73797ee server_cookie=753231cb8fff47c8 in_seq=0 out_seq=0 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.441+0000 7f3584c00640 10 client.26839 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f3584c00640 1 -- 172.21.15.37:0/2550156044 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ea06c3fae0 con 0x55ea06d49370 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f3584c00640 10 client.26839 handle_command_reply: tid=0 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 client.26839 shutdown 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 msgr2=0x55ea06ce0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 0x55ea06ce0b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f358c031040 tx=0x7f358c0050b0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 shutdown_connections 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea06d49370 0x55ea06d4b760 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea06cdd290 0x55ea06ce1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea06cd4c00 0x55ea06ce10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f3584c00640 0 client.26839 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 --2- 172.21.15.37:0/2550156044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea06cd3f20 0x55ea06ce0b80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 >> 172.21.15.37:0/2550156044 conn(0x55ea06c9b200 msgr2=0x55ea06cd05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 shutdown_connections 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 1 -- 172.21.15.37:0/2550156044 wait complete. 2024-03-21T09:17:00.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.442+0000 7f35a479a740 20 client.26839 trim_cache size 0 max 16384 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: "8bf8ca6f-1fd3-491d-84ea-57b857f4d8e3": { 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "8bf8ca6f-1fd3-491d-84ea-57b857f4d8e3", 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:17:00.444 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:17:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f35800a3110 msgr2=0x7f35800a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f35800a3110 0x7f35800a55d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f358c0096c0 tx=0x7f358c02df90 comp rx=0 tx=0).stop 2024-03-21T09:17:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 msgr2=0x7f359c102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:00.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c102710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f358800e9f0 tx=0x7f358800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 shutdown_connections 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f359c106620 0x7f359c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f35800a3110 0x7f35800a55d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f359c100f20 0x7f359c102710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 --2- 172.21.15.37:0/1385241968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f359c100560 0x7f359c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 >> 172.21.15.37:0/1385241968 conn(0x7f359c077750 msgr2=0x7f359c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 shutdown_connections 2024-03-21T09:17:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:00.443+0000 7f35a1a00640 1 -- 172.21.15.37:0/1385241968 wait complete. 2024-03-21T09:17:00.456 INFO:tasks.cephfs.filesystem:scrub status for tag:8bf8ca6f-1fd3-491d-84ea-57b857f4d8e3 - {'path': '/', 'tag': '8bf8ca6f-1fd3-491d-84ea-57b857f4d8e3', 'options': 'recursive,force'} 2024-03-21T09:17:00.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:00 smithi071 ceph-mon[36597]: pgmap v147: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:00.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:00 smithi037 ceph-mon[31830]: pgmap v147: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:00.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:00 smithi053 ceph-mon[36169]: pgmap v147: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:01.602 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:01 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:17:01.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:01 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:01.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:01 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:17:01.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:01 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:01 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:17:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:01 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:01 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:17:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:01 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:01.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:01 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:17:01.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:01 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:01.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:01 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:17:01.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:01 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:02.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:02 smithi071 ceph-mon[36597]: pgmap v148: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:02.504 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:02 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:17:02.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:02 smithi037 ceph-mon[31830]: pgmap v148: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:02.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:02 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:17:02.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:02 smithi053 ceph-mon[36169]: pgmap v148: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:02.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:02 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:17:04.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:04 smithi071 ceph-mon[36597]: pgmap v149: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:04.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:04 smithi037 ceph-mon[31830]: pgmap v149: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:04.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:04 smithi053 ceph-mon[36169]: pgmap v149: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:06.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:06 smithi071 ceph-mon[36597]: pgmap v150: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:17:06.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:06 smithi037 ceph-mon[31830]: pgmap v150: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:17:06.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:06 smithi053 ceph-mon[36169]: pgmap v150: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-21T09:17:08.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:08 smithi071 ceph-mon[36597]: pgmap v151: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:08.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:08 smithi037 ceph-mon[31830]: pgmap v151: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:08.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:08 smithi053 ceph-mon[36169]: pgmap v151: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:10.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:10 smithi071 ceph-mon[36597]: pgmap v152: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:17:10.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:10 smithi037 ceph-mon[31830]: pgmap v152: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:17:10.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:10 smithi053 ceph-mon[36169]: pgmap v152: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:17:12.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:12 smithi071 ceph-mon[36597]: pgmap v153: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:17:12.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:12 smithi037 ceph-mon[31830]: pgmap v153: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:17:12.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:12 smithi053 ceph-mon[36169]: pgmap v153: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:17:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:14 smithi071 ceph-mon[36597]: pgmap v154: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:14.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:14 smithi037 ceph-mon[31830]: pgmap v154: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:14.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:14 smithi053 ceph-mon[36169]: pgmap v154: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:17:16.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:16 smithi071 ceph-mon[36597]: pgmap v155: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:17:16.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:16 smithi037 ceph-mon[31830]: pgmap v155: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:17:16.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:16 smithi053 ceph-mon[36169]: pgmap v155: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-21T09:17:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:18 smithi071 ceph-mon[36597]: pgmap v156: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:17:18.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:18 smithi037 ceph-mon[31830]: pgmap v156: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:17:18.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:18 smithi053 ceph-mon[36169]: pgmap v156: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:17:20.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:20 smithi071 ceph-mon[36597]: pgmap v157: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:17:20.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:20 smithi037 ceph-mon[31830]: pgmap v157: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:17:20.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:20 smithi053 ceph-mon[36169]: pgmap v157: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:17:22.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:22 smithi071 ceph-mon[36597]: pgmap v158: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:17:22.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:22 smithi037 ceph-mon[31830]: pgmap v158: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:17:22.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:22 smithi053 ceph-mon[36169]: pgmap v158: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:17:24.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:24 smithi071 ceph-mon[36597]: pgmap v159: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:17:24.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:24 smithi037 ceph-mon[31830]: pgmap v159: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:17:24.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:24 smithi053 ceph-mon[36169]: pgmap v159: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:17:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:26 smithi071 ceph-mon[36597]: pgmap v160: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:26.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:26 smithi037 ceph-mon[31830]: pgmap v160: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:26.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:26 smithi053 ceph-mon[36169]: pgmap v160: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:27.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:27 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:17:27.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:27 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:17:27.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:27 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:17:28.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:28 smithi071 ceph-mon[36597]: pgmap v161: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:28.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:28 smithi037 ceph-mon[31830]: pgmap v161: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:28.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:28 smithi053 ceph-mon[36169]: pgmap v161: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:29 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:17:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:29 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:17:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:29 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:17:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:29 smithi071 ceph-mon[36597]: pgmap v162: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:29.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:29 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:17:29.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:29 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:17:29.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:29 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:17:29.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:29 smithi037 ceph-mon[31830]: pgmap v162: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:29 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:17:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:29 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:17:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:29 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:17:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:29 smithi053 ceph-mon[36169]: pgmap v162: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:30.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:17:30.457 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:17:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.695+0000 7f6c8a800640 1 -- 172.21.15.37:0/31488221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84107500 msgr2=0x7f6c841078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.695+0000 7f6c8a800640 1 --2- 172.21.15.37:0/31488221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84107500 0x7f6c841078e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009900 tx=0x7f6c7802bc40 comp rx=0 tx=0).stop 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.695+0000 7f6c8a800640 1 -- 172.21.15.37:0/31488221 shutdown_connections 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.696+0000 7f6c8a800640 1 --2- 172.21.15.37:0/31488221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84101e20 0x7f6c8410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.696+0000 7f6c8a800640 1 --2- 172.21.15.37:0/31488221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c841018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.696+0000 7f6c8a800640 1 --2- 172.21.15.37:0/31488221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84107500 0x7f6c841078e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.696+0000 7f6c8a800640 1 -- 172.21.15.37:0/31488221 >> 172.21.15.37:0/31488221 conn(0x7f6c840fcff0 msgr2=0x7f6c840ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.696+0000 7f6c8a800640 1 -- 172.21.15.37:0/31488221 shutdown_connections 2024-03-21T09:17:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.696+0000 7f6c8a800640 1 -- 172.21.15.37:0/31488221 wait complete. 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.697+0000 7f6c8a800640 1 Processor -- start 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.697+0000 7f6c8a800640 1 -- start start 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.697+0000 7f6c8a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c84199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84101e20 0x7f6c8419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c8a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84107500 0x7f6c8419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c8a800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84111850 con 0x7f6c84107500 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c8a800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841116d0 con 0x7f6c84101e20 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c8a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841119d0 con 0x7f6c84101460 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c84199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c84199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43468/0 (socket says 172.21.15.37:43468) 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 -- 172.21.15.37:0/3855976337 learned_addr learned my addr 172.21.15.37:0/3855976337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c88a00640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84101e20 0x7f6c8419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89e00640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84107500 0x7f6c8419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 -- 172.21.15.37:0/3855976337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84101e20 msgr2=0x7f6c8419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84101e20 0x7f6c8419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 -- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84107500 msgr2=0x7f6c8419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84107500 0x7f6c8419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89400640 1 -- 172.21.15.37:0/3855976337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8419ef40 con 0x7f6c84101460 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.698+0000 7f6c89e00640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84107500 0x7f6c8419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:17:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.699+0000 7f6c89400640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c84199f50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005ab0 tx=0x7f6c780042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.699+0000 7f6c82000640 1 -- 172.21.15.37:0/3855976337 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78041070 con 0x7f6c84101460 2024-03-21T09:17:30.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.699+0000 7f6c82000640 1 -- 172.21.15.37:0/3855976337 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c7802ccd0 con 0x7f6c84101460 2024-03-21T09:17:30.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.699+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8419f1d0 con 0x7f6c84101460 2024-03-21T09:17:30.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.699+0000 7f6c82000640 1 -- 172.21.15.37:0/3855976337 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78003d10 con 0x7f6c84101460 2024-03-21T09:17:30.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.699+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c841a6ac0 con 0x7f6c84101460 2024-03-21T09:17:30.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.703+0000 7f6c8ebb0740 1 Processor -- start 2024-03-21T09:17:30.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.703+0000 7f6c8ebb0740 1 -- start start 2024-03-21T09:17:30.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.703+0000 7f6c82000640 1 -- 172.21.15.37:0/3855976337 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6c78044070 con 0x7f6c84101460 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abedd0 0x555e84abdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 0x555e84abe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84acfc70 0x555e84ad2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e84a22250 con 0x555e84abedd0 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e849caa30 con 0x555e84abe0f0 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e84a1e2f0 con 0x555e84acfc70 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c89400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 0x555e84abe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c89400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 0x555e84abe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46104/0 (socket says 172.21.15.37:46104) 2024-03-21T09:17:30.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c89400640 1 -- 172.21.15.37:0/820098478 learned_addr learned my addr 172.21.15.37:0/820098478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c88a00640 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84acfc70 0x555e84ad2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c89e00640 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abedd0 0x555e84abdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c89400640 1 -- 172.21.15.37:0/820098478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84acfc70 msgr2=0x555e84ad2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.704+0000 7f6c89400640 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84acfc70 0x555e84ad2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c89400640 1 -- 172.21.15.37:0/820098478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abedd0 msgr2=0x555e84abdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c89400640 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abedd0 0x555e84abdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c89400640 1 -- 172.21.15.37:0/820098478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e849b5690 con 0x555e84abe0f0 2024-03-21T09:17:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c89400640 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 0x555e84abe570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c780ee960 tx=0x7f6c780f7640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a5f92fa682ad4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c76600640 1 -- 172.21.15.37:0/820098478 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c780f1df0 con 0x555e84abe0f0 2024-03-21T09:17:30.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c76600640 1 -- 172.21.15.37:0/820098478 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c780fe030 con 0x555e84abe0f0 2024-03-21T09:17:30.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.705+0000 7f6c76600640 1 -- 172.21.15.37:0/820098478 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78103880 con 0x555e84abe0f0 2024-03-21T09:17:30.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c82000640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6c6c0a3090 0x7f6c6c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/820098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 msgr2=0x555e84abe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 0x555e84abe570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c780ee960 tx=0x7f6c780f7640 comp rx=0 tx=0).stop 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/820098478 shutdown_connections 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84acfc70 0x555e84ad2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abe0f0 0x555e84abe570 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/820098478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abedd0 0x555e84abdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/820098478 >> 172.21.15.37:0/820098478 conn(0x555e84a85200 msgr2=0x555e84ab8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c82000640 1 -- 172.21.15.37:0/3855976337 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6c780f1530 con 0x7f6c84101460 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/820098478 shutdown_connections 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c89e00640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6c6c0a3090 0x7f6c6c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.706+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/820098478 wait complete. 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.707+0000 7f6c8ebb0740 1 -- start start 2024-03-21T09:17:30.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.707+0000 7f6c89e00640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6c6c0a3090 0x7f6c6c0a5550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c005fd0 tx=0x7f6c7c009890 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.707+0000 7f6c8ebb0740 1 Processor -- start 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.707+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.707+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.707+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84accca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e84ad28a0 con 0x555e84abe0f0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e849dde80 con 0x555e84acfc70 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e84ad2a20 con 0x555e84abedd0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c8ebb0740 10 client.0 _ll_register_callbacks cb 0x555e849f0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84accca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c89400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43486/0 (socket says 172.21.15.37:43486) 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 -- 172.21.15.37:0/2554723270 learned_addr learned my addr 172.21.15.37:0/2554723270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:30.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84accca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46118/0 (socket says 172.21.15.37:46118) 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 -- 172.21.15.37:0/2554723270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 msgr2=0x555e84accca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84accca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 -- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 msgr2=0x555e84acf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c88a00640 1 -- 172.21.15.37:0/2554723270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e848007d0 con 0x555e84abedd0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.708+0000 7f6c89e00640 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84accca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c88a00640 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acc760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6c70012740 tx=0x7f6c70012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cdc3e802ba64250 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c6aa00640 1 -- 172.21.15.37:0/2554723270 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000f920 con 0x555e84abedd0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c780fe030 con 0x555e84abedd0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c6aa00640 1 -- 172.21.15.37:0/2554723270 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6c7000fac0 con 0x555e84abedd0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c6aa00640 1 -- 172.21.15.37:0/2554723270 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c7000fcc0 con 0x555e84abedd0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c8ebb0740 10 client.26851 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555e847363f0 con 0x555e84abedd0 2024-03-21T09:17:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.709+0000 7f6c6aa00640 1 -- 172.21.15.37:0/2554723270 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c780fe030 con 0x555e84abedd0 2024-03-21T09:17:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c6aa00640 1 -- 172.21.15.37:0/2554723270 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6c7001c420 con 0x555e84abedd0 2024-03-21T09:17:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c8ebb0740 10 client.26851 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c8ebb0740 10 client.26851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c8ebb0740 20 client.26851 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84adf900 0x555e84ae1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c8ebb0740 4 client.26851 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e84a2f790 con 0x555e84adf900 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.710+0000 7f6c89400640 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84adf900 0x555e84ae1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.711+0000 7f6c89400640 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84adf900 0x555e84ae1cf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4c5cbc4d8ac4cf server_cookie=3d270fe4aa6e8fe in_seq=0 out_seq=0 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.711+0000 7f6c6aa00640 10 client.26851 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.711+0000 7f6c6aa00640 1 -- 172.21.15.37:0/2554723270 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e84a2f790 con 0x555e84adf900 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.711+0000 7f6c6aa00640 10 client.26851 handle_command_reply: tid=0 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.711+0000 7f6c8ebb0740 1 client.26851 shutdown 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 msgr2=0x555e84acc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acc760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6c70012740 tx=0x7f6c70012c00 comp rx=0 tx=0).stop 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 shutdown_connections 2024-03-21T09:17:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84adf900 0x555e84ae1cf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84accca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acc760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c6aa00640 0 client.26851 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/2554723270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 >> 172.21.15.37:0/2554723270 conn(0x555e84a85200 msgr2=0x555e84aba890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 shutdown_connections 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/2554723270 wait complete. 2024-03-21T09:17:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.712+0000 7f6c8ebb0740 20 client.26851 trim_cache size 0 max 16384 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 Processor -- start 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 -- start start 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84ad1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841aeb40 con 0x555e84abe0f0 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841ac640 con 0x555e84acfc70 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.730+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841aa720 con 0x555e84abedd0 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84ad1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84ad1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46120/0 (socket says 172.21.15.37:46120) 2024-03-21T09:17:30.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 -- 172.21.15.37:0/29779021 learned_addr learned my addr 172.21.15.37:0/29779021 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:30.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c88a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 -- 172.21.15.37:0/29779021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 msgr2=0x555e84acf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 -- 172.21.15.37:0/29779021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 msgr2=0x555e84acee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 -- 172.21.15.37:0/29779021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e846f60f0 con 0x555e84acfc70 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c88a00640 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:30.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.731+0000 7f6c89e00640 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84ad1060 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c01e680 tx=0x7f6c7c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06043481e45031a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c75c00640 1 -- 172.21.15.37:0/29779021 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c02c070 con 0x555e84acfc70 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c75c00640 1 -- 172.21.15.37:0/29779021 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c0278f0 con 0x555e84acfc70 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c75c00640 1 -- 172.21.15.37:0/29779021 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c01db40 con 0x555e84acfc70 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/29779021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 msgr2=0x555e84ad1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84ad1060 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c01e680 tx=0x7f6c7c015000 comp rx=0 tx=0).stop 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/29779021 shutdown_connections 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84acfc70 0x555e84ad1060 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abedd0 0x555e84acf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/29779021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84abe0f0 0x555e84acee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/29779021 >> 172.21.15.37:0/29779021 conn(0x555e84a85200 msgr2=0x555e84acb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/29779021 shutdown_connections 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.732+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/29779021 wait complete. 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 -- start start 2024-03-21T09:17:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 Processor -- start 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abe0f0 0x555e84ac9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 0x555e84aca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84acfc70 0x555e84ad0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841aad20 con 0x555e84acfc70 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841abfe0 con 0x555e84abedd0 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.733+0000 7f6c8ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c841ab320 con 0x555e84abe0f0 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c8ebb0740 10 client.0 _ll_register_callbacks cb 0x7f6c70009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 0x555e84aca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abe0f0 0x555e84ac9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 0x555e84aca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46122/0 (socket says 172.21.15.37:46122) 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 -- 172.21.15.37:0/384148649 learned_addr learned my addr 172.21.15.37:0/384148649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c89e00640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84acfc70 0x555e84ad0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 -- 172.21.15.37:0/384148649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abe0f0 msgr2=0x555e84ac9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abe0f0 0x555e84ac9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 -- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84acfc70 msgr2=0x555e84ad0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84acfc70 0x555e84ad0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 -- 172.21.15.37:0/384148649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e84ad10b0 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c89e00640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84acfc70 0x555e84ad0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c88a00640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 0x555e84aca310 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c700059f0 tx=0x7f6c7000fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892befd6a2706fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.734+0000 7f6c6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c6aa00640 1 -- 172.21.15.37:0/384148649 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7002f060 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c6aa00640 1 -- 172.21.15.37:0/384148649 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6c70005070 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c7c0278f0 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c6aa00640 1 -- 172.21.15.37:0/384148649 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c70005270 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 10 client.26831 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555e84b3bd10 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c6aa00640 1 -- 172.21.15.37:0/384148649 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c0278f0 con 0x555e84abedd0 2024-03-21T09:17:30.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c6aa00640 1 -- 172.21.15.37:0/384148649 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6c70003b60 con 0x555e84abedd0 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 10 client.26831 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 10 client.26831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 20 client.26831 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84b3c440 0x555e84b3e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 4 client.26831 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.735+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e84a29ae0 con 0x555e84b3c440 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.736+0000 7f6c89400640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84b3c440 0x555e84b3e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.736+0000 7f6c89400640 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84b3c440 0x555e84b3e860 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90658ed33573a1a6 server_cookie=9a1d967b30911bd4 in_seq=0 out_seq=0 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.736+0000 7f6c6aa00640 10 client.26831 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.736+0000 7f6c6aa00640 1 -- 172.21.15.37:0/384148649 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e84a29ae0 con 0x555e84b3c440 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.736+0000 7f6c6aa00640 10 client.26831 handle_command_reply: tid=0 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.736+0000 7f6c8ebb0740 1 client.26831 shutdown 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 msgr2=0x555e84aca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 0x555e84aca310 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c700059f0 tx=0x7f6c7000fb30 comp rx=0 tx=0).stop 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 shutdown_connections 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555e84acfc70 0x555e84ad0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555e84b3c440 0x555e84b3e860 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e84abedd0 0x555e84aca310 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 --2- 172.21.15.37:0/384148649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e84abe0f0 0x555e84ac9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c6aa00640 0 client.26831 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 >> 172.21.15.37:0/384148649 conn(0x555e84a85200 msgr2=0x555e84ab7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 shutdown_connections 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 1 -- 172.21.15.37:0/384148649 wait complete. 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.737+0000 7f6c8ebb0740 20 client.26831 trim_cache size 0 max 16384 2024-03-21T09:17:30.735 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:17:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:17:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:17:30.736 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:17:30.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6c6c0a3090 msgr2=0x7f6c6c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6c6c0a3090 0x7f6c6c0a5550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c005fd0 tx=0x7f6c7c009890 comp rx=0 tx=0).stop 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 msgr2=0x7f6c84199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c84199f50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005ab0 tx=0x7f6c780042f0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 shutdown_connections 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84107500 0x7f6c8419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6c6c0a3090 0x7f6c6c0a5550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c84101e20 0x7f6c8419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 --2- 172.21.15.37:0/3855976337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84101460 0x7f6c84199f50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 >> 172.21.15.37:0/3855976337 conn(0x7f6c840fcff0 msgr2=0x7f6c8410cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 shutdown_connections 2024-03-21T09:17:30.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.738+0000 7f6c8a800640 1 -- 172.21.15.37:0/3855976337 wait complete. 2024-03-21T09:17:30.752 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:17:30.752 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:17:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.974+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2753631322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c580743f0 msgr2=0x7f4c5810d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.974+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2753631322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c580743f0 0x7f4c5810d820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009900 tx=0x7f4c4002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2753631322 shutdown_connections 2024-03-21T09:17:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2753631322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c5810dd60 0x7f4c581101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2753631322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c580743f0 0x7f4c5810d820 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2753631322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c58073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2753631322 >> 172.21.15.37:0/2753631322 conn(0x7f4c580fd050 msgr2=0x7f4c580ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2753631322 shutdown_connections 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.975+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2753631322 wait complete. 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.976+0000 7f4c5ca00640 1 Processor -- start 2024-03-21T09:17:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.976+0000 7f4c5ca00640 1 -- start start 2024-03-21T09:17:30.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c5ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c5819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c5ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c580743f0 0x7f4c5819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c5ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c5810dd60 0x7f4c5819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c5ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58113b50 con 0x7f4c580743f0 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c5ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581139d0 con 0x7f4c5810dd60 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c5ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58113cd0 con 0x7f4c58073ad0 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c57400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c5819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c57400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c5819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43520/0 (socket says 172.21.15.37:43520) 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c57400640 1 -- 172.21.15.37:0/2574578248 learned_addr learned my addr 172.21.15.37:0/2574578248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c4fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c580743f0 0x7f4c5819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.977+0000 7f4c57e00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c5810dd60 0x7f4c5819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c4fe00640 1 -- 172.21.15.37:0/2574578248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 msgr2=0x7f4c5819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c4fe00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c5819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c4fe00640 1 -- 172.21.15.37:0/2574578248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c5810dd60 msgr2=0x7f4c5819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c4fe00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c5810dd60 0x7f4c5819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c4fe00640 1 -- 172.21.15.37:0/2574578248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5819f010 con 0x7f4c580743f0 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c57400640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c5819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c57e00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c5810dd60 0x7f4c5819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.978+0000 7f4c4fe00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c580743f0 0x7f4c5819a5b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4c4002c210 tx=0x7f4c40005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.979+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40042020 con 0x7f4c580743f0 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.979+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5819f2a0 con 0x7f4c580743f0 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.979+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c4002ce00 con 0x7f4c580743f0 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.979+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40003a50 con 0x7f4c580743f0 2024-03-21T09:17:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.979+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c581a6b90 con 0x7f4c580743f0 2024-03-21T09:17:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.981+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c58075180 con 0x7f4c580743f0 2024-03-21T09:17:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.983+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4c40045020 con 0x7f4c580743f0 2024-03-21T09:17:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.984+0000 7f4c54c00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4c3c0a3080 0x7f4c3c0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.984+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4c400f1510 con 0x7f4c580743f0 2024-03-21T09:17:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.985+0000 7f4c57400640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4c3c0a3080 0x7f4c3c0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.986+0000 7f4c57400640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4c3c0a3080 0x7f4c3c0a5540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c480095f0 tx=0x7f4c48005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:30.990+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c4004c020 con 0x7f4c580743f0 2024-03-21T09:17:31.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.261+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c58108ee0 con 0x7f4c580743f0 2024-03-21T09:17:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.262+0000 7f4c54c00640 1 -- 172.21.15.37:0/2574578248 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f4c400bdd70 con 0x7f4c580743f0 2024-03-21T09:17:31.256 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:17:31.257 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:17:31.257 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:17:31.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4c3c0a3080 msgr2=0x7f4c3c0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4c3c0a3080 0x7f4c3c0a5540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c480095f0 tx=0x7f4c48005f90 comp rx=0 tx=0).stop 2024-03-21T09:17:31.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c580743f0 msgr2=0x7f4c5819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c580743f0 0x7f4c5819a5b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4c4002c210 tx=0x7f4c40005e80 comp rx=0 tx=0).stop 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 shutdown_connections 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4c3c0a3080 0x7f4c3c0a5540 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c5810dd60 0x7f4c5819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c580743f0 0x7f4c5819a5b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 --2- 172.21.15.37:0/2574578248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c58073ad0 0x7f4c5819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 >> 172.21.15.37:0/2574578248 conn(0x7f4c580fd050 msgr2=0x7f4c580fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 shutdown_connections 2024-03-21T09:17:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.266+0000 7f4c5ca00640 1 -- 172.21.15.37:0/2574578248 wait complete. 2024-03-21T09:17:31.282 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:17:31.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:17:31.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 -- 172.21.15.37:0/4231432849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c1004c0 msgr2=0x7fba2c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 --2- 172.21.15.37:0/4231432849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c1004c0 0x7fba2c1008a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c02bc40 comp rx=0 tx=0).stop 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 -- 172.21.15.37:0/4231432849 shutdown_connections 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 --2- 172.21.15.37:0/4231432849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 0x7fba2c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 --2- 172.21.15.37:0/4231432849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c100e70 0x7fba2c10daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 --2- 172.21.15.37:0/4231432849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c1004c0 0x7fba2c1008a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.492+0000 7fba31600640 1 -- 172.21.15.37:0/4231432849 >> 172.21.15.37:0/4231432849 conn(0x7fba2c077780 msgr2=0x7fba2c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.493+0000 7fba31600640 1 -- 172.21.15.37:0/4231432849 shutdown_connections 2024-03-21T09:17:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.493+0000 7fba31600640 1 -- 172.21.15.37:0/4231432849 wait complete. 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 Processor -- start 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 -- start start 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c1004c0 0x7fba2c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c100e70 0x7fba2c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 0x7fba2c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c113da0 con 0x7fba2c1004c0 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c113c20 con 0x7fba2c10dfe0 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba31600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c113f20 con 0x7fba2c100e70 2024-03-21T09:17:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c1004c0 0x7fba2c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 0x7fba2c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.494+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c1004c0 0x7fba2c19a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39916/0 (socket says 172.21.15.37:39916) 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2aa00640 1 -- 172.21.15.37:0/317888828 learned_addr learned my addr 172.21.15.37:0/317888828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2a000640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c100e70 0x7fba2c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2b400640 1 -- 172.21.15.37:0/317888828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c100e70 msgr2=0x7fba2c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2b400640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c100e70 0x7fba2c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2b400640 1 -- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c1004c0 msgr2=0x7fba2c19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2b400640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c1004c0 0x7fba2c19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2b400640 1 -- 172.21.15.37:0/317888828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c19f260 con 0x7fba2c10dfe0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2a000640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c100e70 0x7fba2c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.495+0000 7fba2b400640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 0x7fba2c19eb30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fba2c069770 tx=0x7fba2400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.496+0000 7fba23400640 1 -- 172.21.15.37:0/317888828 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2400cd50 con 0x7fba2c10dfe0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.496+0000 7fba23400640 1 -- 172.21.15.37:0/317888828 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba24005020 con 0x7fba2c10dfe0 2024-03-21T09:17:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.496+0000 7fba23400640 1 -- 172.21.15.37:0/317888828 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24010690 con 0x7fba2c10dfe0 2024-03-21T09:17:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.496+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c19f550 con 0x7fba2c10dfe0 2024-03-21T09:17:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.496+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c1a6de0 con 0x7fba2c10dfe0 2024-03-21T09:17:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.499+0000 7fba343b6740 1 Processor -- start 2024-03-21T09:17:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba23400640 1 -- 172.21.15.37:0/317888828 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fba240051d0 con 0x7fba2c10dfe0 2024-03-21T09:17:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 -- start start 2024-03-21T09:17:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e36d0 0x561bff4e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4ecc00 0x561bff4ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff4472c0 con 0x561bff4e36d0 2024-03-21T09:17:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff3efc90 con 0x561bff4ecc00 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.501+0000 7fba343b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff443360 con 0x561bff4e4080 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e36d0 0x561bff4e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e36d0 0x561bff4e3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39920/0 (socket says 172.21.15.37:39920) 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2a000640 1 -- 172.21.15.37:0/3651589417 learned_addr learned my addr 172.21.15.37:0/3651589417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2a000640 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4ecc00 0x561bff4ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2aa00640 1 -- 172.21.15.37:0/3651589417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4ecc00 msgr2=0x561bff4ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2aa00640 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4ecc00 0x561bff4ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2aa00640 1 -- 172.21.15.37:0/3651589417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e36d0 msgr2=0x561bff4e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2aa00640 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e36d0 0x561bff4e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.502+0000 7fba2aa00640 1 -- 172.21.15.37:0/3651589417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bff414080 con 0x561bff4e4080 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba2aa00640 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec630 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fba1c02efd0 tx=0x7fba1c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9683f0cb865e2342 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba23400640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fba100a3090 0x7fba100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba17400640 1 -- 172.21.15.37:0/3651589417 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c042070 con 0x561bff4e4080 2024-03-21T09:17:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba17400640 1 -- 172.21.15.37:0/3651589417 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1c02c890 con 0x561bff4e4080 2024-03-21T09:17:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba17400640 1 -- 172.21.15.37:0/3651589417 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c004ee0 con 0x561bff4e4080 2024-03-21T09:17:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba23400640 1 -- 172.21.15.37:0/317888828 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fba240d00f0 con 0x7fba2c10dfe0 2024-03-21T09:17:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.503+0000 7fba2a000640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fba100a3090 0x7fba100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 -- 172.21.15.37:0/3651589417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 msgr2=0x561bff4ec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec630 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fba1c02efd0 tx=0x7fba1c005a20 comp rx=0 tx=0).stop 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 -- 172.21.15.37:0/3651589417 shutdown_connections 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4ecc00 0x561bff4ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec630 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 --2- 172.21.15.37:0/3651589417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e36d0 0x561bff4e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 -- 172.21.15.37:0/3651589417 >> 172.21.15.37:0/3651589417 conn(0x561bff4ddda0 msgr2=0x561bff4de180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba2a000640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fba100a3090 0x7fba100a5550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba18013ac0 tx=0x7fba18009b70 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 -- 172.21.15.37:0/3651589417 shutdown_connections 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 -- 172.21.15.37:0/3651589417 wait complete. 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.504+0000 7fba343b6740 1 -- start start 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 Processor -- start 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e4080 0x561bff4e9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4ecc00 0x561bff4e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff4efb10 con 0x561bff4e4080 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff4030d0 con 0x561bff4e36d0 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bff4efc90 con 0x561bff4ecc00 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba343b6740 10 client.0 _ll_register_callbacks cb 0x561bff407e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4ecc00 0x561bff4e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ec590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46184/0 (socket says 172.21.15.37:46184) 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba2aa00640 1 -- 172.21.15.37:0/2143540838 learned_addr learned my addr 172.21.15.37:0/2143540838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.505+0000 7fba2a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e4080 0x561bff4e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2aa00640 1 -- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4ecc00 msgr2=0x561bff4e9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2aa00640 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4ecc00 0x561bff4e9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2aa00640 1 -- 172.21.15.37:0/2143540838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e4080 msgr2=0x561bff4e9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2aa00640 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e4080 0x561bff4e9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2aa00640 1 -- 172.21.15.37:0/2143540838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bff15b3f0 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2b400640 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4ecc00 0x561bff4e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba2aa00640 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ec590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ab0 tx=0x7fba1c02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf353d3983e22cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba0be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.506+0000 7fba0be00640 1 -- 172.21.15.37:0/2143540838 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c04d070 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba0be00640 1 -- 172.21.15.37:0/2143540838 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fba1c0050e0 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba1c02c890 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba0be00640 1 -- 172.21.15.37:0/2143540838 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba1c05b050 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 10 client.26849 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561bff2257d0 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba0be00640 1 -- 172.21.15.37:0/2143540838 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1c02c890 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba0be00640 1 -- 172.21.15.37:0/2143540838 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fba1c0529f0 con 0x561bff4e36d0 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 10 client.26849 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 10 client.26849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 20 client.26849 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff4fc960 0x561bff4fed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 4 client.26849 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.507+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bff454800 con 0x561bff4fc960 2024-03-21T09:17:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.508+0000 7fba2a000640 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff4fc960 0x561bff4fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.508+0000 7fba2a000640 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff4fc960 0x561bff4fed50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e1001c7018b11aa server_cookie=8301b5193e2c7626 in_seq=0 out_seq=0 2024-03-21T09:17:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.508+0000 7fba0be00640 10 client.26849 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:17:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba0be00640 1 -- 172.21.15.37:0/2143540838 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561bff454800 con 0x561bff4fc960 2024-03-21T09:17:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba0be00640 10 client.26849 handle_command_reply: tid=0 2024-03-21T09:17:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 client.26849 shutdown 2024-03-21T09:17:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 msgr2=0x561bff4ec590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ec590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ab0 tx=0x7fba1c02c660 comp rx=0 tx=0).stop 2024-03-21T09:17:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 shutdown_connections 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4ecc00 0x561bff4e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff4fc960 0x561bff4fed50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4e4080 0x561bff4e9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 --2- 172.21.15.37:0/2143540838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ec590 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba0be00640 0 client.26849 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 >> 172.21.15.37:0/2143540838 conn(0x561bff4ddda0 msgr2=0x561bff4dfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 shutdown_connections 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.509+0000 7fba343b6740 1 -- 172.21.15.37:0/2143540838 wait complete. 2024-03-21T09:17:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.510+0000 7fba343b6740 20 client.26849 trim_cache size 0 max 16384 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.523+0000 7fba343b6740 1 Processor -- start 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.523+0000 7fba343b6740 1 -- start start 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba343b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba343b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba343b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff4ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba343b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1b3000 con 0x561bff4ecc00 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba343b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1b0b00 con 0x561bff4e36d0 2024-03-21T09:17:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba343b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1aeba0 con 0x561bff4e4080 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4dceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46194/0 (socket says 172.21.15.37:46194) 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 -- 172.21.15.37:0/1173164317 learned_addr learned my addr 172.21.15.37:0/1173164317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2a000640 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2b400640 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff4ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 -- 172.21.15.37:0/1173164317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 msgr2=0x561bff4ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 -- 172.21.15.37:0/1173164317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 msgr2=0x561bff4ec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff4ec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2aa00640 1 -- 172.21.15.37:0/1173164317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bff4ef220 con 0x561bff4e36d0 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.524+0000 7fba2b400640 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff4ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.525+0000 7fba2aa00640 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4dceb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fba1c000c00 tx=0x7fba1c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1803cd2663145aea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.525+0000 7fba16a00640 1 -- 172.21.15.37:0/1173164317 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c064070 con 0x561bff4e36d0 2024-03-21T09:17:31.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.525+0000 7fba16a00640 1 -- 172.21.15.37:0/1173164317 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1c02c890 con 0x561bff4e36d0 2024-03-21T09:17:31.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.525+0000 7fba16a00640 1 -- 172.21.15.37:0/1173164317 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c04d040 con 0x561bff4e36d0 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.525+0000 7fba343b6740 1 -- 172.21.15.37:0/1173164317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 msgr2=0x561bff4dceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.525+0000 7fba343b6740 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4dceb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fba1c000c00 tx=0x7fba1c005a20 comp rx=0 tx=0).stop 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 -- 172.21.15.37:0/1173164317 shutdown_connections 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff4ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 --2- 172.21.15.37:0/1173164317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4dceb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 -- 172.21.15.37:0/1173164317 >> 172.21.15.37:0/1173164317 conn(0x561bff4ddda0 msgr2=0x561bff4e4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 -- 172.21.15.37:0/1173164317 shutdown_connections 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 -- 172.21.15.37:0/1173164317 wait complete. 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 -- start start 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.526+0000 7fba343b6740 1 Processor -- start 2024-03-21T09:17:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff556e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1af1a0 con 0x561bff4ecc00 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1b04a0 con 0x561bff4e36d0 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1af7e0 con 0x561bff4e4080 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba343b6740 10 client.0 _ll_register_callbacks cb 0x7fba18009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba2a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.527+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46196/0 (socket says 172.21.15.37:46196) 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 -- 172.21.15.37:0/335064547 learned_addr learned my addr 172.21.15.37:0/335064547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2b400640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff556e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 -- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 msgr2=0x561bff4ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 -- 172.21.15.37:0/335064547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 msgr2=0x561bff556e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff556e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 -- 172.21.15.37:0/335064547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bff5573d0 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2a000640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba2aa00640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ff5f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ec0 tx=0x7fba1c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d79a8f386b7caf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba0be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba0be00640 1 -- 172.21.15.37:0/335064547 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c064030 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.528+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba1c02c890 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba0be00640 1 -- 172.21.15.37:0/335064547 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fba1c02c660 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba0be00640 1 -- 172.21.15.37:0/335064547 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba1c06e050 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 10 client.26855 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561bff558500 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba0be00640 1 -- 172.21.15.37:0/335064547 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1c02c890 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba0be00640 1 -- 172.21.15.37:0/335064547 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fba1c004ee0 con 0x561bff4e36d0 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 10 client.26855 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:31.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 10 client.26855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:17:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 20 client.26855 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff558ab0 0x561bff55aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 4 client.26855 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:17:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.529+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561bff4dd5b0 con 0x561bff558ab0 2024-03-21T09:17:31.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba2a000640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff558ab0 0x561bff55aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba2a000640 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff558ab0 0x561bff55aea0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c8cd273f067e91f server_cookie=4195a2a7077dc325 in_seq=0 out_seq=0 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba0be00640 10 client.26855 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba0be00640 1 -- 172.21.15.37:0/335064547 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561bff4dd5b0 con 0x561bff558ab0 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba0be00640 10 client.26855 handle_command_reply: tid=0 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba343b6740 1 client.26855 shutdown 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 msgr2=0x561bff4ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.530+0000 7fba343b6740 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ff5f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ec0 tx=0x7fba1c005c00 comp rx=0 tx=0).stop 2024-03-21T09:17:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 shutdown_connections 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x561bff558ab0 0x561bff55aea0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561bff4ecc00 0x561bff556e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bff4e4080 0x561bff4ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 --2- 172.21.15.37:0/335064547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561bff4e36d0 0x561bff4ff5f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 >> 172.21.15.37:0/335064547 conn(0x561bff4ddda0 msgr2=0x561bff4dcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba0be00640 0 client.26855 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 shutdown_connections 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 1 -- 172.21.15.37:0/335064547 wait complete. 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba343b6740 20 client.26855 trim_cache size 0 max 16384 2024-03-21T09:17:31.529 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:17:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fba100a3090 msgr2=0x7fba100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba31600640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fba100a3090 0x7fba100a5550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba18013ac0 tx=0x7fba18009b70 comp rx=0 tx=0).stop 2024-03-21T09:17:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 msgr2=0x7fba2c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.531+0000 7fba31600640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 0x7fba2c19eb30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fba2c069770 tx=0x7fba2400ee60 comp rx=0 tx=0).stop 2024-03-21T09:17:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 shutdown_connections 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fba100a3090 0x7fba100a5550 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2c10dfe0 0x7fba2c19eb30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c100e70 0x7fba2c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 --2- 172.21.15.37:0/317888828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba2c1004c0 0x7fba2c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 >> 172.21.15.37:0/317888828 conn(0x7fba2c077780 msgr2=0x7fba2c10f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 shutdown_connections 2024-03-21T09:17:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.532+0000 7fba31600640 1 -- 172.21.15.37:0/317888828 wait complete. 2024-03-21T09:17:31.549 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:17:31.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 -- 172.21.15.37:0/569366863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102630 msgr2=0x7f04bc102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 --2- 172.21.15.37:0/569366863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102630 0x7f04bc102a10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009860 tx=0x7f04ac02bc20 comp rx=0 tx=0).stop 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 -- 172.21.15.37:0/569366863 shutdown_connections 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 --2- 172.21.15.37:0/569366863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04bc10ba50 0x7f04bc10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 --2- 172.21.15.37:0/569366863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102f50 0x7f04bc10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 --2- 172.21.15.37:0/569366863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102630 0x7f04bc102a10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 -- 172.21.15.37:0/569366863 >> 172.21.15.37:0/569366863 conn(0x7f04bc077130 msgr2=0x7f04bc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 -- 172.21.15.37:0/569366863 shutdown_connections 2024-03-21T09:17:31.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.774+0000 7f04c0c00640 1 -- 172.21.15.37:0/569366863 wait complete. 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.775+0000 7f04c0c00640 1 Processor -- start 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 -- start start 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 0x7f04bc199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102f50 0x7f04bc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04bc10ba50 0x7f04bc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc111800 con 0x7f04bc10ba50 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc111680 con 0x7f04bc102f50 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04c0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc111980 con 0x7f04bc102630 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 0x7f04bc199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04bc10ba50 0x7f04bc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102f50 0x7f04bc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 0x7f04bc199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43600/0 (socket says 172.21.15.37:43600) 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.776+0000 7f04bb400640 1 -- 172.21.15.37:0/2976016844 learned_addr learned my addr 172.21.15.37:0/2976016844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04bb400640 1 -- 172.21.15.37:0/2976016844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102f50 msgr2=0x7f04bc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04bb400640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102f50 0x7f04bc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04bb400640 1 -- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04bc10ba50 msgr2=0x7f04bc19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04bb400640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04bc10ba50 0x7f04bc19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04bb400640 1 -- 172.21.15.37:0/2976016844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc19ef50 con 0x7f04bc102630 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04bb400640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 0x7f04bc199f60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f04ac037040 tx=0x7f04ac0344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04b8c00640 1 -- 172.21.15.37:0/2976016844 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac041020 con 0x7f04bc102630 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04b8c00640 1 -- 172.21.15.37:0/2976016844 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04ac02ce50 con 0x7f04bc102630 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc19f1e0 con 0x7f04bc102630 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.777+0000 7f04b8c00640 1 -- 172.21.15.37:0/2976016844 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac039040 con 0x7f04bc102630 2024-03-21T09:17:31.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.778+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc1a6ad0 con 0x7f04bc102630 2024-03-21T09:17:31.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.781+0000 7f04b8c00640 1 -- 172.21.15.37:0/2976016844 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f04ac044020 con 0x7f04bc102630 2024-03-21T09:17:31.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.782+0000 7f04b8c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f04a00a3110 0x7f04a00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.782+0000 7f04b8c00640 1 -- 172.21.15.37:0/2976016844 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f04ac0f0fd0 con 0x7f04bc102630 2024-03-21T09:17:31.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.782+0000 7f04c4e3d740 1 Processor -- start 2024-03-21T09:17:31.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.782+0000 7f04b3e00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f04a00a3110 0x7f04a00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.782+0000 7f04c4e3d740 1 -- start start 2024-03-21T09:17:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ed8b0 0x55be4d8edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ee260 0x55be4d8f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4d8512c0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4d7f9c90 con 0x55be4d8f6de0 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4d84d360 con 0x55be4d8ee260 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46220/0 (socket says 172.21.15.37:46220) 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04bbe00640 1 -- 172.21.15.37:0/4152881287 learned_addr learned my addr 172.21.15.37:0/4152881287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04bb400640 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ed8b0 0x55be4d8edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04b3e00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f04a00a3110 0x7f04a00a55d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0ff520 tx=0x7f04a4009260 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04b3e00640 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ee260 0x55be4d8f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04bbe00640 1 -- 172.21.15.37:0/4152881287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ee260 msgr2=0x55be4d8f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.783+0000 7f04bbe00640 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ee260 0x55be4d8f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04bbe00640 1 -- 172.21.15.37:0/4152881287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ed8b0 msgr2=0x55be4d8edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04bbe00640 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ed8b0 0x55be4d8edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04bbe00640 1 -- 172.21.15.37:0/4152881287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be4d81e080 con 0x55be4d8f6de0 2024-03-21T09:17:31.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04bbe00640 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8f9230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009f50 tx=0x7f04b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3faffec7f840c270 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04abe00640 1 -- 172.21.15.37:0/4152881287 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4004d20 con 0x55be4d8f6de0 2024-03-21T09:17:31.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04abe00640 1 -- 172.21.15.37:0/4152881287 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04b4004050 con 0x55be4d8f6de0 2024-03-21T09:17:31.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.784+0000 7f04abe00640 1 -- 172.21.15.37:0/4152881287 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4005280 con 0x55be4d8f6de0 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 -- 172.21.15.37:0/4152881287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 msgr2=0x55be4d8f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8f9230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009f50 tx=0x7f04b40118d0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 -- 172.21.15.37:0/4152881287 shutdown_connections 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8f9230 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ee260 0x55be4d8f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/4152881287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ed8b0 0x55be4d8edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 -- 172.21.15.37:0/4152881287 >> 172.21.15.37:0/4152881287 conn(0x55be4d8e7f80 msgr2=0x55be4d8e8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 -- 172.21.15.37:0/4152881287 shutdown_connections 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 -- 172.21.15.37:0/4152881287 wait complete. 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.785+0000 7f04c4e3d740 1 -- start start 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 Processor -- start 2024-03-21T09:17:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 0x55be4d8f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8f6de0 0x55be4d8f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4d8f9d50 con 0x55be4d8f6de0 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4d80d0d0 con 0x55be4d8ee260 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4d8f9ed0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04c4e3d740 10 client.0 _ll_register_callbacks cb 0x55be4d811e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 0x55be4d8f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43618/0 (socket says 172.21.15.37:43618) 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 0x55be4d8f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46230/0 (socket says 172.21.15.37:46230) 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.786+0000 7f04bb400640 1 -- 172.21.15.37:0/1266372640 learned_addr learned my addr 172.21.15.37:0/1266372640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bbe00640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8f6de0 0x55be4d8f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bb400640 1 -- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 msgr2=0x55be4d8f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bb400640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 0x55be4d8f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bb400640 1 -- 172.21.15.37:0/1266372640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8f6de0 msgr2=0x55be4d8f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bb400640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8f6de0 0x55be4d8f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bb400640 1 -- 172.21.15.37:0/1266372640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be4d5653f0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04b3e00640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 0x55be4d8f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04bb400640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f66e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0ee490 tx=0x7f04ac0bca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d202ae1e93e35839 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04a8c00640 1 -- 172.21.15.37:0/1266372640 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0d6060 con 0x55be4d8ed8b0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04a8c00640 1 -- 172.21.15.37:0/1266372640 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f04ac0dd0c0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.787+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b4004050 con 0x55be4d8ed8b0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04a8c00640 1 -- 172.21.15.37:0/1266372640 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ac0d6200 con 0x55be4d8ed8b0 2024-03-21T09:17:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 10 client.26893 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55be4d62f7d0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04a8c00640 1 -- 172.21.15.37:0/1266372640 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04b4004050 con 0x55be4d8ed8b0 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04a8c00640 1 -- 172.21.15.37:0/1266372640 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f04ac03c030 con 0x55be4d8ed8b0 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 10 client.26893 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 10 client.26893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 20 client.26893 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d906c10 0x55be4d909060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 4 client.26893 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.788+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be4d85e800 con 0x55be4d906c10 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.789+0000 7f04b3e00640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d906c10 0x55be4d909060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.789+0000 7f04b3e00640 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d906c10 0x55be4d909060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8df5308480b630f5 server_cookie=56c6775e2474fac7 in_seq=0 out_seq=0 2024-03-21T09:17:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.789+0000 7f04a8c00640 10 client.26893 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:17:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04a8c00640 1 -- 172.21.15.37:0/1266372640 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be4d85e800 con 0x55be4d906c10 2024-03-21T09:17:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04a8c00640 10 client.26893 handle_command_reply: tid=0 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 client.26893 shutdown 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 msgr2=0x55be4d8f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f66e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0ee490 tx=0x7f04ac0bca30 comp rx=0 tx=0).stop 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 shutdown_connections 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d906c10 0x55be4d909060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8f6de0 0x55be4d8f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ee260 0x55be4d8f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.790+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1266372640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f66e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.791+0000 7f04a8c00640 0 client.26893 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.791+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 >> 172.21.15.37:0/1266372640 conn(0x55be4d8e7f80 msgr2=0x55be4d8e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.791+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 shutdown_connections 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.791+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1266372640 wait complete. 2024-03-21T09:17:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.791+0000 7f04c4e3d740 20 client.26893 trim_cache size 0 max 16384 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.806+0000 7f04c4e3d740 1 Processor -- start 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.806+0000 7f04c4e3d740 1 -- start start 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.806+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1b2eb0 con 0x55be4d8ee260 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1b09b0 con 0x55be4d8f6de0 2024-03-21T09:17:31.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1aea50 con 0x55be4d8ed8b0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43626/0 (socket says 172.21.15.37:43626) 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 -- 172.21.15.37:0/236484342 learned_addr learned my addr 172.21.15.37:0/236484342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bbe00640 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04b3e00640 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 -- 172.21.15.37:0/236484342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 msgr2=0x55be4d8eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8eeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 -- 172.21.15.37:0/236484342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 msgr2=0x55be4d8f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.807+0000 7f04bb400640 1 -- 172.21.15.37:0/236484342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be4d8edff0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04bb400640 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f61a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0bd060 tx=0x7f04ac0bcb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88fb5b25b84ee164 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04ab400640 1 -- 172.21.15.37:0/236484342 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0d6060 con 0x55be4d8ed8b0 2024-03-21T09:17:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04ab400640 1 -- 172.21.15.37:0/236484342 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04ac02cca0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04ab400640 1 -- 172.21.15.37:0/236484342 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0c5650 con 0x55be4d8ed8b0 2024-03-21T09:17:31.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 -- 172.21.15.37:0/236484342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 msgr2=0x55be4d8f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f61a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0bd060 tx=0x7f04ac0bcb60 comp rx=0 tx=0).stop 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 -- 172.21.15.37:0/236484342 shutdown_connections 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8f6de0 0x55be4d8eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/236484342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8ed8b0 0x55be4d8f61a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 -- 172.21.15.37:0/236484342 >> 172.21.15.37:0/236484342 conn(0x55be4d8e7f80 msgr2=0x55be4d8ee690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.808+0000 7f04c4e3d740 1 -- 172.21.15.37:0/236484342 shutdown_connections 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.809+0000 7f04c4e3d740 1 -- 172.21.15.37:0/236484342 wait complete. 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.809+0000 7f04c4e3d740 1 -- start start 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.809+0000 7f04c4e3d740 1 Processor -- start 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.809+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 0x55be4d8fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.809+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04c4e3d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8f6de0 0x55be4d8fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1af050 con 0x55be4d8ee260 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1b0350 con 0x55be4d8ed8b0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04c4e3d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1af690 con 0x55be4d8f6de0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04c4e3d740 10 client.0 _ll_register_callbacks cb 0x7f04ac0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 0x55be4d8fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8f6de0 0x55be4d8fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 0x55be4d8fc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46238/0 (socket says 172.21.15.37:46238) 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 -- 172.21.15.37:0/1402229126 learned_addr learned my addr 172.21.15.37:0/1402229126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 -- 172.21.15.37:0/1402229126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8f6de0 msgr2=0x55be4d8fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8f6de0 0x55be4d8fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 -- 172.21.15.37:0/1402229126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 msgr2=0x55be4d8fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 -- 172.21.15.37:0/1402229126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be4d9615b0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04bb400640 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 0x55be4d8fc600 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f04ac02cc50 tx=0x7f04ac0bb5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65444b69388ac97 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.810+0000 7f04a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04a8c00640 1 -- 172.21.15.37:0/1402229126 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0d6060 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04a8c00640 1 -- 172.21.15.37:0/1402229126 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f04ac0a60c0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04ac02cca0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04a8c00640 1 -- 172.21.15.37:0/1402229126 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ac0a62c0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 10 client.26873 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55be4d962890 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04a8c00640 1 -- 172.21.15.37:0/1402229126 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04ac02cca0 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04a8c00640 1 -- 172.21.15.37:0/1402229126 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f04ac0bc640 con 0x55be4d8ed8b0 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 10 client.26873 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 10 client.26873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 20 client.26873 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d962b80 0x55be4d964f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:31.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 4 client.26873 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.811+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be4d858b50 con 0x55be4d962b80 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.812+0000 7f04b3e00640 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d962b80 0x55be4d964f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.812+0000 7f04b3e00640 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d962b80 0x55be4d964f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc9b1c120436b591 server_cookie=77b9628979049b4c in_seq=0 out_seq=0 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.812+0000 7f04a8c00640 10 client.26873 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.812+0000 7f04a8c00640 1 -- 172.21.15.37:0/1402229126 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be4d858b50 con 0x55be4d962b80 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.812+0000 7f04a8c00640 10 client.26873 handle_command_reply: tid=0 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.812+0000 7f04c4e3d740 1 client.26873 shutdown 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 msgr2=0x55be4d8fc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 0x55be4d8fc600 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f04ac02cc50 tx=0x7f04ac0bb5e0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 shutdown_connections 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55be4d962b80 0x55be4d964f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be4d8f6de0 0x55be4d8fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be4d8ee260 0x55be4d8fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 --2- 172.21.15.37:0/1402229126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55be4d8ed8b0 0x55be4d8fc600 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 >> 172.21.15.37:0/1402229126 conn(0x55be4d8e7f80 msgr2=0x55be4d8eef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04a8c00640 0 client.26873 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:17:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 shutdown_connections 2024-03-21T09:17:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 1 -- 172.21.15.37:0/1402229126 wait complete. 2024-03-21T09:17:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.813+0000 7f04c4e3d740 20 client.26873 trim_cache size 0 max 16384 2024-03-21T09:17:31.812 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f04a00a3110 msgr2=0x7f04a00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f04a00a3110 0x7f04a00a55d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0ff520 tx=0x7f04a4009260 comp rx=0 tx=0).stop 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 msgr2=0x7f04bc199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 0x7f04bc199f60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f04ac037040 tx=0x7f04ac0344c0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 shutdown_connections 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04bc10ba50 0x7f04bc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f04a00a3110 0x7f04a00a55d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc102f50 0x7f04bc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 --2- 172.21.15.37:0/2976016844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04bc102630 0x7f04bc199f60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 >> 172.21.15.37:0/2976016844 conn(0x7f04bc077130 msgr2=0x7f04bc100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 shutdown_connections 2024-03-21T09:17:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:31.814+0000 7f04c0c00640 1 -- 172.21.15.37:0/2976016844 wait complete. 2024-03-21T09:17:31.827 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:17:31.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:17:32.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 -- 172.21.15.37:0/258133443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c104200 msgr2=0x7f7e1c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 --2- 172.21.15.37:0/258133443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c104200 0x7f7e1c110710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009be0 tx=0x7f7e1402bef0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 -- 172.21.15.37:0/258133443 shutdown_connections 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 --2- 172.21.15.37:0/258133443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c104200 0x7f7e1c110710 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 --2- 172.21.15.37:0/258133443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e1c103840 0x7f7e1c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 --2- 172.21.15.37:0/258133443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c109850 0x7f7e1c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 -- 172.21.15.37:0/258133443 >> 172.21.15.37:0/258133443 conn(0x7f7e1c0ff2f0 msgr2=0x7f7e1c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 -- 172.21.15.37:0/258133443 shutdown_connections 2024-03-21T09:17:32.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.035+0000 7f7e22800640 1 -- 172.21.15.37:0/258133443 wait complete. 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.036+0000 7f7e22800640 1 Processor -- start 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.036+0000 7f7e22800640 1 -- start start 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.036+0000 7f7e22800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 0x7f7e1c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e22800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e1c104200 0x7f7e1c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e22800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c109850 0x7f7e1c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e22800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c113cb0 con 0x7f7e1c104200 2024-03-21T09:17:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e22800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c113b30 con 0x7f7e1c109850 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e22800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c113e30 con 0x7f7e1c103840 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 0x7f7e1c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 0x7f7e1c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43648/0 (socket says 172.21.15.37:43648) 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 -- 172.21.15.37:0/3841534156 learned_addr learned my addr 172.21.15.37:0/3841534156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1be00640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c109850 0x7f7e1c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e1c104200 0x7f7e1c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 -- 172.21.15.37:0/3841534156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c109850 msgr2=0x7f7e1c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c109850 0x7f7e1c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 -- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e1c104200 msgr2=0x7f7e1c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e1c104200 0x7f7e1c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.037+0000 7f7e1b400640 1 -- 172.21.15.37:0/3841534156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c19f0c0 con 0x7f7e1c103840 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.038+0000 7f7e1b400640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 0x7f7e1c19a0d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00e9f0 tx=0x7f7e0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.038+0000 7f7e13e00640 1 -- 172.21.15.37:0/3841534156 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00ccc0 con 0x7f7e1c103840 2024-03-21T09:17:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.038+0000 7f7e13e00640 1 -- 172.21.15.37:0/3841534156 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c005500 con 0x7f7e1c103840 2024-03-21T09:17:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.038+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c19f3b0 con 0x7f7e1c103840 2024-03-21T09:17:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.038+0000 7f7e13e00640 1 -- 172.21.15.37:0/3841534156 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c010690 con 0x7f7e1c103840 2024-03-21T09:17:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.038+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c1a6c40 con 0x7f7e1c103840 2024-03-21T09:17:32.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.042+0000 7f7e255b7740 1 Processor -- start 2024-03-21T09:17:32.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 -- start start 2024-03-21T09:17:32.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e13e00640 1 -- 172.21.15.37:0/3841534156 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7e0c0046c0 con 0x7f7e1c103840 2024-03-21T09:17:32.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a5bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624299bf2c0 con 0x562429a5c080 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562429967c90 con 0x562429a64c00 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e255b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624299bb360 con 0x562429a5b6d0 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.043+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a5bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a5bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43656/0 (socket says 172.21.15.37:43656) 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 -- 172.21.15.37:0/3773406846 learned_addr learned my addr 172.21.15.37:0/3773406846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1b400640 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 -- 172.21.15.37:0/3773406846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 msgr2=0x562429a67050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a67050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 -- 172.21.15.37:0/3773406846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 msgr2=0x562429a64630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 -- 172.21.15.37:0/3773406846 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242998c080 con 0x562429a5b6d0 2024-03-21T09:17:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.044+0000 7f7e1be00640 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a5bab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009be0 tx=0x7f7e140098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306084435f957aef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e10c00640 1 -- 172.21.15.37:0/3773406846 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14043030 con 0x562429a5b6d0 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e10c00640 1 -- 172.21.15.37:0/3773406846 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e14033d40 con 0x562429a5b6d0 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e10c00640 1 -- 172.21.15.37:0/3773406846 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1402ca00 con 0x562429a5b6d0 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e13e00640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7e000a3090 0x7f7e000a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e13e00640 1 -- 172.21.15.37:0/3841534156 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7e0c0cfe60 con 0x7f7e1c103840 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e1b400640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7e000a3090 0x7f7e000a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e255b7740 1 -- 172.21.15.37:0/3773406846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 msgr2=0x562429a5bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.045+0000 7f7e255b7740 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a5bab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009be0 tx=0x7f7e140098f0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 -- 172.21.15.37:0/3773406846 shutdown_connections 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a67050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 --2- 172.21.15.37:0/3773406846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a5bab0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 -- 172.21.15.37:0/3773406846 >> 172.21.15.37:0/3773406846 conn(0x562429a55da0 msgr2=0x562429a56180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 -- 172.21.15.37:0/3773406846 shutdown_connections 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 -- 172.21.15.37:0/3773406846 wait complete. 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e1b400640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7e000a3090 0x7f7e000a5550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0cd2c0 tx=0x7f7e0c0bc040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 -- start start 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.046+0000 7f7e255b7740 1 Processor -- start 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a64530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562429a67b10 con 0x562429a5c080 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242997b0d0 con 0x562429a64c00 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562429a67c90 con 0x562429a5b6d0 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e255b7740 10 client.0 _ll_register_callbacks cb 0x56242997fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40028/0 (socket says 172.21.15.37:40028) 2024-03-21T09:17:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e1aa00640 1 -- 172.21.15.37:0/1240193851 learned_addr learned my addr 172.21.15.37:0/1240193851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.047+0000 7f7e1be00640 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e1aa00640 1 -- 172.21.15.37:0/1240193851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 msgr2=0x562429a64530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a64530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e1aa00640 1 -- 172.21.15.37:0/1240193851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 msgr2=0x562429a61ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a61ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e1aa00640 1 -- 172.21.15.37:0/1240193851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624296d33f0 con 0x562429a5c080 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a618c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009e00 tx=0x7f7e080117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55784a5d42d4f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e05600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e05600640 1 -- 172.21.15.37:0/1240193851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08012860 con 0x562429a5c080 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e14033d40 con 0x562429a5c080 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.048+0000 7f7e05600640 1 -- 172.21.15.37:0/1240193851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7e08012a00 con 0x562429a5c080 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e05600640 1 -- 172.21.15.37:0/1240193851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e08025050 con 0x562429a5c080 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e255b7740 10 client.16665 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56242979d7d0 con 0x562429a5c080 2024-03-21T09:17:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e05600640 1 -- 172.21.15.37:0/1240193851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e14033d40 con 0x562429a5c080 2024-03-21T09:17:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e05600640 1 -- 172.21.15.37:0/1240193851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7e0800f040 con 0x562429a5c080 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e255b7740 10 client.16665 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e255b7740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e255b7740 20 client.16665 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.049+0000 7f7e255b7740 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429a74ad0 0x562429a76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.050+0000 7f7e255b7740 4 client.16665 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.050+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624299cc800 con 0x562429a74ad0 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.050+0000 7f7e1be00640 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429a74ad0 0x562429a76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.050+0000 7f7e1be00640 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429a74ad0 0x562429a76ef0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526d7360d5af5830 server_cookie=ad6e3cec0b0d1f4e in_seq=0 out_seq=0 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.050+0000 7f7e05600640 10 client.16665 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e05600640 1 -- 172.21.15.37:0/1240193851 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5624299cc800 con 0x562429a74ad0 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e05600640 10 client.16665 handle_command_reply: tid=0 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 client.16665 shutdown 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 msgr2=0x562429a618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a618c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009e00 tx=0x7f7e080117b0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 shutdown_connections 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429a74ad0 0x562429a76ef0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a64c00 0x562429a61ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a618c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e255b7740 1 --2- 172.21.15.37:0/1240193851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5b6d0 0x562429a64530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.052+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 >> 172.21.15.37:0/1240193851 conn(0x562429a55da0 msgr2=0x562429a57af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.051+0000 7f7e05600640 0 client.16665 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.052+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 shutdown_connections 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.052+0000 7f7e255b7740 1 -- 172.21.15.37:0/1240193851 wait complete. 2024-03-21T09:17:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.052+0000 7f7e255b7740 20 client.16665 trim_cache size 0 max 16384 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.066+0000 7f7e255b7740 1 Processor -- start 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.066+0000 7f7e255b7740 1 -- start start 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a64c00 0x562429a5eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e255b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1b3020 con 0x562429a5c080 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e255b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1b0b20 con 0x562429a5b6d0 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e255b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1aebc0 con 0x562429a64c00 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a63ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46278/0 (socket says 172.21.15.37:46278) 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 -- 172.21.15.37:0/2270954276 learned_addr learned my addr 172.21.15.37:0/2270954276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a64c00 0x562429a5eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 -- 172.21.15.37:0/2270954276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a64c00 msgr2=0x562429a5eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a64c00 0x562429a5eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 -- 172.21.15.37:0/2270954276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 msgr2=0x562429a64530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.067+0000 7f7e1be00640 1 -- 172.21.15.37:0/2270954276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562429a555c0 con 0x562429a5b6d0 2024-03-21T09:17:32.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.068+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a63ff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x562429a62cc0 tx=0x7f7e14005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcb83a54d333382 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.068+0000 7f7e07e00640 1 -- 172.21.15.37:0/2270954276 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14043040 con 0x562429a5b6d0 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.068+0000 7f7e07e00640 1 -- 172.21.15.37:0/2270954276 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e14033d40 con 0x562429a5b6d0 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.068+0000 7f7e07e00640 1 -- 172.21.15.37:0/2270954276 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14054880 con 0x562429a5b6d0 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 -- 172.21.15.37:0/2270954276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 msgr2=0x562429a63ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a63ff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x562429a62cc0 tx=0x7f7e14005fd0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 -- 172.21.15.37:0/2270954276 shutdown_connections 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a64c00 0x562429a5eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a5c080 0x562429a64530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2270954276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a63ff0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 -- 172.21.15.37:0/2270954276 >> 172.21.15.37:0/2270954276 conn(0x562429a55da0 msgr2=0x562429a54eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 -- 172.21.15.37:0/2270954276 shutdown_connections 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 -- 172.21.15.37:0/2270954276 wait complete. 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 -- start start 2024-03-21T09:17:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.069+0000 7f7e255b7740 1 Processor -- start 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 0x562429a6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a64c00 0x562429a6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1af1c0 con 0x562429a64c00 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1b04c0 con 0x562429a5b6d0 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1af800 con 0x562429a5c080 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e255b7740 10 client.0 _ll_register_callbacks cb 0x7f7e08009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 0x562429a6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 0x562429a6b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43692/0 (socket says 172.21.15.37:43692) 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e1aa00640 1 -- 172.21.15.37:0/2262230697 learned_addr learned my addr 172.21.15.37:0/2262230697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.070+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a64c00 0x562429a6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e1aa00640 1 -- 172.21.15.37:0/2262230697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 msgr2=0x562429a6cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a6cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e1aa00640 1 -- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a64c00 msgr2=0x562429a6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a64c00 0x562429a6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e1aa00640 1 -- 172.21.15.37:0/2262230697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562429a6bda0 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e1aa00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 0x562429a6b030 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562429a62e90 tx=0x7f7e080195b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=687524ad23a2d466 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e05600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e05600640 1 -- 172.21.15.37:0/2262230697 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08005850 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e14033d40 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.071+0000 7f7e05600640 1 -- 172.21.15.37:0/2262230697 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7e08005250 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e05600640 1 -- 172.21.15.37:0/2262230697 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0803a050 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 10 client.26917 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562429ad13d0 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e05600640 1 -- 172.21.15.37:0/2262230697 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e14033d40 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e05600640 1 -- 172.21.15.37:0/2262230697 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7e080196b0 con 0x562429a5c080 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 10 client.26917 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 20 client.26917 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429ad1aa0 0x562429ad3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 4 client.26917 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:17:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.072+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624299c6b50 con 0x562429ad1aa0 2024-03-21T09:17:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.073+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429ad1aa0 0x562429ad3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.073+0000 7f7e1be00640 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429ad1aa0 0x562429ad3ec0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da26f4397e3f7561 server_cookie=25fe72f1305f36ef in_seq=0 out_seq=0 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.073+0000 7f7e05600640 10 client.26917 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.073+0000 7f7e05600640 1 -- 172.21.15.37:0/2262230697 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5624299c6b50 con 0x562429ad1aa0 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.073+0000 7f7e05600640 10 client.26917 handle_command_reply: tid=0 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.073+0000 7f7e255b7740 1 client.26917 shutdown 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 msgr2=0x562429a6b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 0x562429a6b030 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562429a62e90 tx=0x7f7e080195b0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 shutdown_connections 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562429a64c00 0x562429a6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562429a5c080 0x562429a6b030 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562429ad1aa0 0x562429ad3ec0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 --2- 172.21.15.37:0/2262230697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562429a5b6d0 0x562429a6cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 >> 172.21.15.37:0/2262230697 conn(0x562429a55da0 msgr2=0x562429a5c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e05600640 0 client.26917 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 shutdown_connections 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 1 -- 172.21.15.37:0/2262230697 wait complete. 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.074+0000 7f7e255b7740 20 client.26917 trim_cache size 0 max 16384 2024-03-21T09:17:32.072 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:17:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7e000a3090 msgr2=0x7f7e000a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7e000a3090 0x7f7e000a5550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0cd2c0 tx=0x7f7e0c0bc040 comp rx=0 tx=0).stop 2024-03-21T09:17:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 msgr2=0x7f7e1c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 0x7f7e1c19a0d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00e9f0 tx=0x7f7e0c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 shutdown_connections 2024-03-21T09:17:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7e000a3090 0x7f7e000a5550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c109850 0x7f7e1c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e1c104200 0x7f7e1c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 --2- 172.21.15.37:0/3841534156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e1c103840 0x7f7e1c19a0d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 >> 172.21.15.37:0/3841534156 conn(0x7f7e1c0ff2f0 msgr2=0x7f7e1c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 shutdown_connections 2024-03-21T09:17:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:32.075+0000 7f7e22800640 1 -- 172.21.15.37:0/3841534156 wait complete. 2024-03-21T09:17:32.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:32 smithi037 ceph-mon[31830]: pgmap v163: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:32.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:32 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2574578248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:17:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:32 smithi071 ceph-mon[36597]: pgmap v163: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:32.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:32 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2574578248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:17:32.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:32 smithi053 ceph-mon[36169]: pgmap v163: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:32.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:32 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2574578248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:17:33.091 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:17:33.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:17:33.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 -- 172.21.15.37:0/1820030683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58103660 msgr2=0x7fae58103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 --2- 172.21.15.37:0/1820030683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58103660 0x7fae58103ae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae44009900 tx=0x7fae4402bc30 comp rx=0 tx=0).stop 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 -- 172.21.15.37:0/1820030683 shutdown_connections 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 --2- 172.21.15.37:0/1820030683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae58104020 0x7fae58110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 --2- 172.21.15.37:0/1820030683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58103660 0x7fae58103ae0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 --2- 172.21.15.37:0/1820030683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae58109670 0x7fae58109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 -- 172.21.15.37:0/1820030683 >> 172.21.15.37:0/1820030683 conn(0x7fae580ff150 msgr2=0x7fae58101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 -- 172.21.15.37:0/1820030683 shutdown_connections 2024-03-21T09:17:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.353+0000 7fae5cc00640 1 -- 172.21.15.37:0/1820030683 wait complete. 2024-03-21T09:17:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.354+0000 7fae5cc00640 1 Processor -- start 2024-03-21T09:17:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 -- start start 2024-03-21T09:17:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae58103660 0x7fae58199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 0x7fae5819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae58109670 0x7fae5819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae58113ad0 con 0x7fae58109670 2024-03-21T09:17:33.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae58113950 con 0x7fae58104020 2024-03-21T09:17:33.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae5cc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae58113c50 con 0x7fae58103660 2024-03-21T09:17:33.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae56a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 0x7fae5819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae57400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae58103660 0x7fae58199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae56a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 0x7fae5819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46300/0 (socket says 172.21.15.37:46300) 2024-03-21T09:17:33.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae56a00640 1 -- 172.21.15.37:0/213523121 learned_addr learned my addr 172.21.15.37:0/213523121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.355+0000 7fae57e00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae58109670 0x7fae5819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae56a00640 1 -- 172.21.15.37:0/213523121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae58103660 msgr2=0x7fae58199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae56a00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae58103660 0x7fae58199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae56a00640 1 -- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae58109670 msgr2=0x7fae5819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae56a00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae58109670 0x7fae5819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae56a00640 1 -- 172.21.15.37:0/213523121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5819ef80 con 0x7fae58104020 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae56a00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 0x7fae5819a4d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fae4402efd0 tx=0x7fae440042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae4fe00640 1 -- 172.21.15.37:0/213523121 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44041070 con 0x7fae58104020 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae4fe00640 1 -- 172.21.15.37:0/213523121 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae4402ccc0 con 0x7fae58104020 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5819f210 con 0x7fae58104020 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.356+0000 7fae4fe00640 1 -- 172.21.15.37:0/213523121 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44003d40 con 0x7fae58104020 2024-03-21T09:17:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.357+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae581a6b00 con 0x7fae58104020 2024-03-21T09:17:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae4fe00640 1 -- 172.21.15.37:0/213523121 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fae44044070 con 0x7fae58104020 2024-03-21T09:17:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 Processor -- start 2024-03-21T09:17:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 -- start start 2024-03-21T09:17:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689660670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689669790 0x55668966bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566895c4eb0 con 0x556689660290 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668956da60 con 0x556689660bb0 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.360+0000 7fae60ff1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566895c0f50 con 0x556689669790 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689660670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689660670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40084/0 (socket says 172.21.15.37:40084) 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46310/0 (socket says 172.21.15.37:46310) 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57e00640 1 -- 172.21.15.37:0/3643291202 learned_addr learned my addr 172.21.15.37:0/3643291202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae56a00640 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689669790 0x55668966bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 -- 172.21.15.37:0/3643291202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689669790 msgr2=0x55668966bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689669790 0x55668966bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 -- 172.21.15.37:0/3643291202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 msgr2=0x556689660670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689660670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 -- 172.21.15.37:0/3643291202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566893a17e0 con 0x556689660bb0 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.361+0000 7fae57400640 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669250 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fae48009c20 tx=0x7fae48011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e453d3d965a70411 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.362+0000 7fae43e00640 1 -- 172.21.15.37:0/3643291202 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48004050 con 0x556689660bb0 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.362+0000 7fae43e00640 1 -- 172.21.15.37:0/3643291202 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae48005500 con 0x556689660bb0 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.362+0000 7fae43e00640 1 -- 172.21.15.37:0/3643291202 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48004db0 con 0x556689660bb0 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.362+0000 7fae60ff1740 1 -- 172.21.15.37:0/3643291202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 msgr2=0x556689669250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.362+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669250 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fae48009c20 tx=0x7fae48011810 comp rx=0 tx=0).stop 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.362+0000 7fae4fe00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae3c0a3090 0x7fae3c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 -- 172.21.15.37:0/3643291202 shutdown_connections 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689669790 0x55668966bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669250 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3643291202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689660670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 -- 172.21.15.37:0/3643291202 >> 172.21.15.37:0/3643291202 conn(0x55668965a9c0 msgr2=0x55668965ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae4fe00640 1 -- 172.21.15.37:0/213523121 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fae440f1600 con 0x7fae58104020 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae57e00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae3c0a3090 0x7fae3c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 -- 172.21.15.37:0/3643291202 shutdown_connections 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 -- 172.21.15.37:0/3643291202 wait complete. 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.363+0000 7fae60ff1740 1 -- start start 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae57e00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae3c0a3090 0x7fae3c0a5550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae50005fd0 tx=0x7fae50009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 Processor -- start 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689669100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 0x556689666490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668966c580 con 0x556689660290 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556689580eb0 con 0x556689669790 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668966c700 con 0x556689660bb0 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae60ff1740 10 client.0 _ll_register_callbacks cb 0x556689575150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:33.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae56a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 0x556689666490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae57e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.364+0000 7fae56a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 0x556689666490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43728/0 (socket says 172.21.15.37:43728) 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 -- 172.21.15.37:0/1009910064 learned_addr learned my addr 172.21.15.37:0/1009910064 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae57400640 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689669100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 -- 172.21.15.37:0/1009910064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 msgr2=0x5566896669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 -- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 msgr2=0x556689669100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689669100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 -- 172.21.15.37:0/1009910064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556689401f70 con 0x556689660bb0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae56a00640 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 0x556689666490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae440bd400 tx=0x7fae44003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3224e29bdea00bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae40c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae40c00640 1 -- 172.21.15.37:0/1009910064 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4404b050 con 0x556689660bb0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae48005500 con 0x556689660bb0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae40c00640 1 -- 172.21.15.37:0/1009910064 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fae440eebd0 con 0x556689660bb0 2024-03-21T09:17:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.365+0000 7fae40c00640 1 -- 172.21.15.37:0/1009910064 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae440b99a0 con 0x556689660bb0 2024-03-21T09:17:33.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 10 client.26929 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:33.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556689299170 con 0x556689660bb0 2024-03-21T09:17:33.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae40c00640 1 -- 172.21.15.37:0/1009910064 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae48005500 con 0x556689660bb0 2024-03-21T09:17:33.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae40c00640 1 -- 172.21.15.37:0/1009910064 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae440f1190 con 0x556689660bb0 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 10 client.26929 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 20 client.26929 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556689679730 0x55668967bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 4 client.26929 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566895d23f0 con 0x556689679730 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.366+0000 7fae57400640 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556689679730 0x55668967bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.367+0000 7fae57400640 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556689679730 0x55668967bb60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dfc18abcb80dec8 server_cookie=b19d6268e0dcf0bb in_seq=0 out_seq=0 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.367+0000 7fae40c00640 10 client.26929 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae40c00640 1 -- 172.21.15.37:0/1009910064 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5566895d23f0 con 0x556689679730 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae40c00640 10 client.26929 handle_command_reply: tid=0 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 client.26929 shutdown 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 msgr2=0x556689666490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 0x556689666490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae440bd400 tx=0x7fae44003ee0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 shutdown_connections 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556689679730 0x55668967bb60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae40c00640 0 client.26929 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660bb0 0x556689666490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1009910064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660290 0x556689669100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 >> 172.21.15.37:0/1009910064 conn(0x55668965a9c0 msgr2=0x55668965c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 shutdown_connections 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.368+0000 7fae60ff1740 1 -- 172.21.15.37:0/1009910064 wait complete. 2024-03-21T09:17:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.369+0000 7fae60ff1740 20 client.26929 trim_cache size 0 max 16384 2024-03-21T09:17:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.386+0000 7fae60ff1740 1 Processor -- start 2024-03-21T09:17:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 -- start start 2024-03-21T09:17:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x556689668bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689669790 0x556689674e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581b2eb0 con 0x556689669790 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581b09b0 con 0x556689660bb0 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae60ff1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581aea50 con 0x556689660290 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae56a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae57400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x556689668bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689669790 0x556689674e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae56a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46334/0 (socket says 172.21.15.37:46334) 2024-03-21T09:17:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.387+0000 7fae56a00640 1 -- 172.21.15.37:0/1113899811 learned_addr learned my addr 172.21.15.37:0/1113899811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae56a00640 1 -- 172.21.15.37:0/1113899811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 msgr2=0x556689668bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae56a00640 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x556689668bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae56a00640 1 -- 172.21.15.37:0/1113899811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689669790 msgr2=0x556689674e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae56a00640 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689669790 0x556689674e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae56a00640 1 -- 172.21.15.37:0/1113899811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55668965a190 con 0x556689660bb0 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae56a00640 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fae440ba080 tx=0x7fae44005ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c737da952b06e55 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae43400640 1 -- 172.21.15.37:0/1113899811 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4404b070 con 0x556689660bb0 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae43400640 1 -- 172.21.15.37:0/1113899811 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae440c6110 con 0x556689660bb0 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.388+0000 7fae43400640 1 -- 172.21.15.37:0/1113899811 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae440bd060 con 0x556689660bb0 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 -- 172.21.15.37:0/1113899811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 msgr2=0x556689669100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fae440ba080 tx=0x7fae44005ba0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 -- 172.21.15.37:0/1113899811 shutdown_connections 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689669790 0x556689674e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689660bb0 0x556689669100 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 --2- 172.21.15.37:0/1113899811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x556689668bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 -- 172.21.15.37:0/1113899811 >> 172.21.15.37:0/1113899811 conn(0x55668965a9c0 msgr2=0x556689659a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 -- 172.21.15.37:0/1113899811 shutdown_connections 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 -- 172.21.15.37:0/1113899811 wait complete. 2024-03-21T09:17:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 -- start start 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.389+0000 7fae60ff1740 1 Processor -- start 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x55668966eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660bb0 0x55668966f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581af050 con 0x556689660bb0 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581b0350 con 0x556689669790 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581af690 con 0x556689660290 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae60ff1740 10 client.0 _ll_register_callbacks cb 0x7fae440045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae57400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x55668966eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae57400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x55668966eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43738/0 (socket says 172.21.15.37:43738) 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae57400640 1 -- 172.21.15.37:0/3829379362 learned_addr learned my addr 172.21.15.37:0/3829379362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae57e00640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.390+0000 7fae56a00640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660bb0 0x55668966f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57400640 1 -- 172.21.15.37:0/3829379362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 msgr2=0x5566896cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57400640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57400640 1 -- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660bb0 msgr2=0x55668966f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57400640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660bb0 0x55668966f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57400640 1 -- 172.21.15.37:0/3829379362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566896cf610 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57e00640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae57400640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x55668966eb10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fae48009c20 tx=0x7fae48009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c032f7dda3aeb76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae40c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae40c00640 1 -- 172.21.15.37:0/3829379362 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48031060 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae440c6110 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae40c00640 1 -- 172.21.15.37:0/3829379362 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fae48004740 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae40c00640 1 -- 172.21.15.37:0/3829379362 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae48038050 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae60ff1740 10 client.26935 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.391+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5566896d6a00 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae40c00640 1 -- 172.21.15.37:0/3829379362 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae440c6110 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae40c00640 1 -- 172.21.15.37:0/3829379362 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae48003960 con 0x556689660290 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae60ff1740 10 client.26935 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae60ff1740 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae60ff1740 20 client.26935 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566896d6ee0 0x5566896d9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae60ff1740 4 client.26935 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5566895cc740 con 0x5566896d6ee0 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.392+0000 7fae56a00640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566896d6ee0 0x5566896d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.393+0000 7fae56a00640 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566896d6ee0 0x5566896d9300 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2373ba56f7cf09d0 server_cookie=ea9931283474b8a in_seq=0 out_seq=0 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.393+0000 7fae40c00640 10 client.26935 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.394+0000 7fae40c00640 1 -- 172.21.15.37:0/3829379362 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5566895cc740 con 0x5566896d6ee0 2024-03-21T09:17:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.394+0000 7fae40c00640 10 client.26935 handle_command_reply: tid=0 2024-03-21T09:17:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.394+0000 7fae60ff1740 1 client.26935 shutdown 2024-03-21T09:17:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 msgr2=0x55668966eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x55668966eb10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fae48009c20 tx=0x7fae48009390 comp rx=0 tx=0).stop 2024-03-21T09:17:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 shutdown_connections 2024-03-21T09:17:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566896d6ee0 0x5566896d9300 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556689669790 0x5566896cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556689660bb0 0x55668966f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae40c00640 0 client.26935 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 --2- 172.21.15.37:0/3829379362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556689660290 0x55668966eb10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 >> 172.21.15.37:0/3829379362 conn(0x55668965a9c0 msgr2=0x556689660f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 shutdown_connections 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 1 -- 172.21.15.37:0/3829379362 wait complete. 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.395+0000 7fae60ff1740 20 client.26935 trim_cache size 0 max 16384 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:17:33.392 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "05502ad9-14ba-45d1-b6a1-7ce9c191331b", 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae3c0a3090 msgr2=0x7fae3c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae3c0a3090 0x7fae3c0a5550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae50005fd0 tx=0x7fae50009500 comp rx=0 tx=0).stop 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 msgr2=0x7fae5819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 0x7fae5819a4d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fae4402efd0 tx=0x7fae440042f0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 shutdown_connections 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae58109670 0x7fae5819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae3c0a3090 0x7fae3c0a5550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae58104020 0x7fae5819a4d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.396+0000 7fae5cc00640 1 --2- 172.21.15.37:0/213523121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae58103660 0x7fae58199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.397+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 >> 172.21.15.37:0/213523121 conn(0x7fae580ff150 msgr2=0x7fae580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.397+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 shutdown_connections 2024-03-21T09:17:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.397+0000 7fae5cc00640 1 -- 172.21.15.37:0/213523121 wait complete. 2024-03-21T09:17:33.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:17:33.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:17:33.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 -- 172.21.15.37:0/392680615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c1028c0 msgr2=0x7f072c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 --2- 172.21.15.37:0/392680615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c1028c0 0x7f072c102ca0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f071c009900 tx=0x7f071c02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 -- 172.21.15.37:0/392680615 shutdown_connections 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 --2- 172.21.15.37:0/392680615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c10bc90 0x7f072c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 --2- 172.21.15.37:0/392680615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c1031e0 0x7f072c10b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 --2- 172.21.15.37:0/392680615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c1028c0 0x7f072c102ca0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 -- 172.21.15.37:0/392680615 >> 172.21.15.37:0/392680615 conn(0x7f072c0776c0 msgr2=0x7f072c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 -- 172.21.15.37:0/392680615 shutdown_connections 2024-03-21T09:17:33.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.633+0000 7f0733000640 1 -- 172.21.15.37:0/392680615 wait complete. 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.634+0000 7f0733000640 1 Processor -- start 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 -- start start 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 0x7f072c100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c1031e0 0x7f072c100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c10bc90 0x7f072c100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c111a70 con 0x7f072c1031e0 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1118f0 con 0x7f072c10bc90 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0733000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c111bf0 con 0x7f072c1028c0 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f072be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 0x7f072c100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f072be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 0x7f072c100510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43764/0 (socket says 172.21.15.37:43764) 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f072be00640 1 -- 172.21.15.37:0/2326126084 learned_addr learned my addr 172.21.15.37:0/2326126084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f0731000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c10bc90 0x7f072c100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.635+0000 7f072b400640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c1031e0 0x7f072c100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f072be00640 1 -- 172.21.15.37:0/2326126084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c10bc90 msgr2=0x7f072c100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f072be00640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c10bc90 0x7f072c100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f072be00640 1 -- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c1031e0 msgr2=0x7f072c100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f072be00640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c1031e0 0x7f072c100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f072be00640 1 -- 172.21.15.37:0/2326126084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c1aa910 con 0x7f072c1028c0 2024-03-21T09:17:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f072be00640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 0x7f072c100510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f071c000c00 tx=0x7f071c0043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f0728c00640 1 -- 172.21.15.37:0/2326126084 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c042020 con 0x7f072c1028c0 2024-03-21T09:17:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f0728c00640 1 -- 172.21.15.37:0/2326126084 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f071c005000 con 0x7f072c1028c0 2024-03-21T09:17:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f0728c00640 1 -- 172.21.15.37:0/2326126084 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c003b20 con 0x7f072c1028c0 2024-03-21T09:17:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.636+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c1aaae0 con 0x7f072c1028c0 2024-03-21T09:17:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.637+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c1ab080 con 0x7f072c1028c0 2024-03-21T09:17:33.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.639+0000 7f0728c00640 1 -- 172.21.15.37:0/2326126084 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f071c045020 con 0x7f072c1028c0 2024-03-21T09:17:33.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 Processor -- start 2024-03-21T09:17:33.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 -- start start 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 0x562394c15950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c1f290 0x562394c21680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562394b7a250 con 0x562394c15f20 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562394b22a30 con 0x562394c1f290 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0735e29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562394b762f0 con 0x562394c16c00 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0731000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 0x562394c15950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0731000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 0x562394c15950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43774/0 (socket says 172.21.15.37:43774) 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f0731000640 1 -- 172.21.15.37:0/516916367 learned_addr learned my addr 172.21.15.37:0/516916367 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.640+0000 7f072be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f072b400640 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c1f290 0x562394c21680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0731000640 1 -- 172.21.15.37:0/516916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c1f290 msgr2=0x562394c21680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0731000640 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c1f290 0x562394c21680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0731000640 1 -- 172.21.15.37:0/516916367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 msgr2=0x562394c163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0731000640 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0731000640 1 -- 172.21.15.37:0/516916367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562394b0d690 con 0x562394c16c00 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0728c00640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07100a3110 0x7f07100a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0731000640 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 0x562394c15950 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0724009f30 tx=0x7f07240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783da64ecdee436b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.641+0000 7f0728c00640 1 -- 172.21.15.37:0/2326126084 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f071c0f1770 con 0x7f072c1028c0 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f072be00640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07100a3110 0x7f07100a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f0721600640 1 -- 172.21.15.37:0/516916367 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072401e070 con 0x562394c16c00 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f0721600640 1 -- 172.21.15.37:0/516916367 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0724003c30 con 0x562394c16c00 2024-03-21T09:17:33.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f0721600640 1 -- 172.21.15.37:0/516916367 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072400f980 con 0x562394c16c00 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f0735e29740 1 -- 172.21.15.37:0/516916367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 msgr2=0x562394c15950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f072be00640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07100a3110 0x7f07100a55d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x562394c17150 tx=0x7f071c0f7530 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.642+0000 7f0735e29740 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 0x562394c15950 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0724009f30 tx=0x7f07240118d0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 -- 172.21.15.37:0/516916367 shutdown_connections 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c1f290 0x562394c21680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 --2- 172.21.15.37:0/516916367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c16c00 0x562394c15950 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 -- 172.21.15.37:0/516916367 >> 172.21.15.37:0/516916367 conn(0x562394bdd200 msgr2=0x562394c10970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 -- 172.21.15.37:0/516916367 shutdown_connections 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 -- 172.21.15.37:0/516916367 wait complete. 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 -- start start 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.643+0000 7f0735e29740 1 Processor -- start 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c1c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562394c22100 con 0x562394c15f20 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562394b35e80 con 0x562394c16c00 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562394c22280 con 0x562394c1f290 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0735e29740 10 client.0 _ll_register_callbacks cb 0x562394b48130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46380/0 (socket says 172.21.15.37:46380) 2024-03-21T09:17:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 -- 172.21.15.37:0/980211474 learned_addr learned my addr 172.21.15.37:0/980211474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0731000640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0731000640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c1c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 -- 172.21.15.37:0/980211474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 msgr2=0x562394c1c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c1c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 -- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 msgr2=0x562394c1e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f072b400640 1 -- 172.21.15.37:0/980211474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623949587d0 con 0x562394c16c00 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.644+0000 7f0731000640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c1c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f072b400640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1bd00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f071800bf30 tx=0x7f07180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9816db5ba905206 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0716000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0716000640 1 -- 172.21.15.37:0/980211474 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718004020 con 0x562394c16c00 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0716000640 1 -- 172.21.15.37:0/980211474 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f07180041c0 con 0x562394c16c00 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0724003c30 con 0x562394c16c00 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0716000640 1 -- 172.21.15.37:0/980211474 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0718024050 con 0x562394c16c00 2024-03-21T09:17:33.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0735e29740 10 client.26921 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.645+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56239488e3f0 con 0x562394c16c00 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0716000640 1 -- 172.21.15.37:0/980211474 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0724003c30 con 0x562394c16c00 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0716000640 1 -- 172.21.15.37:0/980211474 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0718003830 con 0x562394c16c00 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0735e29740 10 client.26921 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0735e29740 10 client.26921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0735e29740 20 client.26921 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0735e29740 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c2f310 0x562394c31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0735e29740 4 client.26921 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562394b87790 con 0x562394c2f310 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.646+0000 7f0731000640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c2f310 0x562394c31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.647+0000 7f0731000640 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c2f310 0x562394c31700 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71e0450aae52961 server_cookie=aa37a7a12ee09354 in_seq=0 out_seq=0 2024-03-21T09:17:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.647+0000 7f0716000640 10 client.26921 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0716000640 1 -- 172.21.15.37:0/980211474 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562394b87790 con 0x562394c2f310 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0716000640 10 client.26921 handle_command_reply: tid=0 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 client.26921 shutdown 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 msgr2=0x562394c1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1bd00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f071800bf30 tx=0x7f07180118d0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 shutdown_connections 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c1c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c2f310 0x562394c31700 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1bd00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 --2- 172.21.15.37:0/980211474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0716000640 0 client.26921 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 >> 172.21.15.37:0/980211474 conn(0x562394bdd200 msgr2=0x562394c126c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 shutdown_connections 2024-03-21T09:17:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 1 -- 172.21.15.37:0/980211474 wait complete. 2024-03-21T09:17:33.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.648+0000 7f0735e29740 20 client.26921 trim_cache size 0 max 16384 2024-03-21T09:17:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 Processor -- start 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 -- start start 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c20680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1b74f0 con 0x562394c15f20 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1b4ff0 con 0x562394c16c00 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.680+0000 7f0735e29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1b3090 con 0x562394c1f290 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46394/0 (socket says 172.21.15.37:46394) 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 -- 172.21.15.37:0/2260363825 learned_addr learned my addr 172.21.15.37:0/2260363825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f0731000640 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f0731000640 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 -- 172.21.15.37:0/2260363825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 msgr2=0x562394c20680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c20680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 -- 172.21.15.37:0/2260363825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 msgr2=0x562394c1e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 -- 172.21.15.37:0/2260363825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56239484e0f0 con 0x562394c16c00 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f0731000640 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:17:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.681+0000 7f072b400640 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1e970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0718003cd0 tx=0x7f0718005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6648280cfd5bebeb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0720c00640 1 -- 172.21.15.37:0/2260363825 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718000e60 con 0x562394c16c00 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0720c00640 1 -- 172.21.15.37:0/2260363825 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0718005040 con 0x562394c16c00 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0720c00640 1 -- 172.21.15.37:0/2260363825 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071802c070 con 0x562394c16c00 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0735e29740 1 -- 172.21.15.37:0/2260363825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 msgr2=0x562394c1e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0735e29740 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1e970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0718003cd0 tx=0x7f0718005900 comp rx=0 tx=0).stop 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0735e29740 1 -- 172.21.15.37:0/2260363825 shutdown_connections 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0735e29740 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c20680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0735e29740 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c1e970 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.682+0000 7f0735e29740 1 --2- 172.21.15.37:0/2260363825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c1e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.683+0000 7f0735e29740 1 -- 172.21.15.37:0/2260363825 >> 172.21.15.37:0/2260363825 conn(0x562394bdd200 msgr2=0x562394c1a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.683+0000 7f0735e29740 1 -- 172.21.15.37:0/2260363825 shutdown_connections 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.683+0000 7f0735e29740 1 -- 172.21.15.37:0/2260363825 wait complete. 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.683+0000 7f0735e29740 1 -- start start 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.683+0000 7f0735e29740 1 Processor -- start 2024-03-21T09:17:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c19280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1b3690 con 0x562394c15f20 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1b4990 con 0x562394c16c00 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1b3cd0 con 0x562394c1f290 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0735e29740 10 client.0 _ll_register_callbacks cb 0x7f071800bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0731000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c18d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:46408/0 (socket says 172.21.15.37:46408) 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 -- 172.21.15.37:0/3674493732 learned_addr learned my addr 172.21.15.37:0/3674493732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0731000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c26f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40168/0 (socket says 172.21.15.37:40168) 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f0731000640 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c19280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 -- 172.21.15.37:0/3674493732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 msgr2=0x562394c19280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c19280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 -- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 msgr2=0x562394c26f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c26f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.684+0000 7f072b400640 1 -- 172.21.15.37:0/3674493732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562394c197c0 con 0x562394c16c00 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f072b400640 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c18d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0718003d20 tx=0x7f0718003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133ec213ad746987 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0716000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0716000640 1 -- 172.21.15.37:0/3674493732 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071802c040 con 0x562394c16c00 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0718005040 con 0x562394c16c00 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0716000640 1 -- 172.21.15.37:0/3674493732 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0718005400 con 0x562394c16c00 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0716000640 1 -- 172.21.15.37:0/3674493732 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f071803f050 con 0x562394c16c00 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0735e29740 10 client.26927 fetch_fsmap learned FSMap version 30 2024-03-21T09:17:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.685+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562394c8b720 con 0x562394c16c00 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0716000640 1 -- 172.21.15.37:0/3674493732 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0718005040 con 0x562394c16c00 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0716000640 1 -- 172.21.15.37:0/3674493732 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0718037020 con 0x562394c16c00 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0735e29740 10 client.26927 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0735e29740 10 client.26927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0735e29740 20 client.26927 populate_metadata read hostname 'smithi037' 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0735e29740 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c8bd50 0x562394c8e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0735e29740 4 client.26927 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562394b81ae0 con 0x562394c8bd50 2024-03-21T09:17:33.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.686+0000 7f0731000640 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c8bd50 0x562394c8e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.687+0000 7f0731000640 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c8bd50 0x562394c8e170 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7061d1348a561232 server_cookie=a560f6016bec97c3 in_seq=0 out_seq=0 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.687+0000 7f0716000640 10 client.26927 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.687+0000 7f0716000640 1 -- 172.21.15.37:0/3674493732 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562394b81ae0 con 0x562394c8bd50 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.687+0000 7f0716000640 10 client.26927 handle_command_reply: tid=0 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.687+0000 7f0735e29740 1 client.26927 shutdown 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 msgr2=0x562394c18d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c18d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0718003d20 tx=0x7f0718003820 comp rx=0 tx=0).stop 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 shutdown_connections 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562394c1f290 0x562394c19280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562394c8bd50 0x562394c8e170 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562394c16c00 0x562394c18d40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 --2- 172.21.15.37:0/3674493732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562394c15f20 0x562394c26f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0716000640 0 client.26927 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:17:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 >> 172.21.15.37:0/3674493732 conn(0x562394bdd200 msgr2=0x562394c11d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 shutdown_connections 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 1 -- 172.21.15.37:0/3674493732 wait complete. 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.688+0000 7f0735e29740 20 client.26927 trim_cache size 0 max 16384 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "05502ad9-14ba-45d1-b6a1-7ce9c191331b": { 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "05502ad9-14ba-45d1-b6a1-7ce9c191331b", 2024-03-21T09:17:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:17:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:17:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:17:33.687 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:17:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07100a3110 msgr2=0x7f07100a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07100a3110 0x7f07100a55d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x562394c17150 tx=0x7f071c0f7530 comp rx=0 tx=0).stop 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 msgr2=0x7f072c100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 0x7f072c100510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f071c000c00 tx=0x7f071c0043c0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 shutdown_connections 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f07100a3110 0x7f07100a55d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f072c10bc90 0x7f072c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c1031e0 0x7f072c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 --2- 172.21.15.37:0/2326126084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f072c1028c0 0x7f072c100510 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 >> 172.21.15.37:0/2326126084 conn(0x7f072c0776c0 msgr2=0x7f072c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 shutdown_connections 2024-03-21T09:17:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:17:33.689+0000 7f0733000640 1 -- 172.21.15.37:0/2326126084 wait complete. 2024-03-21T09:17:33.710 INFO:tasks.cephfs.filesystem:scrub status for tag:05502ad9-14ba-45d1-b6a1-7ce9c191331b - {'path': '/', 'tag': '05502ad9-14ba-45d1-b6a1-7ce9c191331b', 'options': 'recursive,force'} 2024-03-21T09:17:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:34 smithi071 ceph-mon[36597]: pgmap v164: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:34 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:17:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:34 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:34 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:17:34.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:34 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:34 smithi037 ceph-mon[31830]: pgmap v164: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:34.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:34 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:17:34.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:34 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:34.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:34 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:17:34.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:34 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:34 smithi053 ceph-mon[36169]: pgmap v164: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:34 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:17:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:34 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:34 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:17:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:34 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:17:36.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:36 smithi071 ceph-mon[36597]: pgmap v165: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:36.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:36 smithi037 ceph-mon[31830]: pgmap v165: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:36.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:36 smithi053 ceph-mon[36169]: pgmap v165: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:37.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:37 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:17:37.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:37 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:17:37.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:37 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:17:38.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:38 smithi053 ceph-mon[36169]: pgmap v166: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:38.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:38 smithi071 ceph-mon[36597]: pgmap v166: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:38.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:38 smithi037 ceph-mon[31830]: pgmap v166: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:40.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:40 smithi071 ceph-mon[36597]: pgmap v167: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:40.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:40 smithi037 ceph-mon[31830]: pgmap v167: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:40.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:40 smithi053 ceph-mon[36169]: pgmap v167: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:42.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:42 smithi071 ceph-mon[36597]: pgmap v168: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:42.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:42 smithi037 ceph-mon[31830]: pgmap v168: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:42.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:42 smithi053 ceph-mon[36169]: pgmap v168: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:44.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:44 smithi071 ceph-mon[36597]: pgmap v169: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:44.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:44 smithi037 ceph-mon[31830]: pgmap v169: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:44.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:44 smithi053 ceph-mon[36169]: pgmap v169: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:46.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:46 smithi071 ceph-mon[36597]: pgmap v170: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:46.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:46 smithi037 ceph-mon[31830]: pgmap v170: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:46.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:46 smithi053 ceph-mon[36169]: pgmap v170: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:17:48.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:48 smithi071 ceph-mon[36597]: pgmap v171: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:48.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:48 smithi037 ceph-mon[31830]: pgmap v171: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:48.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:48 smithi053 ceph-mon[36169]: pgmap v171: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:17:50.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:50 smithi071 ceph-mon[36597]: pgmap v172: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:50.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:50 smithi037 ceph-mon[31830]: pgmap v172: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:50.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:50 smithi053 ceph-mon[36169]: pgmap v172: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:52.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:52 smithi071 ceph-mon[36597]: pgmap v173: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:52.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:52 smithi037 ceph-mon[31830]: pgmap v173: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:52 smithi053 ceph-mon[36169]: pgmap v173: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:54.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:54 smithi071 ceph-mon[36597]: pgmap v174: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:54.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:54 smithi037 ceph-mon[31830]: pgmap v174: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:54.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:54 smithi053 ceph-mon[36169]: pgmap v174: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:56.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:56 smithi071 ceph-mon[36597]: pgmap v175: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:56.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:56 smithi037 ceph-mon[31830]: pgmap v175: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:56 smithi053 ceph-mon[36169]: pgmap v175: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:58.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:58 smithi071 ceph-mon[36597]: pgmap v176: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:58.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:58 smithi037 ceph-mon[31830]: pgmap v176: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:58.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:58 smithi053 ceph-mon[36169]: pgmap v176: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:17:59.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:17:59 smithi037 ceph-mon[31830]: pgmap v177: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:59.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:17:59 smithi053 ceph-mon[36169]: pgmap v177: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:17:59.853 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:17:59 smithi071 ceph-mon[36597]: pgmap v177: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:02.555 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:02 smithi071 ceph-mon[36597]: pgmap v178: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:02.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:02 smithi037 ceph-mon[31830]: pgmap v178: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:02 smithi053 ceph-mon[36169]: pgmap v178: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:03.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:18:03.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:18:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 -- 172.21.15.37:0/1163078203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40101e60 msgr2=0x7f8e4010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 --2- 172.21.15.37:0/1163078203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40101e60 0x7f8e4010e320 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009cd0 tx=0x7f8e3802c0e0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 -- 172.21.15.37:0/1163078203 shutdown_connections 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 --2- 172.21.15.37:0/1163078203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40101e60 0x7f8e4010e320 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 --2- 172.21.15.37:0/1163078203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e40101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 --2- 172.21.15.37:0/1163078203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e40107560 0x7f8e40107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 -- 172.21.15.37:0/1163078203 >> 172.21.15.37:0/1163078203 conn(0x7f8e400fcfb0 msgr2=0x7f8e400ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 -- 172.21.15.37:0/1163078203 shutdown_connections 2024-03-21T09:18:03.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.956+0000 7f8e46a00640 1 -- 172.21.15.37:0/1163078203 wait complete. 2024-03-21T09:18:03.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.958+0000 7f8e46a00640 1 Processor -- start 2024-03-21T09:18:03.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.958+0000 7f8e46a00640 1 -- start start 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.958+0000 7f8e46a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e4019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e46a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e40101e60 0x7f8e4019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e46a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40107560 0x7f8e40197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e46a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40111820 con 0x7f8e40101e60 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e46a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401116a0 con 0x7f8e40107560 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e46a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401119a0 con 0x7f8e401014a0 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e4019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e44c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e40101e60 0x7f8e4019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e4019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54146/0 (socket says 172.21.15.37:54146) 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 -- 172.21.15.37:0/57118797 learned_addr learned my addr 172.21.15.37:0/57118797 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e46000640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40107560 0x7f8e40197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 -- 172.21.15.37:0/57118797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40107560 msgr2=0x7f8e40197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40107560 0x7f8e40197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 -- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e40101e60 msgr2=0x7f8e4019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e40101e60 0x7f8e4019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.959+0000 7f8e45600640 1 -- 172.21.15.37:0/57118797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40198240 con 0x7f8e401014a0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.960+0000 7f8e45600640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e4019d910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400ea60 tx=0x7f8e3400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.960+0000 7f8e3e000640 1 -- 172.21.15.37:0/57118797 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400cde0 con 0x7f8e401014a0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.960+0000 7f8e3e000640 1 -- 172.21.15.37:0/57118797 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e340035f0 con 0x7f8e401014a0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.960+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40198530 con 0x7f8e401014a0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.960+0000 7f8e3e000640 1 -- 172.21.15.37:0/57118797 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34010670 con 0x7f8e401014a0 2024-03-21T09:18:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.960+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e401a47d0 con 0x7f8e401014a0 2024-03-21T09:18:03.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.963+0000 7f8e4adfc740 1 Processor -- start 2024-03-21T09:18:03.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 -- start start 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce42531e0 0x55fce4251f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce4252980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 0x55fce425dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce41b2250 con 0x55fce42531e0 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce415aa30 con 0x55fce4252500 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce41ae2f0 con 0x55fce425b870 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e46000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce42531e0 0x55fce4251f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e44c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 0x55fce425dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e44c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 0x55fce425dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54156/0 (socket says 172.21.15.37:54156) 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.964+0000 7f8e44c00640 1 -- 172.21.15.37:0/621398918 learned_addr learned my addr 172.21.15.37:0/621398918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e44c00640 1 -- 172.21.15.37:0/621398918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 msgr2=0x55fce4252980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e44c00640 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce4252980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e44c00640 1 -- 172.21.15.37:0/621398918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce42531e0 msgr2=0x55fce4251f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e44c00640 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce42531e0 0x55fce4251f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e44c00640 1 -- 172.21.15.37:0/621398918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce4145690 con 0x55fce425b870 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e3e000640 1 -- 172.21.15.37:0/57118797 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8e340037a0 con 0x7f8e401014a0 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e44c00640 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 0x55fce425dc60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009c60 tx=0x7f8e2c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f5946245ebb17c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e32600640 1 -- 172.21.15.37:0/621398918 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004230 con 0x55fce425b870 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e32600640 1 -- 172.21.15.37:0/621398918 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c0043d0 con 0x55fce425b870 2024-03-21T09:18:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.965+0000 7f8e32600640 1 -- 172.21.15.37:0/621398918 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004de0 con 0x55fce425b870 2024-03-21T09:18:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 -- 172.21.15.37:0/621398918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 msgr2=0x55fce425dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 0x55fce425dc60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009c60 tx=0x7f8e2c011810 comp rx=0 tx=0).stop 2024-03-21T09:18:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 -- 172.21.15.37:0/621398918 shutdown_connections 2024-03-21T09:18:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce425b870 0x55fce425dc60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce4252980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/621398918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce42531e0 0x55fce4251f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 -- 172.21.15.37:0/621398918 >> 172.21.15.37:0/621398918 conn(0x55fce4215200 msgr2=0x55fce424cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 -- 172.21.15.37:0/621398918 shutdown_connections 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e4adfc740 1 -- 172.21.15.37:0/621398918 wait complete. 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.966+0000 7f8e3e000640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8e280a3020 0x7f8e280a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.967+0000 7f8e3e000640 1 -- 172.21.15.37:0/57118797 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e340cfde0 con 0x7f8e401014a0 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.967+0000 7f8e44c00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8e280a3020 0x7f8e280a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.967+0000 7f8e4adfc740 1 -- start start 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.967+0000 7f8e4adfc740 1 Processor -- start 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e44c00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8e280a3020 0x7f8e280a54e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0099f0 tx=0x7f8e2c0096a0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 0x55fce425afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce42531e0 0x55fce4258350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4258890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce425e6e0 con 0x55fce425b870 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce416de80 con 0x55fce42531e0 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce425e860 con 0x55fce4252500 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e4adfc740 10 client.0 _ll_register_callbacks cb 0x55fce4180130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 0x55fce425afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e44c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce42531e0 0x55fce4258350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 0x55fce425afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54164/0 (socket says 172.21.15.37:54164) 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e44c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce42531e0 0x55fce4258350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45512/0 (socket says 172.21.15.37:45512) 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 -- 172.21.15.37:0/243531303 learned_addr learned my addr 172.21.15.37:0/243531303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e46000640 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4258890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 -- 172.21.15.37:0/243531303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce42531e0 msgr2=0x55fce4258350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce42531e0 0x55fce4258350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 -- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 msgr2=0x55fce4258890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4258890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.968+0000 7f8e45600640 1 -- 172.21.15.37:0/243531303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce3f907d0 con 0x55fce4252500 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e45600640 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 0x55fce425afc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400c760 tx=0x7f8e3409b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c650139a4fe9d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e26a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e26a00640 1 -- 172.21.15.37:0/243531303 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3409ff80 con 0x55fce4252500 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e26a00640 1 -- 172.21.15.37:0/243531303 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e340984a0 con 0x55fce4252500 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e2c0043d0 con 0x55fce4252500 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e26a00640 1 -- 172.21.15.37:0/243531303 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e340a4840 con 0x55fce4252500 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e4adfc740 10 client.26962 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fce3ec63f0 con 0x55fce4252500 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.969+0000 7f8e26a00640 1 -- 172.21.15.37:0/243531303 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c0043d0 con 0x55fce4252500 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e26a00640 1 -- 172.21.15.37:0/243531303 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8e3409b310 con 0x55fce4252500 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e4adfc740 10 client.26962 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e4adfc740 10 client.26962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e4adfc740 20 client.26962 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce426b800 0x55fce426dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e4adfc740 4 client.26962 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fce41bf790 con 0x55fce426b800 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.970+0000 7f8e44c00640 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce426b800 0x55fce426dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.971+0000 7f8e44c00640 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce426b800 0x55fce426dbf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4631d30f4968fdef server_cookie=f0441c2dce1f77e3 in_seq=0 out_seq=0 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.971+0000 7f8e26a00640 10 client.26962 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.971+0000 7f8e26a00640 1 -- 172.21.15.37:0/243531303 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fce41bf790 con 0x55fce426b800 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.971+0000 7f8e26a00640 10 client.26962 handle_command_reply: tid=0 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 client.26962 shutdown 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 msgr2=0x55fce425afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 0x55fce425afc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400c760 tx=0x7f8e3409b790 comp rx=0 tx=0).stop 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 shutdown_connections 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4258890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce426b800 0x55fce426dbf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce42531e0 0x55fce4258350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/243531303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce4252500 0x55fce425afc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 >> 172.21.15.37:0/243531303 conn(0x55fce4215200 msgr2=0x55fce424eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e26a00640 0 client.26962 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:03.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 shutdown_connections 2024-03-21T09:18:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 1 -- 172.21.15.37:0/243531303 wait complete. 2024-03-21T09:18:03.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.972+0000 7f8e4adfc740 20 client.26962 trim_cache size 0 max 16384 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.995+0000 7f8e4adfc740 1 Processor -- start 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 -- start start 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce4252500 0x55fce425aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce425afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 0x55fce425cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401aea20 con 0x55fce4252500 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401ac520 con 0x55fce425b870 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401aa5c0 con 0x55fce42531e0 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e46000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 0x55fce425cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e44c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce425afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e45600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce4252500 0x55fce425aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e46000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 0x55fce425cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45520/0 (socket says 172.21.15.37:45520) 2024-03-21T09:18:03.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.996+0000 7f8e46000640 1 -- 172.21.15.37:0/3129230377 learned_addr learned my addr 172.21.15.37:0/3129230377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e46000640 1 -- 172.21.15.37:0/3129230377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 msgr2=0x55fce425afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e46000640 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce425afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e46000640 1 -- 172.21.15.37:0/3129230377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce4252500 msgr2=0x55fce425aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e46000640 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce4252500 0x55fce425aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e46000640 1 -- 172.21.15.37:0/3129230377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce3e860f0 con 0x55fce425b870 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e44c00640 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce425afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e46000640 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 0x55fce425cc60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8e38004af0 tx=0x7f8e38005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a2d63f42b92c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:03.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e31c00640 1 -- 172.21.15.37:0/3129230377 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38039040 con 0x55fce425b870 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e31c00640 1 -- 172.21.15.37:0/3129230377 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e38032e70 con 0x55fce425b870 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.997+0000 7f8e31c00640 1 -- 172.21.15.37:0/3129230377 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380042a0 con 0x55fce425b870 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 -- 172.21.15.37:0/3129230377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 msgr2=0x55fce425cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 0x55fce425cc60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8e38004af0 tx=0x7f8e38005d80 comp rx=0 tx=0).stop 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 -- 172.21.15.37:0/3129230377 shutdown_connections 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce425b870 0x55fce425cc60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce425afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/3129230377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce4252500 0x55fce425aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 -- 172.21.15.37:0/3129230377 >> 172.21.15.37:0/3129230377 conn(0x55fce4215200 msgr2=0x55fce424ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 -- 172.21.15.37:0/3129230377 shutdown_connections 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.998+0000 7f8e4adfc740 1 -- 172.21.15.37:0/3129230377 wait complete. 2024-03-21T09:18:03.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 -- start start 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 Processor -- start 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce42555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce4255b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4256070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401aabc0 con 0x55fce425b870 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401abec0 con 0x55fce4252500 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401ab200 con 0x55fce42531e0 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e4adfc740 10 client.0 _ll_register_callbacks cb 0x7f8e3400e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:03.999+0000 7f8e45600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce42555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce42555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45536/0 (socket says 172.21.15.37:45536) 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 -- 172.21.15.37:0/4260451595 learned_addr learned my addr 172.21.15.37:0/4260451595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:03.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e44c00640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce4255b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e46000640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4256070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 -- 172.21.15.37:0/4260451595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 msgr2=0x55fce4255b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce4255b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 -- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 msgr2=0x55fce4256070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4256070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 -- 172.21.15.37:0/4260451595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce425c300 con 0x55fce4252500 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e45600640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce42555f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e34098800 tx=0x7f8e3409b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548d22a92989dd2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e26a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e26a00640 1 -- 172.21.15.37:0/4260451595 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3409b480 con 0x55fce4252500 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e26a00640 1 -- 172.21.15.37:0/4260451595 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e34097cc0 con 0x55fce4252500 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.000+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fce41b9ae0 con 0x55fce4252500 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e26a00640 1 -- 172.21.15.37:0/4260451595 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3409c530 con 0x55fce4252500 2024-03-21T09:18:04.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 10 client.26942 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fce42c75d0 con 0x55fce4252500 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e26a00640 1 -- 172.21.15.37:0/4260451595 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fce41b9ae0 con 0x55fce4252500 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e26a00640 1 -- 172.21.15.37:0/4260451595 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8e340baba0 con 0x55fce4252500 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 10 client.26942 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 20 client.26942 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce42c7c70 0x55fce42ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 4 client.26942 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fce41373a0 con 0x55fce42c7c70 2024-03-21T09:18:04.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.001+0000 7f8e44c00640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce42c7c70 0x55fce42ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e44c00640 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce42c7c70 0x55fce42ca090 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=839c32bff9213e0b server_cookie=1b0adca1e00b0a38 in_seq=0 out_seq=0 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e26a00640 10 client.26942 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e26a00640 1 -- 172.21.15.37:0/4260451595 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fce41373a0 con 0x55fce42c7c70 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e26a00640 10 client.26942 handle_command_reply: tid=0 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e4adfc740 1 client.26942 shutdown 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 msgr2=0x55fce42555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce42555f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e34098800 tx=0x7f8e3409b8c0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 shutdown_connections 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.002+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fce425b870 0x55fce4256070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce42531e0 0x55fce4255b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fce42c7c70 0x55fce42ca090 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 1 --2- 172.21.15.37:0/4260451595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fce4252500 0x55fce42555f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e26a00640 0 client.26942 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 >> 172.21.15.37:0/4260451595 conn(0x55fce4215200 msgr2=0x55fce4256c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 shutdown_connections 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 1 -- 172.21.15.37:0/4260451595 wait complete. 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.003+0000 7f8e4adfc740 20 client.26942 trim_cache size 0 max 16384 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:18:04.004 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:18:04.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8e280a3020 msgr2=0x7f8e280a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8e280a3020 0x7f8e280a54e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0099f0 tx=0x7f8e2c0096a0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 msgr2=0x7f8e4019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e4019d910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400ea60 tx=0x7f8e3400ef20 comp rx=0 tx=0).stop 2024-03-21T09:18:04.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 shutdown_connections 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8e280a3020 0x7f8e280a54e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e40107560 0x7f8e40197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e40101e60 0x7f8e4019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 --2- 172.21.15.37:0/57118797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e401014a0 0x7f8e4019d910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 >> 172.21.15.37:0/57118797 conn(0x7f8e400fcfb0 msgr2=0x7f8e400fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 shutdown_connections 2024-03-21T09:18:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.004+0000 7f8e46a00640 1 -- 172.21.15.37:0/57118797 wait complete. 2024-03-21T09:18:04.017 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:18:04.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:18:04.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1281667146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101550 msgr2=0x7fcfd41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1281667146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101550 0x7fcfd41019d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009930 tx=0x7fcfc002bc30 comp rx=0 tx=0).stop 2024-03-21T09:18:04.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1281667146 shutdown_connections 2024-03-21T09:18:04.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1281667146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd4101f10 0x7fcfd410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1281667146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101550 0x7fcfd41019d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1281667146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4107610 0x7fcfd41079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1281667146 >> 172.21.15.37:0/1281667146 conn(0x7fcfd40fd060 msgr2=0x7fcfd40ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1281667146 shutdown_connections 2024-03-21T09:18:04.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.222+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1281667146 wait complete. 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.223+0000 7fcfd3e00640 1 Processor -- start 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.223+0000 7fcfd3e00640 1 -- start start 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 0x7fcfd4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101f10 0x7fcfd419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd4107610 0x7fcfd419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4111920 con 0x7fcfd4107610 2024-03-21T09:18:04.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41117a0 con 0x7fcfd4101550 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4111aa0 con 0x7fcfd4101f10 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 0x7fcfd4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 0x7fcfd4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45556/0 (socket says 172.21.15.37:45556) 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd2a00640 1 -- 172.21.15.37:0/1268786774 learned_addr learned my addr 172.21.15.37:0/1268786774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd2000640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101f10 0x7fcfd419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.224+0000 7fcfd3400640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd4107610 0x7fcfd419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2000640 1 -- 172.21.15.37:0/1268786774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 msgr2=0x7fcfd4199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2000640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 0x7fcfd4199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2000640 1 -- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd4107610 msgr2=0x7fcfd419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2000640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd4107610 0x7fcfd419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2000640 1 -- 172.21.15.37:0/1268786774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd419efa0 con 0x7fcfd4101f10 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2a00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 0x7fcfd4199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd2000640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101f10 0x7fcfd419a4f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0036040 tx=0x7fcfc0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0043020 con 0x7fcfd4101f10 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfc00054e0 con 0x7fcfd4101f10 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd419f230 con 0x7fcfd4101f10 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.225+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0003b80 con 0x7fcfd4101f10 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.226+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd41a6b20 con 0x7fcfd4101f10 2024-03-21T09:18:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.227+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd404f470 con 0x7fcfd4101f10 2024-03-21T09:18:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.228+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcfc0046020 con 0x7fcfd4101f10 2024-03-21T09:18:04.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.231+0000 7fcfcb400640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcfb80a3100 0x7fcfb80a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.231+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcfc00f19e0 con 0x7fcfd4101f10 2024-03-21T09:18:04.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.231+0000 7fcfd2a00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcfb80a3100 0x7fcfb80a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.232+0000 7fcfd2a00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcfb80a3100 0x7fcfb80a55c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011440 tx=0x7fcfc4005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.233+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcfc004d020 con 0x7fcfd4101f10 2024-03-21T09:18:04.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:04 smithi037 ceph-mon[31830]: pgmap v179: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:18:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.538+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfd41a6cf0 con 0x7fcfd4101f10 2024-03-21T09:18:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.539+0000 7fcfcb400640 1 -- 172.21.15.37:0/1268786774 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fcfc00be1e0 con 0x7fcfd4101f10 2024-03-21T09:18:04.535 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:18:04.535 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:18:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:18:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.541+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcfb80a3100 msgr2=0x7fcfb80a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.541+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcfb80a3100 0x7fcfb80a55c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011440 tx=0x7fcfc4005f90 comp rx=0 tx=0).stop 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.541+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101f10 msgr2=0x7fcfd419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.541+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101f10 0x7fcfd419a4f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0036040 tx=0x7fcfc0004360 comp rx=0 tx=0).stop 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 shutdown_connections 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd4107610 0x7fcfd419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4101f10 0x7fcfd419a4f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcfb80a3100 0x7fcfb80a55c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 --2- 172.21.15.37:0/1268786774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4101550 0x7fcfd4199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 >> 172.21.15.37:0/1268786774 conn(0x7fcfd40fd060 msgr2=0x7fcfd40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 shutdown_connections 2024-03-21T09:18:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.542+0000 7fcfd3e00640 1 -- 172.21.15.37:0/1268786774 wait complete. 2024-03-21T09:18:04.558 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:18:04.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:18:04.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:04 smithi071 ceph-mon[36597]: pgmap v179: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:18:04.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:04 smithi053 ceph-mon[36169]: pgmap v179: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-21T09:18:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 -- 172.21.15.37:0/2809384747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 msgr2=0x7f3ab0101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 --2- 172.21.15.37:0/2809384747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0101940 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009930 tx=0x7f3a9c02bc20 comp rx=0 tx=0).stop 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 -- 172.21.15.37:0/2809384747 shutdown_connections 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 --2- 172.21.15.37:0/2809384747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab0101e80 0x7f3ab010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 --2- 172.21.15.37:0/2809384747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0101940 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 --2- 172.21.15.37:0/2809384747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0107580 0x7f3ab0107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.802+0000 7f3aafe00640 1 -- 172.21.15.37:0/2809384747 >> 172.21.15.37:0/2809384747 conn(0x7f3ab00fcfb0 msgr2=0x7f3ab00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.803+0000 7f3aafe00640 1 -- 172.21.15.37:0/2809384747 shutdown_connections 2024-03-21T09:18:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.803+0000 7f3aafe00640 1 -- 172.21.15.37:0/2809384747 wait complete. 2024-03-21T09:18:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.804+0000 7f3aafe00640 1 Processor -- start 2024-03-21T09:18:04.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.804+0000 7f3aafe00640 1 -- start start 2024-03-21T09:18:04.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aafe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aafe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab0101e80 0x7f3ab0108890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aafe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0107580 0x7f3ab0108dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aafe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01118b0 con 0x7f3ab0107580 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aafe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0111730 con 0x7f3ab0101e80 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aafe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0111a30 con 0x7f3ab01014c0 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aaea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aaea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54216/0 (socket says 172.21.15.37:54216) 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aaea00640 1 -- 172.21.15.37:0/3945060083 learned_addr learned my addr 172.21.15.37:0/3945060083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aaf400640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0107580 0x7f3ab0108dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.805+0000 7f3aae000640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab0101e80 0x7f3ab0108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aaea00640 1 -- 172.21.15.37:0/3945060083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab0101e80 msgr2=0x7f3ab0108890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aaea00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab0101e80 0x7f3ab0108890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aaea00640 1 -- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0107580 msgr2=0x7f3ab0108dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aaea00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0107580 0x7f3ab0108dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aaea00640 1 -- 172.21.15.37:0/3945060083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab01aa600 con 0x7f3ab01014c0 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aaea00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0108350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9f0 tx=0x7f3aa000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aa7400640 1 -- 172.21.15.37:0/3945060083 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cd80 con 0x7f3ab01014c0 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aa7400640 1 -- 172.21.15.37:0/3945060083 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0005500 con 0x7f3ab01014c0 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.806+0000 7f3aa7400640 1 -- 172.21.15.37:0/3945060083 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0010690 con 0x7f3ab01014c0 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.807+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab01aa8f0 con 0x7f3ab01014c0 2024-03-21T09:18:04.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.807+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab01aadc0 con 0x7f3ab01014c0 2024-03-21T09:18:04.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.811+0000 7f3ab87f5740 1 Processor -- start 2024-03-21T09:18:04.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.811+0000 7f3ab87f5740 1 -- start start 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.811+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.811+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565bb260 0x55eb565c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.811+0000 7f3aa7400640 1 -- 172.21.15.37:0/3945060083 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3aa00046c0 con 0x7f3ab01014c0 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb565c6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb5651e2c0 con 0x55eb565c3de0 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb564c6c90 con 0x55eb565bb260 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb5651a360 con 0x55eb565ba8b0 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565bac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54226/0 (socket says 172.21.15.37:54226) 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 -- 172.21.15.37:0/2330114949 learned_addr learned my addr 172.21.15.37:0/2330114949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:04.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaea00640 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565bb260 0x55eb565c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aae000640 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb565c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 -- 172.21.15.37:0/2330114949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565bb260 msgr2=0x55eb565c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565bb260 0x55eb565c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 -- 172.21.15.37:0/2330114949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 msgr2=0x55eb565c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb565c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aaf400640 1 -- 172.21.15.37:0/2330114949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb564eb080 con 0x55eb565ba8b0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.812+0000 7f3aae000640 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb565c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565bac90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009fd0 tx=0x7f3aa80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f02963ac9a20cb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3a9be00640 1 -- 172.21.15.37:0/2330114949 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8004d20 con 0x55eb565ba8b0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3a9be00640 1 -- 172.21.15.37:0/2330114949 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3aa8004050 con 0x55eb565ba8b0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3a9be00640 1 -- 172.21.15.37:0/2330114949 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8005260 con 0x55eb565ba8b0 2024-03-21T09:18:04.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3aa7400640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3a940a3110 0x7f3a940a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3aa7400640 1 -- 172.21.15.37:0/3945060083 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3aa00cff20 con 0x7f3ab01014c0 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2330114949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 msgr2=0x55eb565bac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.813+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565bac90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009fd0 tx=0x7f3aa80118d0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3aaea00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3a940a3110 0x7f3a940a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2330114949 shutdown_connections 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb565c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565bb260 0x55eb565c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2330114949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565bac90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2330114949 >> 172.21.15.37:0/2330114949 conn(0x55eb565b4f80 msgr2=0x55eb565b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2330114949 shutdown_connections 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2330114949 wait complete. 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3aaea00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3a940a3110 0x7f3a940a55d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55eb565ba380 tx=0x7f3aa00d39d0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.814+0000 7f3ab87f5740 1 -- start start 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 Processor -- start 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565c3de0 0x55eb565c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb565c6d50 con 0x55eb565bb260 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb564da0d0 con 0x55eb565c3de0 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb565c6ed0 con 0x55eb565ba8b0 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3ab87f5740 10 client.0 _ll_register_callbacks cb 0x55eb564dee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54242/0 (socket says 172.21.15.37:54242) 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3aaf400640 1 -- 172.21.15.37:0/74999052 learned_addr learned my addr 172.21.15.37:0/74999052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3aae000640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.815+0000 7f3aaf400640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565c3de0 0x55eb565c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3aaf400640 1 -- 172.21.15.37:0/74999052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565c3de0 msgr2=0x55eb565c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3aaf400640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565c3de0 0x55eb565c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3aaf400640 1 -- 172.21.15.37:0/74999052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 msgr2=0x55eb565c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3aaf400640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3aaf400640 1 -- 172.21.15.37:0/74999052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb562323f0 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3aaf400640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565c36e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3aa801b040 tx=0x7f3aa8005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9a2266822c1ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3a98c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3a98c00640 1 -- 172.21.15.37:0/74999052 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80046a0 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3a98c00640 1 -- 172.21.15.37:0/74999052 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3aa800f4e0 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.816+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aa8004050 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3a98c00640 1 -- 172.21.15.37:0/74999052 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa8036050 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 10 client.26986 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eb562fc7d0 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3a98c00640 1 -- 172.21.15.37:0/74999052 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3aa8004050 con 0x55eb565ba8b0 2024-03-21T09:18:04.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3a98c00640 1 -- 172.21.15.37:0/74999052 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3aa8025040 con 0x55eb565ba8b0 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 10 client.26986 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 10 client.26986 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 20 client.26986 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb565d3b80 0x55eb565d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 4 client.26986 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.817+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb5652b800 con 0x55eb565d3b80 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.818+0000 7f3aae000640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb565d3b80 0x55eb565d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.818+0000 7f3aae000640 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb565d3b80 0x55eb565d5f70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0cee151f91de0d9 server_cookie=ee8d60834d44f758 in_seq=0 out_seq=0 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.818+0000 7f3a98c00640 10 client.26986 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3a98c00640 1 -- 172.21.15.37:0/74999052 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eb5652b800 con 0x55eb565d3b80 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3a98c00640 10 client.26986 handle_command_reply: tid=0 2024-03-21T09:18:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 client.26986 shutdown 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 msgr2=0x55eb565c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565c36e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3aa801b040 tx=0x7f3aa8005bd0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 shutdown_connections 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565c3de0 0x55eb565c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb565d3b80 0x55eb565d5f70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/74999052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565ba8b0 0x55eb565c36e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3a98c00640 0 client.26986 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 >> 172.21.15.37:0/74999052 conn(0x55eb565b4f80 msgr2=0x55eb565b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 shutdown_connections 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.819+0000 7f3ab87f5740 1 -- 172.21.15.37:0/74999052 wait complete. 2024-03-21T09:18:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.820+0000 7f3ab87f5740 20 client.26986 trim_cache size 0 max 16384 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.833+0000 7f3ab87f5740 1 Processor -- start 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.833+0000 7f3ab87f5740 1 -- start start 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565c3de0 0x55eb565c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01b73f0 con 0x55eb565bb260 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01b4ef0 con 0x55eb565ba8b0 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01b2f90 con 0x55eb565c3de0 2024-03-21T09:18:04.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aae000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45596/0 (socket says 172.21.15.37:45596) 2024-03-21T09:18:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 -- 172.21.15.37:0/2255575421 learned_addr learned my addr 172.21.15.37:0/2255575421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:04.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565c3de0 0x55eb565c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 -- 172.21.15.37:0/2255575421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565c3de0 msgr2=0x55eb565c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565c3de0 0x55eb565c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 -- 172.21.15.37:0/2255575421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 msgr2=0x55eb565c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.834+0000 7f3aaf400640 1 -- 172.21.15.37:0/2255575421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb565c6c30 con 0x55eb565ba8b0 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.835+0000 7f3aaf400640 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c03b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800fd20 tx=0x7f3aa8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c0a5f3b69ced57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.835+0000 7f3a9b400640 1 -- 172.21.15.37:0/2255575421 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8009c50 con 0x55eb565ba8b0 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.835+0000 7f3a9b400640 1 -- 172.21.15.37:0/2255575421 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3aa802d070 con 0x55eb565ba8b0 2024-03-21T09:18:04.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.835+0000 7f3a9b400640 1 -- 172.21.15.37:0/2255575421 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80044a0 con 0x55eb565ba8b0 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2255575421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 msgr2=0x55eb565c03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c03b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800fd20 tx=0x7f3aa8005fd0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2255575421 shutdown_connections 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565c3de0 0x55eb565c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565bb260 0x55eb565c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/2255575421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c03b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2255575421 >> 172.21.15.37:0/2255575421 conn(0x55eb565b4f80 msgr2=0x55eb565b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2255575421 shutdown_connections 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 -- 172.21.15.37:0/2255575421 wait complete. 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 -- start start 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.836+0000 7f3ab87f5740 1 Processor -- start 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 0x55eb565ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb5662e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01b3590 con 0x55eb565c3de0 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01b4890 con 0x55eb565ba8b0 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01b3bd0 con 0x55eb565bb260 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3ab87f5740 10 client.0 _ll_register_callbacks cb 0x7f3aa00d38a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 0x55eb565ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aaf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 0x55eb565ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54260/0 (socket says 172.21.15.37:54260) 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 -- 172.21.15.37:0/924847044 learned_addr learned my addr 172.21.15.37:0/924847044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aaf400640 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb5662e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 -- 172.21.15.37:0/924847044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 msgr2=0x55eb565c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 -- 172.21.15.37:0/924847044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 msgr2=0x55eb5662e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb5662e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.837+0000 7f3aae000640 1 -- 172.21.15.37:0/924847044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb5662e780 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3aae000640 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 0x55eb565ca1e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c02c230 tx=0x7f3a9c004110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70270b74e627ea1b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3a98c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3a98c00640 1 -- 172.21.15.37:0/924847044 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c004410 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aa802d070 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3a98c00640 1 -- 172.21.15.37:0/924847044 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3a9c02c870 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3a98c00640 1 -- 172.21.15.37:0/924847044 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a9c05d050 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3ab87f5740 10 client.26992 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.838+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eb5662fb10 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3a98c00640 1 -- 172.21.15.37:0/924847044 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3aa802d070 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3a98c00640 1 -- 172.21.15.37:0/924847044 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3a9c005120 con 0x55eb565bb260 2024-03-21T09:18:04.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3ab87f5740 10 client.26992 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:04.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3ab87f5740 10 client.26992 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:18:04.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3ab87f5740 20 client.26992 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:04.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb56630270 0x55eb56632690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:04.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3ab87f5740 4 client.26992 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:18:04.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb56525b50 con 0x55eb56630270 2024-03-21T09:18:04.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.839+0000 7f3aaf400640 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb56630270 0x55eb56632690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3a98c00640 10 client.26992 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3aaf400640 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb56630270 0x55eb56632690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cdf7bdc14fcb70b server_cookie=edd0a02b00481381 in_seq=0 out_seq=0 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3a98c00640 1 -- 172.21.15.37:0/924847044 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb56525b50 con 0x55eb56630270 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3a98c00640 10 client.26992 handle_command_reply: tid=0 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 client.26992 shutdown 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 msgr2=0x55eb565ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 0x55eb565ca1e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c02c230 tx=0x7f3a9c004110 comp rx=0 tx=0).stop 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 shutdown_connections 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55eb56630270 0x55eb56632690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb565c3de0 0x55eb5662e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3a98c00640 0 client.26992 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:04.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb565bb260 0x55eb565ca1e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 --2- 172.21.15.37:0/924847044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb565ba8b0 0x55eb565c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 >> 172.21.15.37:0/924847044 conn(0x55eb565b4f80 msgr2=0x55eb565bb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 shutdown_connections 2024-03-21T09:18:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 1 -- 172.21.15.37:0/924847044 wait complete. 2024-03-21T09:18:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.840+0000 7f3ab87f5740 20 client.26992 trim_cache size 0 max 16384 2024-03-21T09:18:04.841 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:18:04.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3a940a3110 msgr2=0x7f3a940a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3a940a3110 0x7f3a940a55d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55eb565ba380 tx=0x7f3aa00d39d0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 msgr2=0x7f3ab0108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0108350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9f0 tx=0x7f3aa000edf0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 shutdown_connections 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab0107580 0x7f3ab0108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3a940a3110 0x7f3a940a55d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ab0101e80 0x7f3ab0108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 --2- 172.21.15.37:0/3945060083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ab01014c0 0x7f3ab0108350 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 >> 172.21.15.37:0/3945060083 conn(0x7f3ab00fcfb0 msgr2=0x7f3ab00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 shutdown_connections 2024-03-21T09:18:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:04.841+0000 7f3aafe00640 1 -- 172.21.15.37:0/3945060083 wait complete. 2024-03-21T09:18:04.856 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:18:04.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:18:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 -- 172.21.15.37:0/2933251030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac101e80 msgr2=0x7fa5ac10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 --2- 172.21.15.37:0/2933251030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac101e80 0x7fa5ac10e340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009930 tx=0x7fa5a402bc30 comp rx=0 tx=0).stop 2024-03-21T09:18:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 -- 172.21.15.37:0/2933251030 shutdown_connections 2024-03-21T09:18:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 --2- 172.21.15.37:0/2933251030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac101e80 0x7fa5ac10e340 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 --2- 172.21.15.37:0/2933251030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 --2- 172.21.15.37:0/2933251030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac107580 0x7fa5ac107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 -- 172.21.15.37:0/2933251030 >> 172.21.15.37:0/2933251030 conn(0x7fa5ac0fcfb0 msgr2=0x7fa5ac0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 -- 172.21.15.37:0/2933251030 shutdown_connections 2024-03-21T09:18:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.080+0000 7fa5b2000640 1 -- 172.21.15.37:0/2933251030 wait complete. 2024-03-21T09:18:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.081+0000 7fa5b2000640 1 Processor -- start 2024-03-21T09:18:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 -- start start 2024-03-21T09:18:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac101e80 0x7fa5ac19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac107580 0x7fa5ac19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1118b0 con 0x7fa5ac101e80 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac111730 con 0x7fa5ac1014c0 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b2000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac111a30 con 0x7fa5ac107580 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac107580 0x7fa5ac19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac101e80 0x7fa5ac19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.082+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45608/0 (socket says 172.21.15.37:45608) 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 -- 172.21.15.37:0/195104110 learned_addr learned my addr 172.21.15.37:0/195104110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac107580 0x7fa5ac19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54278/0 (socket says 172.21.15.37:54278) 2024-03-21T09:18:05.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 -- 172.21.15.37:0/195104110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac107580 msgr2=0x7fa5ac19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac107580 0x7fa5ac19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 -- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac101e80 msgr2=0x7fa5ac19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac101e80 0x7fa5ac19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 -- 172.21.15.37:0/195104110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac19ef00 con 0x7fa5ac1014c0 2024-03-21T09:18:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac199f10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00eab0 tx=0x7fa59c00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.083+0000 7fa5a9600640 1 -- 172.21.15.37:0/195104110 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00ce40 con 0x7fa5ac1014c0 2024-03-21T09:18:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.084+0000 7fa5a9600640 1 -- 172.21.15.37:0/195104110 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa59c011950 con 0x7fa5ac1014c0 2024-03-21T09:18:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.084+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac19f1f0 con 0x7fa5ac1014c0 2024-03-21T09:18:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.084+0000 7fa5a9600640 1 -- 172.21.15.37:0/195104110 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c010ae0 con 0x7fa5ac1014c0 2024-03-21T09:18:05.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.084+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac1a6a80 con 0x7fa5ac1014c0 2024-03-21T09:18:05.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 Processor -- start 2024-03-21T09:18:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 -- start start 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a671c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a67a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a67d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965a5d52c0 con 0x55965a6718b0 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965a57dc90 con 0x55965a672260 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.087+0000 7fa5b6418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965a5d1360 con 0x55965a67ade0 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5b1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a671c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a67d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5b1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a671c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46486/0 (socket says 172.21.15.37:46486) 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5b1600640 1 -- 172.21.15.37:0/1058963612 learned_addr learned my addr 172.21.15.37:0/1058963612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a67d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54294/0 (socket says 172.21.15.37:54294) 2024-03-21T09:18:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 -- 172.21.15.37:0/1058963612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 msgr2=0x55965a67a810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:18:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a67a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 -- 172.21.15.37:0/1058963612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 msgr2=0x55965a671c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a671c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 -- 172.21.15.37:0/1058963612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965a5a2080 con 0x55965a67ade0 2024-03-21T09:18:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.088+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a67d230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa598009bd0 tx=0x7fa598011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f03c6b145df405 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.089+0000 7fa5a9600640 1 -- 172.21.15.37:0/195104110 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa59c011b00 con 0x7fa5ac1014c0 2024-03-21T09:18:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.089+0000 7fa5a1c00640 1 -- 172.21.15.37:0/1058963612 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598004230 con 0x55965a67ade0 2024-03-21T09:18:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.089+0000 7fa5a1c00640 1 -- 172.21.15.37:0/1058963612 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa598009e40 con 0x55965a67ade0 2024-03-21T09:18:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.089+0000 7fa5a1c00640 1 -- 172.21.15.37:0/1058963612 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598004d90 con 0x55965a67ade0 2024-03-21T09:18:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.089+0000 7fa5b6418740 1 -- 172.21.15.37:0/1058963612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 msgr2=0x55965a67d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.089+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a67d230 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa598009bd0 tx=0x7fa598011810 comp rx=0 tx=0).stop 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 -- 172.21.15.37:0/1058963612 shutdown_connections 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a67d230 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a67a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1058963612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a671c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 -- 172.21.15.37:0/1058963612 >> 172.21.15.37:0/1058963612 conn(0x55965a66bf80 msgr2=0x55965a66c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 -- 172.21.15.37:0/1058963612 shutdown_connections 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 -- 172.21.15.37:0/1058963612 wait complete. 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 -- start start 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5a9600640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5900a3020 0x7fa5900a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5b6418740 1 Processor -- start 2024-03-21T09:18:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.090+0000 7fa5a9600640 1 -- 172.21.15.37:0/195104110 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa59c0d0170 con 0x7fa5ac1014c0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b1600640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5900a3020 0x7fa5900a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a67a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a677af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a6780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965a67dd50 con 0x55965a67ade0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965a5910d0 con 0x55965a6718b0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965a67ded0 con 0x55965a672260 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b6418740 10 client.0 _ll_register_callbacks cb 0x55965a595e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a67a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a67a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45622/0 (socket says 172.21.15.37:45622) 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5b0c00640 1 -- 172.21.15.37:0/1941317333 learned_addr learned my addr 172.21.15.37:0/1941317333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.091+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a677af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b0c00640 1 -- 172.21.15.37:0/1941317333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 msgr2=0x55965a677af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a677af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b0c00640 1 -- 172.21.15.37:0/1941317333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 msgr2=0x55965a6780b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a6780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b0c00640 1 -- 172.21.15.37:0/1941317333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55965a2e93f0 con 0x55965a6718b0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b1600640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5900a3020 0x7fa5900a54e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4036040 tx=0x7fa5a402df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a67a700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa59c098b30 tx=0x7fa59c09bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df46c4c3c12d3fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa596000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa596000640 1 -- 172.21.15.37:0/1941317333 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0b5060 con 0x55965a6718b0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa596000640 1 -- 172.21.15.37:0/1941317333 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa59c0987d0 con 0x55965a6718b0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.092+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965a5e2800 con 0x55965a6718b0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa596000640 1 -- 172.21.15.37:0/1941317333 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa59c09c160 con 0x55965a6718b0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 10 client.26975 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55965a3b37d0 con 0x55965a6718b0 2024-03-21T09:18:05.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa596000640 1 -- 172.21.15.37:0/1941317333 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55965a5e2800 con 0x55965a6718b0 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa596000640 1 -- 172.21.15.37:0/1941317333 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa59c00f040 con 0x55965a6718b0 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 10 client.26975 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 10 client.26975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 20 client.26975 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a68ac30 0x55965a68d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 4 client.26975 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55965a5dcb50 con 0x55965a68ac30 2024-03-21T09:18:05.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.093+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a68ac30 0x55965a68d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.094+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a68ac30 0x55965a68d020 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=641acd5926d595a7 server_cookie=d6e56f4606466927 in_seq=0 out_seq=0 2024-03-21T09:18:05.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.094+0000 7fa596000640 10 client.26975 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:05.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa596000640 1 -- 172.21.15.37:0/1941317333 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55965a5dcb50 con 0x55965a68ac30 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa596000640 10 client.26975 handle_command_reply: tid=0 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 client.26975 shutdown 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 msgr2=0x55965a67a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a67a700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa59c098b30 tx=0x7fa59c09bd60 comp rx=0 tx=0).stop 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 shutdown_connections 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a68ac30 0x55965a68d020 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a6780b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a677af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1941317333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a67a700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 >> 172.21.15.37:0/1941317333 conn(0x55965a66bf80 msgr2=0x55965a66dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa596000640 0 client.26975 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 shutdown_connections 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.095+0000 7fa5b6418740 1 -- 172.21.15.37:0/1941317333 wait complete. 2024-03-21T09:18:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.096+0000 7fa5b6418740 20 client.26975 trim_cache size 0 max 16384 2024-03-21T09:18:05.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.109+0000 7fa5b6418740 1 Processor -- start 2024-03-21T09:18:05.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.109+0000 7fa5b6418740 1 -- start start 2024-03-21T09:18:05.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a66b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a67a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a677280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b6418740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1b2ea0 con 0x55965a67ade0 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b6418740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1b09a0 con 0x55965a6718b0 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b6418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1aea40 con 0x55965a672260 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a66b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a66b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45628/0 (socket says 172.21.15.37:45628) 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 -- 172.21.15.37:0/3116560591 learned_addr learned my addr 172.21.15.37:0/3116560591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5abe00640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a67a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b1600640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a677280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 -- 172.21.15.37:0/3116560591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 msgr2=0x55965a67a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a67a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 -- 172.21.15.37:0/3116560591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 msgr2=0x55965a677280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a677280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b0c00640 1 -- 172.21.15.37:0/3116560591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965a678420 con 0x55965a6718b0 2024-03-21T09:18:05.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5abe00640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a67a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:05.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.110+0000 7fa5b1600640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a677280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:05.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a66b160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0d00e0 tx=0x7fa59c0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78b407a7042b015 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5a1200640 1 -- 172.21.15.37:0/3116560591 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0a4b70 con 0x55965a6718b0 2024-03-21T09:18:05.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5a1200640 1 -- 172.21.15.37:0/3116560591 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa59c09c3c0 con 0x55965a6718b0 2024-03-21T09:18:05.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5a1200640 1 -- 172.21.15.37:0/3116560591 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0bc140 con 0x55965a6718b0 2024-03-21T09:18:05.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 -- 172.21.15.37:0/3116560591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 msgr2=0x55965a66b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a66b160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0d00e0 tx=0x7fa59c0094a0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 -- 172.21.15.37:0/3116560591 shutdown_connections 2024-03-21T09:18:05.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a67ade0 0x55965a677280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a672260 0x55965a67a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 --2- 172.21.15.37:0/3116560591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a6718b0 0x55965a66b160 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 -- 172.21.15.37:0/3116560591 >> 172.21.15.37:0/3116560591 conn(0x55965a66bf80 msgr2=0x55965a66dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.111+0000 7fa5b6418740 1 -- 172.21.15.37:0/3116560591 shutdown_connections 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.112+0000 7fa5b6418740 1 -- 172.21.15.37:0/3116560591 wait complete. 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.112+0000 7fa5b6418740 1 -- start start 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.112+0000 7fa5b6418740 1 Processor -- start 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.112+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a682d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.112+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a680130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b6418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a680670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b6418740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1af040 con 0x55965a6718b0 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b6418740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1b0340 con 0x55965a672260 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b6418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1af680 con 0x55965a67ade0 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b6418740 10 client.0 _ll_register_callbacks cb 0x7fa59c09bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a682d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a680130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a682d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46508/0 (socket says 172.21.15.37:46508) 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a680670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b0c00640 1 -- 172.21.15.37:0/1935957741 learned_addr learned my addr 172.21.15.37:0/1935957741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5abe00640 1 -- 172.21.15.37:0/1935957741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 msgr2=0x55965a680670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a680670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5abe00640 1 -- 172.21.15.37:0/1935957741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 msgr2=0x55965a682d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a682d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5abe00640 1 -- 172.21.15.37:0/1935957741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55965a680d10 con 0x55965a672260 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.113+0000 7fa5b1600640 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a680670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa5abe00640 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a680130 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa5980048f0 tx=0x7fa598009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318539137cb8303b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa596000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa596000640 1 -- 172.21.15.37:0/1935957741 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59801e040 con 0x55965a672260 2024-03-21T09:18:05.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa596000640 1 -- 172.21.15.37:0/1935957741 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa5980040c0 con 0x55965a672260 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa59c09c3c0 con 0x55965a672260 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa596000640 1 -- 172.21.15.37:0/1935957741 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa598038050 con 0x55965a672260 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa5b6418740 10 client.26981 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55965a6e6370 con 0x55965a672260 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.114+0000 7fa596000640 1 -- 172.21.15.37:0/1935957741 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa59c09c3c0 con 0x55965a672260 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa596000640 1 -- 172.21.15.37:0/1935957741 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa598018070 con 0x55965a672260 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b6418740 10 client.26981 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b6418740 10 client.26981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b6418740 20 client.26981 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a6e6820 0x55965a6e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b6418740 4 client.26981 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55965a55a500 con 0x55965a6e6820 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a6e6820 0x55965a6e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa5b0c00640 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a6e6820 0x55965a6e8c40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a651c4f5eb73a35 server_cookie=a03f2c3d25e8b232 in_seq=0 out_seq=0 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.115+0000 7fa596000640 10 client.26981 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa596000640 1 -- 172.21.15.37:0/1935957741 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55965a55a500 con 0x55965a6e6820 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa596000640 10 client.26981 handle_command_reply: tid=0 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 client.26981 shutdown 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 msgr2=0x55965a680130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a680130 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa5980048f0 tx=0x7fa598009310 comp rx=0 tx=0).stop 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 shutdown_connections 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55965a6e6820 0x55965a6e8c40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55965a67ade0 0x55965a680670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965a672260 0x55965a680130 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 --2- 172.21.15.37:0/1935957741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55965a6718b0 0x55965a682d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 >> 172.21.15.37:0/1935957741 conn(0x55965a66bf80 msgr2=0x55965a679a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa596000640 0 client.26981 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:05.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 shutdown_connections 2024-03-21T09:18:05.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 1 -- 172.21.15.37:0/1935957741 wait complete. 2024-03-21T09:18:05.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.116+0000 7fa5b6418740 20 client.26981 trim_cache size 0 max 16384 2024-03-21T09:18:05.117 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5900a3020 msgr2=0x7fa5900a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5900a3020 0x7fa5900a54e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4036040 tx=0x7fa5a402df90 comp rx=0 tx=0).stop 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 msgr2=0x7fa5ac199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac199f10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00eab0 tx=0x7fa59c00ef70 comp rx=0 tx=0).stop 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 shutdown_connections 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5ac107580 0x7fa5ac19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac101e80 0x7fa5ac19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa5900a3020 0x7fa5900a54e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 --2- 172.21.15.37:0/195104110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac1014c0 0x7fa5ac199f10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 >> 172.21.15.37:0/195104110 conn(0x7fa5ac0fcfb0 msgr2=0x7fa5ac0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 shutdown_connections 2024-03-21T09:18:05.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.117+0000 7fa5b2000640 1 -- 172.21.15.37:0/195104110 wait complete. 2024-03-21T09:18:05.132 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:18:05.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:18:05.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 -- 172.21.15.37:0/2562402044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 msgr2=0x7ff74c100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 --2- 172.21.15.37:0/2562402044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c100630 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff740009900 tx=0x7ff74002bcc0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 -- 172.21.15.37:0/2562402044 shutdown_connections 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 --2- 172.21.15.37:0/2562402044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74c10de00 0x7ff74c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 --2- 172.21.15.37:0/2562402044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c100b70 0x7ff74c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 --2- 172.21.15.37:0/2562402044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c100630 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.349+0000 7ff752c00640 1 -- 172.21.15.37:0/2562402044 >> 172.21.15.37:0/2562402044 conn(0x7ff74c077130 msgr2=0x7ff74c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.350+0000 7ff752c00640 1 -- 172.21.15.37:0/2562402044 shutdown_connections 2024-03-21T09:18:05.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.350+0000 7ff752c00640 1 -- 172.21.15.37:0/2562402044 wait complete. 2024-03-21T09:18:05.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.352+0000 7ff752c00640 1 Processor -- start 2024-03-21T09:18:05.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.352+0000 7ff752c00640 1 -- start start 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74c100b70 0x7ff74c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c10de00 0x7ff74c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c113ad0 con 0x7ff74c100b70 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c113950 con 0x7ff74c10de00 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c113c50 con 0x7ff74c100250 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54348/0 (socket says 172.21.15.37:54348) 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 -- 172.21.15.37:0/1402533517 learned_addr learned my addr 172.21.15.37:0/1402533517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff750e00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74c100b70 0x7ff74c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff752200640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c10de00 0x7ff74c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 -- 172.21.15.37:0/1402533517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c10de00 msgr2=0x7ff74c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c10de00 0x7ff74c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 -- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74c100b70 msgr2=0x7ff74c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74c100b70 0x7ff74c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.353+0000 7ff751800640 1 -- 172.21.15.37:0/1402533517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c19ef80 con 0x7ff74c100250 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.354+0000 7ff751800640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c199f90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff740000c00 tx=0x7ff7400043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.354+0000 7ff74a000640 1 -- 172.21.15.37:0/1402533517 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740042020 con 0x7ff74c100250 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.354+0000 7ff74a000640 1 -- 172.21.15.37:0/1402533517 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff740005000 con 0x7ff74c100250 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.354+0000 7ff74a000640 1 -- 172.21.15.37:0/1402533517 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740003b20 con 0x7ff74c100250 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.354+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c19f210 con 0x7ff74c100250 2024-03-21T09:18:05.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.354+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a6b00 con 0x7ff74c100250 2024-03-21T09:18:05.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.357+0000 7ff74a000640 1 -- 172.21.15.37:0/1402533517 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff740045020 con 0x7ff74c100250 2024-03-21T09:18:05.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.357+0000 7ff756e20740 1 Processor -- start 2024-03-21T09:18:05.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 -- start start 2024-03-21T09:18:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba047c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba050810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba053230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b9fab2c0 con 0x5587ba0478b0 2024-03-21T09:18:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b9f53c90 con 0x5587ba048260 2024-03-21T09:18:05.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff756e20740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b9fa7360 con 0x5587ba050de0 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff751800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba050810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff751800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba050810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45658/0 (socket says 172.21.15.37:45658) 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff751800640 1 -- 172.21.15.37:0/1763092509 learned_addr learned my addr 172.21.15.37:0/1763092509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.358+0000 7ff752200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba047c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff750e00640 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba053230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff751800640 1 -- 172.21.15.37:0/1763092509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 msgr2=0x5587ba053230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff751800640 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba053230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff751800640 1 -- 172.21.15.37:0/1763092509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 msgr2=0x5587ba047c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff751800640 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba047c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff751800640 1 -- 172.21.15.37:0/1763092509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b9f78080 con 0x5587ba048260 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff74a000640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7340a3110 0x7ff7340a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff751800640 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba050810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7400eeb10 tx=0x7ff7400f7b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80d82f01de8daaa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.359+0000 7ff74a000640 1 -- 172.21.15.37:0/1402533517 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff7400f1770 con 0x7ff74c100250 2024-03-21T09:18:05.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff752200640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7340a3110 0x7ff7340a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff73ea00640 1 -- 172.21.15.37:0/1763092509 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400b9b10 con 0x5587ba048260 2024-03-21T09:18:05.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff73ea00640 1 -- 172.21.15.37:0/1763092509 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff740004c80 con 0x5587ba048260 2024-03-21T09:18:05.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff73ea00640 1 -- 172.21.15.37:0/1763092509 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400bd1d0 con 0x5587ba048260 2024-03-21T09:18:05.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff756e20740 1 -- 172.21.15.37:0/1763092509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 msgr2=0x5587ba050810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff756e20740 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba050810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7400eeb10 tx=0x7ff7400f7b20 comp rx=0 tx=0).stop 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff752200640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7340a3110 0x7ff7340a55d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff74c074430 tx=0x7ff744009600 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff756e20740 1 -- 172.21.15.37:0/1763092509 shutdown_connections 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff756e20740 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba053230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff756e20740 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba050810 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.360+0000 7ff756e20740 1 --2- 172.21.15.37:0/1763092509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba047c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.361+0000 7ff756e20740 1 -- 172.21.15.37:0/1763092509 >> 172.21.15.37:0/1763092509 conn(0x5587ba041f80 msgr2=0x5587ba042360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.361+0000 7ff756e20740 1 -- 172.21.15.37:0/1763092509 shutdown_connections 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.361+0000 7ff756e20740 1 -- 172.21.15.37:0/1763092509 wait complete. 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.361+0000 7ff756e20740 1 -- start start 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.361+0000 7ff756e20740 1 Processor -- start 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 0x5587ba050700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba048260 0x5587ba04da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba04e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba053b10 con 0x5587ba048260 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b9f670d0 con 0x5587ba0478b0 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba053c90 con 0x5587ba050de0 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff756e20740 10 client.0 _ll_register_callbacks cb 0x5587b9f6be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff752200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba04e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff751800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 0x5587ba050700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff750e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba048260 0x5587ba04da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff751800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 0x5587ba050700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45662/0 (socket says 172.21.15.37:45662) 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff751800640 1 -- 172.21.15.37:0/3793618556 learned_addr learned my addr 172.21.15.37:0/3793618556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff752200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba04e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54370/0 (socket says 172.21.15.37:54370) 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.362+0000 7ff751800640 1 -- 172.21.15.37:0/3793618556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 msgr2=0x5587ba04e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff751800640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba04e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff751800640 1 -- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba048260 msgr2=0x5587ba04da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff751800640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba048260 0x5587ba04da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff751800640 1 -- 172.21.15.37:0/3793618556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587b9cbf3f0 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff750e00640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba048260 0x5587ba04da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff752200640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba04e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff751800640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 0x5587ba050700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff740005d90 tx=0x7ff7400092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c516a05a8c9c0450 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff733400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff733400640 1 -- 172.21.15.37:0/3793618556 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400bd3b0 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff733400640 1 -- 172.21.15.37:0/3793618556 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff740003cc0 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff740004c80 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff733400640 1 -- 172.21.15.37:0/3793618556 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7400bd640 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff756e20740 10 client.26993 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.363+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5587b9d897d0 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff733400640 1 -- 172.21.15.37:0/3793618556 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff740004c80 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff733400640 1 -- 172.21.15.37:0/3793618556 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff7400051b0 con 0x5587ba0478b0 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff756e20740 10 client.26993 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff756e20740 10 client.26993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:05.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff756e20740 20 client.26993 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff756e20740 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba060820 0x5587ba062c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff756e20740 4 client.26993 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587b9fb8800 con 0x5587ba060820 2024-03-21T09:18:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.364+0000 7ff750e00640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba060820 0x5587ba062c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.365+0000 7ff750e00640 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba060820 0x5587ba062c10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec67d1097615224 server_cookie=350aa742c8ac5c13 in_seq=0 out_seq=0 2024-03-21T09:18:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.365+0000 7ff733400640 10 client.26993 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff733400640 1 -- 172.21.15.37:0/3793618556 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5587b9fb8800 con 0x5587ba060820 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff733400640 10 client.26993 handle_command_reply: tid=0 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 client.26993 shutdown 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 msgr2=0x5587ba050700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 0x5587ba050700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff740005d90 tx=0x7ff7400092d0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 shutdown_connections 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba04e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba048260 0x5587ba04da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba060820 0x5587ba062c10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 --2- 172.21.15.37:0/3793618556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba0478b0 0x5587ba050700 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff733400640 0 client.26993 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 >> 172.21.15.37:0/3793618556 conn(0x5587ba041f80 msgr2=0x5587ba043cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.366+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 shutdown_connections 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.367+0000 7ff756e20740 1 -- 172.21.15.37:0/3793618556 wait complete. 2024-03-21T09:18:05.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.367+0000 7ff756e20740 20 client.26993 trim_cache size 0 max 16384 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.380+0000 7ff756e20740 1 Processor -- start 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.380+0000 7ff756e20740 1 -- start start 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff756e20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba041160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff756e20740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba0501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff756e20740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba050700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff756e20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1b2fd0 con 0x5587ba0478b0 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff756e20740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1b0ad0 con 0x5587ba048260 2024-03-21T09:18:05.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff756e20740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1aeb70 con 0x5587ba050de0 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba0501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff752200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba050700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba0501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45676/0 (socket says 172.21.15.37:45676) 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff752200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba050700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54384/0 (socket says 172.21.15.37:54384) 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 -- 172.21.15.37:0/4723288 learned_addr learned my addr 172.21.15.37:0/4723288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff751800640 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba041160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 -- 172.21.15.37:0/4723288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 msgr2=0x5587ba050700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba050700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 -- 172.21.15.37:0/4723288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 msgr2=0x5587ba041160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba041160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff750e00640 1 -- 172.21.15.37:0/4723288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587ba04e3a0 con 0x5587ba048260 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.381+0000 7ff751800640 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba041160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.382+0000 7ff750e00640 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba0501c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x5587ba063210 tx=0x7ff738005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac473ed441c1632f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.382+0000 7ff73e000640 1 -- 172.21.15.37:0/4723288 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380120d0 con 0x5587ba048260 2024-03-21T09:18:05.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.382+0000 7ff73e000640 1 -- 172.21.15.37:0/4723288 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff738012270 con 0x5587ba048260 2024-03-21T09:18:05.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.382+0000 7ff73e000640 1 -- 172.21.15.37:0/4723288 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380053a0 con 0x5587ba048260 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.382+0000 7ff756e20740 1 -- 172.21.15.37:0/4723288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 msgr2=0x5587ba0501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.382+0000 7ff756e20740 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba0501c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x5587ba063210 tx=0x7ff738005f90 comp rx=0 tx=0).stop 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 -- 172.21.15.37:0/4723288 shutdown_connections 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba050700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba0501c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 --2- 172.21.15.37:0/4723288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba041160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 -- 172.21.15.37:0/4723288 >> 172.21.15.37:0/4723288 conn(0x5587ba041f80 msgr2=0x5587ba042360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 -- 172.21.15.37:0/4723288 shutdown_connections 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 -- 172.21.15.37:0/4723288 wait complete. 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 -- start start 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.383+0000 7ff756e20740 1 Processor -- start 2024-03-21T09:18:05.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba056980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba056ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1af170 con 0x5587ba0478b0 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1b0470 con 0x5587ba048260 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1af7b0 con 0x5587ba050de0 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff756e20740 10 client.0 _ll_register_callbacks cb 0x7ff744009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff750e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba056ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff752200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff750e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba056ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45682/0 (socket says 172.21.15.37:45682) 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff751800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba056980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff750e00640 1 -- 172.21.15.37:0/372133083 learned_addr learned my addr 172.21.15.37:0/372133083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff752200640 1 -- 172.21.15.37:0/372133083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 msgr2=0x5587ba056ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff752200640 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba056ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff752200640 1 -- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 msgr2=0x5587ba056980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff752200640 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba056980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff752200640 1 -- 172.21.15.37:0/372133083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587ba0b5450 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.384+0000 7ff750e00640 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba056ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff752200640 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba0b4e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7440099d0 tx=0x7ff744004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd030b3bd1b43c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff733400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff733400640 1 -- 172.21.15.37:0/372133083 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74402f060 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff738012270 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff733400640 1 -- 172.21.15.37:0/372133083 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff74401d400 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff733400640 1 -- 172.21.15.37:0/372133083 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff744036050 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff756e20740 10 client.27028 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5587ba0bc8a0 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.385+0000 7ff733400640 1 -- 172.21.15.37:0/372133083 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff738012270 con 0x5587ba050de0 2024-03-21T09:18:05.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff733400640 1 -- 172.21.15.37:0/372133083 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff744029070 con 0x5587ba050de0 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff756e20740 10 client.27028 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff756e20740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff756e20740 20 client.27028 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff756e20740 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba0bcf80 0x5587ba0bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff756e20740 4 client.27028 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587b9fb2b50 con 0x5587ba0bcf80 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.386+0000 7ff751800640 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba0bcf80 0x5587ba0bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff751800640 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba0bcf80 0x5587ba0bf3a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284fd7de828bee77 server_cookie=d81992ec0d96aa5f in_seq=0 out_seq=0 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff733400640 10 client.27028 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff733400640 1 -- 172.21.15.37:0/372133083 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587b9fb2b50 con 0x5587ba0bcf80 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff733400640 10 client.27028 handle_command_reply: tid=0 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 client.27028 shutdown 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 msgr2=0x5587ba0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba0b4e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7440099d0 tx=0x7ff744004a40 comp rx=0 tx=0).stop 2024-03-21T09:18:05.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 shutdown_connections 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ba050de0 0x5587ba0b4e10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5587ba0bcf80 0x5587ba0bf3a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5587ba048260 0x5587ba056ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 --2- 172.21.15.37:0/372133083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ba0478b0 0x5587ba056980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 >> 172.21.15.37:0/372133083 conn(0x5587ba041f80 msgr2=0x5587ba048640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff733400640 0 client.27028 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 shutdown_connections 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.387+0000 7ff756e20740 1 -- 172.21.15.37:0/372133083 wait complete. 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff756e20740 20 client.27028 trim_cache size 0 max 16384 2024-03-21T09:18:05.388 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:18:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7340a3110 msgr2=0x7ff7340a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7340a3110 0x7ff7340a55d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff74c074430 tx=0x7ff744009600 comp rx=0 tx=0).stop 2024-03-21T09:18:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 msgr2=0x7ff74c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c199f90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff740000c00 tx=0x7ff7400043c0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 shutdown_connections 2024-03-21T09:18:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7340a3110 0x7ff7340a55d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff74c10de00 0x7ff74c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74c100b70 0x7ff74c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 --2- 172.21.15.37:0/1402533517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff74c100250 0x7ff74c199f90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.388+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 >> 172.21.15.37:0/1402533517 conn(0x7ff74c077130 msgr2=0x7ff74c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.389+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 shutdown_connections 2024-03-21T09:18:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:05.389+0000 7ff752c00640 1 -- 172.21.15.37:0/1402533517 wait complete. 2024-03-21T09:18:05.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:05 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1268786774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:18:05.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:05 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1268786774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:18:05.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:05 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1268786774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:18:06.402 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:18:06.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:18:06.403 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:18:06.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:06 smithi071 ceph-mon[36597]: pgmap v180: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-03-21T09:18:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 -- 172.21.15.37:0/279070562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d8102630 msgr2=0x7fa3d8102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 --2- 172.21.15.37:0/279070562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d8102630 0x7fa3d8102a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009900 tx=0x7fa3c802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 -- 172.21.15.37:0/279070562 shutdown_connections 2024-03-21T09:18:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 --2- 172.21.15.37:0/279070562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d810ba50 0x7fa3d810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 --2- 172.21.15.37:0/279070562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3d8102f50 0x7fa3d810b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 --2- 172.21.15.37:0/279070562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d8102630 0x7fa3d8102a10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 -- 172.21.15.37:0/279070562 >> 172.21.15.37:0/279070562 conn(0x7fa3d8077130 msgr2=0x7fa3d80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 -- 172.21.15.37:0/279070562 shutdown_connections 2024-03-21T09:18:06.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.630+0000 7fa3de200640 1 -- 172.21.15.37:0/279070562 wait complete. 2024-03-21T09:18:06.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.631+0000 7fa3de200640 1 Processor -- start 2024-03-21T09:18:06.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.631+0000 7fa3de200640 1 -- start start 2024-03-21T09:18:06.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3de200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3d8102630 0x7fa3d8199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3de200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8102f50 0x7fa3d819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3de200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 0x7fa3d819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3de200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8111820 con 0x7fa3d8102630 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3de200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81116a0 con 0x7fa3d8102f50 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3de200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81119a0 con 0x7fa3d810ba50 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 0x7fa3d819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8102f50 0x7fa3d819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3d8102630 0x7fa3d8199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 0x7fa3d819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54406/0 (socket says 172.21.15.37:54406) 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3dd800640 1 -- 172.21.15.37:0/2918657435 learned_addr learned my addr 172.21.15.37:0/2918657435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.632+0000 7fa3dd800640 1 -- 172.21.15.37:0/2918657435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8102f50 msgr2=0x7fa3d819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8102f50 0x7fa3d819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3dd800640 1 -- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3d8102630 msgr2=0x7fa3d8199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3d8102630 0x7fa3d8199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3dd800640 1 -- 172.21.15.37:0/2918657435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d819ef10 con 0x7fa3d810ba50 2024-03-21T09:18:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 0x7fa3d819e7e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000eb10 tx=0x7fa3d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3d6000640 1 -- 172.21.15.37:0/2918657435 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004b20 con 0x7fa3d810ba50 2024-03-21T09:18:06.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3d6000640 1 -- 172.21.15.37:0/2918657435 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0004cc0 con 0x7fa3d810ba50 2024-03-21T09:18:06.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3d6000640 1 -- 172.21.15.37:0/2918657435 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0010760 con 0x7fa3d810ba50 2024-03-21T09:18:06.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d819f200 con 0x7fa3d810ba50 2024-03-21T09:18:06.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.633+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d81a6a90 con 0x7fa3d810ba50 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.636+0000 7fa3d6000640 1 -- 172.21.15.37:0/2918657435 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa3d0005020 con 0x7fa3d810ba50 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.636+0000 7fa3e25d1740 1 Processor -- start 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 -- start start 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61c146eb0 con 0x55b61c1e2bb0 2024-03-21T09:18:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61c0efa60 con 0x55b61c1e2290 2024-03-21T09:18:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61c142f50 con 0x55b61c1eb790 2024-03-21T09:18:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45716/0 (socket says 172.21.15.37:45716) 2024-03-21T09:18:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3cfe00640 1 -- 172.21.15.37:0/3032081619 learned_addr learned my addr 172.21.15.37:0/3032081619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3dce00640 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.637+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3cfe00640 1 -- 172.21.15.37:0/3032081619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 msgr2=0x55b61c1edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3cfe00640 1 -- 172.21.15.37:0/3032081619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 msgr2=0x55b61c1eb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3cfe00640 1 -- 172.21.15.37:0/3032081619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b61bf237e0 con 0x55b61c1e2290 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1e2670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00127c0 tx=0x7fa3c000b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e590da557dc601 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.638+0000 7fa3d6000640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3bc0a3090 0x7fa3bc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3ce000640 1 -- 172.21.15.37:0/3032081619 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012e00 con 0x55b61c1e2290 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3d6000640 1 -- 172.21.15.37:0/2918657435 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa3d00d0240 con 0x7fa3d810ba50 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3ce000640 1 -- 172.21.15.37:0/3032081619 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c0010c50 con 0x55b61c1e2290 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3ce000640 1 -- 172.21.15.37:0/3032081619 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c001eb60 con 0x55b61c1e2290 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3dce00640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3bc0a3090 0x7fa3bc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3e25d1740 1 -- 172.21.15.37:0/3032081619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 msgr2=0x55b61c1e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.639+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1e2670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00127c0 tx=0x7fa3c000b7c0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 -- 172.21.15.37:0/3032081619 shutdown_connections 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/3032081619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1e2670 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 -- 172.21.15.37:0/3032081619 >> 172.21.15.37:0/3032081619 conn(0x55b61c1dc9c0 msgr2=0x55b61c1dcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 -- 172.21.15.37:0/3032081619 shutdown_connections 2024-03-21T09:18:06.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3dce00640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3bc0a3090 0x7fa3bc0a5550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3c802efd0 tx=0x7fa3c802cf90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 -- 172.21.15.37:0/3032081619 wait complete. 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.640+0000 7fa3e25d1740 1 -- start start 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 Processor -- start 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 0x55b61c1eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c1e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1eb790 0x55b61c1e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61c1ee580 con 0x55b61c1e2290 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61c102eb0 con 0x55b61c1eb790 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.641+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61c1ee700 con 0x55b61c1e2bb0 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3e25d1740 10 client.0 _ll_register_callbacks cb 0x55b61c0f7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 0x55b61c1eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3dd800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1eb790 0x55b61c1e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c1e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 0x55b61c1eb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46592/0 (socket says 172.21.15.37:46592) 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3dce00640 1 -- 172.21.15.37:0/793797094 learned_addr learned my addr 172.21.15.37:0/793797094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3dd800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1eb790 0x55b61c1e89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45722/0 (socket says 172.21.15.37:45722) 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3cfe00640 1 -- 172.21.15.37:0/793797094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1eb790 msgr2=0x55b61c1e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1eb790 0x55b61c1e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3cfe00640 1 -- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 msgr2=0x55b61c1eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 0x55b61c1eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3cfe00640 1 -- 172.21.15.37:0/793797094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b61bf83f70 con 0x55b61c1e2bb0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.642+0000 7fa3dce00640 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 0x55b61c1eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c1e8490 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000b6b0 tx=0x7fa3c0012560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=159a2811a110e15e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3c6a00640 1 -- 172.21.15.37:0/793797094 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0010e00 con 0x55b61c1e2bb0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3c0010c50 con 0x55b61c1e2bb0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3c6a00640 1 -- 172.21.15.37:0/793797094 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa3c0010950 con 0x55b61c1e2bb0 2024-03-21T09:18:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3c6a00640 1 -- 172.21.15.37:0/793797094 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c0034050 con 0x55b61c1e2bb0 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3e25d1740 10 client.27040 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.643+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b61be1b170 con 0x55b61c1e2bb0 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3c6a00640 1 -- 172.21.15.37:0/793797094 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c0010c50 con 0x55b61c1e2bb0 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3c6a00640 1 -- 172.21.15.37:0/793797094 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa3c002f030 con 0x55b61c1e2bb0 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3e25d1740 10 client.27040 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3e25d1740 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3e25d1740 20 client.27040 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c1fb730 0x55b61c1fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3e25d1740 4 client.27040 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b61c1543f0 con 0x55b61c1fb730 2024-03-21T09:18:06.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.644+0000 7fa3dce00640 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c1fb730 0x55b61c1fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.645+0000 7fa3dce00640 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c1fb730 0x55b61c1fdb20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785723d560885fe6 server_cookie=1c496b0c3f00b5c8 in_seq=0 out_seq=0 2024-03-21T09:18:06.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.645+0000 7fa3c6a00640 10 client.27040 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3c6a00640 1 -- 172.21.15.37:0/793797094 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b61c1543f0 con 0x55b61c1fb730 2024-03-21T09:18:06.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3c6a00640 10 client.27040 handle_command_reply: tid=0 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 client.27040 shutdown 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 msgr2=0x55b61c1e8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c1e8490 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000b6b0 tx=0x7fa3c0012560 comp rx=0 tx=0).stop 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 shutdown_connections 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c1fb730 0x55b61c1fdb20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1eb790 0x55b61c1e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c1e8490 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/793797094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2290 0x55b61c1eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 >> 172.21.15.37:0/793797094 conn(0x55b61c1dc9c0 msgr2=0x55b61c1de710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3c6a00640 0 client.27040 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 shutdown_connections 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.646+0000 7fa3e25d1740 1 -- 172.21.15.37:0/793797094 wait complete. 2024-03-21T09:18:06.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.647+0000 7fa3e25d1740 20 client.27040 trim_cache size 0 max 16384 2024-03-21T09:18:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.664+0000 7fa3e25d1740 1 Processor -- start 2024-03-21T09:18:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.664+0000 7fa3e25d1740 1 -- start start 2024-03-21T09:18:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81b2f00 con 0x55b61c1e2bb0 2024-03-21T09:18:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81b0a00 con 0x55b61c1e2290 2024-03-21T09:18:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81aeaa0 con 0x55b61c1eb790 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1eaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45736/0 (socket says 172.21.15.37:45736) 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54442/0 (socket says 172.21.15.37:54442) 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3dce00640 1 -- 172.21.15.37:0/2095403738 learned_addr learned my addr 172.21.15.37:0/2095403738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.665+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3dd800640 1 -- 172.21.15.37:0/2095403738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 msgr2=0x55b61c1eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3dd800640 1 -- 172.21.15.37:0/2095403738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 msgr2=0x55b61c1eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3dd800640 1 -- 172.21.15.37:0/2095403738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b61c1edef0 con 0x55b61c1eb790 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1f6e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00cd590 tx=0x7fa3d00985d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50df9889cbb20f72 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3cd600640 1 -- 172.21.15.37:0/2095403738 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00a0170 con 0x55b61c1eb790 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3cd600640 1 -- 172.21.15.37:0/2095403738 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3d009c930 con 0x55b61c1eb790 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.666+0000 7fa3cd600640 1 -- 172.21.15.37:0/2095403738 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00cd9a0 con 0x55b61c1eb790 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2095403738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 msgr2=0x55b61c1f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1f6e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00cd590 tx=0x7fa3d00985d0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2095403738 shutdown_connections 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1eb790 0x55b61c1f6e10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1e2bb0 0x55b61c1eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2095403738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c1eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2095403738 >> 172.21.15.37:0/2095403738 conn(0x55b61c1dc9c0 msgr2=0x55b61c1e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2095403738 shutdown_connections 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2095403738 wait complete. 2024-03-21T09:18:06.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 -- start start 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.667+0000 7fa3e25d1740 1 Processor -- start 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c250130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c250670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1eb790 0x55b61c1ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81af0a0 con 0x55b61c1eb790 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81b03a0 con 0x55b61c1e2290 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81af6e0 con 0x55b61c1e2bb0 2024-03-21T09:18:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3e25d1740 10 client.0 _ll_register_callbacks cb 0x7fa3c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c250130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c250670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3dce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c250130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45744/0 (socket says 172.21.15.37:45744) 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3dce00640 1 -- 172.21.15.37:0/2168926306 learned_addr learned my addr 172.21.15.37:0/2168926306 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.668+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1eb790 0x55b61c1ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dce00640 1 -- 172.21.15.37:0/2168926306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 msgr2=0x55b61c250670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dce00640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c250670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dce00640 1 -- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1eb790 msgr2=0x55b61c1ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dce00640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1eb790 0x55b61c1ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dce00640 1 -- 172.21.15.37:0/2168926306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b61c1efc40 con 0x55b61c1e2290 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dd800640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1eb790 0x55b61c1ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c250670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3dce00640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c250130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8004230 tx=0x7fa3c8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62a3773a297e01d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3c6a00640 1 -- 172.21.15.37:0/2168926306 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8005110 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3d009c930 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3c6a00640 1 -- 172.21.15.37:0/2168926306 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa3c80052b0 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.669+0000 7fa3c6a00640 1 -- 172.21.15.37:0/2168926306 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c80054b0 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 10 client.27017 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b61c257970 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3c6a00640 1 -- 172.21.15.37:0/2168926306 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3d009c930 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3c6a00640 1 -- 172.21.15.37:0/2168926306 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa3c8046040 con 0x55b61c1e2290 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 10 client.27017 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 10 client.27017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 20 client.27017 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c258020 0x55b61c25a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 4 client.27017 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:18:06.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b61c14e740 con 0x55b61c258020 2024-03-21T09:18:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.670+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c258020 0x55b61c25a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.671+0000 7fa3cfe00640 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c258020 0x55b61c25a440 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58766841299508c6 server_cookie=49d32175e128606f in_seq=0 out_seq=0 2024-03-21T09:18:06.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.671+0000 7fa3c6a00640 10 client.27017 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.672+0000 7fa3c6a00640 1 -- 172.21.15.37:0/2168926306 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b61c14e740 con 0x55b61c258020 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.672+0000 7fa3c6a00640 10 client.27017 handle_command_reply: tid=0 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.672+0000 7fa3e25d1740 1 client.27017 shutdown 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 msgr2=0x55b61c250130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c250130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8004230 tx=0x7fa3c8009260 comp rx=0 tx=0).stop 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 shutdown_connections 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61c1eb790 0x55b61c1ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b61c1e2bb0 0x55b61c250670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b61c258020 0x55b61c25a440 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 --2- 172.21.15.37:0/2168926306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b61c1e2290 0x55b61c250130 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 >> 172.21.15.37:0/2168926306 conn(0x55b61c1dc9c0 msgr2=0x55b61c1db1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3c6a00640 0 client.27017 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 shutdown_connections 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.673+0000 7fa3e25d1740 1 -- 172.21.15.37:0/2168926306 wait complete. 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.674+0000 7fa3e25d1740 20 client.27017 trim_cache size 0 max 16384 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "cb7e0fb3-4664-4a5e-8ebd-027a7f72af54", 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:18:06.672 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3bc0a3090 msgr2=0x7fa3bc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3bc0a3090 0x7fa3bc0a5550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3c802efd0 tx=0x7fa3c802cf90 comp rx=0 tx=0).stop 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 msgr2=0x7fa3d819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 0x7fa3d819e7e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000eb10 tx=0x7fa3d000c2d0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 shutdown_connections 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3d810ba50 0x7fa3d819e7e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa3bc0a3090 0x7fa3bc0a5550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8102f50 0x7fa3d819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 --2- 172.21.15.37:0/2918657435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3d8102630 0x7fa3d8199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 >> 172.21.15.37:0/2918657435 conn(0x7fa3d8077130 msgr2=0x7fa3d81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 shutdown_connections 2024-03-21T09:18:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.675+0000 7fa3de200640 1 -- 172.21.15.37:0/2918657435 wait complete. 2024-03-21T09:18:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:06 smithi037 ceph-mon[31830]: pgmap v180: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-03-21T09:18:06.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:18:06.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:18:06.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:06 smithi053 ceph-mon[36169]: pgmap v180: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 -- 172.21.15.37:0/3216747283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 msgr2=0x7f6ebc1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 --2- 172.21.15.37:0/3216747283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc1018c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009900 tx=0x7f6eac02bc30 comp rx=0 tx=0).stop 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 -- 172.21.15.37:0/3216747283 shutdown_connections 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 --2- 172.21.15.37:0/3216747283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ebc101e00 0x7f6ebc10e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 --2- 172.21.15.37:0/3216747283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc1018c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 --2- 172.21.15.37:0/3216747283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ebc1074e0 0x7f6ebc1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 -- 172.21.15.37:0/3216747283 >> 172.21.15.37:0/3216747283 conn(0x7f6ebc0fcf50 msgr2=0x7f6ebc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 -- 172.21.15.37:0/3216747283 shutdown_connections 2024-03-21T09:18:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.926+0000 7f6ec3200640 1 -- 172.21.15.37:0/3216747283 wait complete. 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 Processor -- start 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 -- start start 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ebc101e00 0x7f6ebc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ebc1074e0 0x7f6ebc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc111840 con 0x7f6ebc101e00 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1116c0 con 0x7f6ebc101440 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.927+0000 7f6ec3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1119c0 con 0x7f6ebc1074e0 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45764/0 (socket says 172.21.15.37:45764) 2024-03-21T09:18:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 -- 172.21.15.37:0/1758165448 learned_addr learned my addr 172.21.15.37:0/1758165448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1400640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ebc101e00 0x7f6ebc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec2800640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ebc1074e0 0x7f6ebc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 -- 172.21.15.37:0/1758165448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ebc1074e0 msgr2=0x7f6ebc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ebc1074e0 0x7f6ebc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 -- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ebc101e00 msgr2=0x7f6ebc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ebc101e00 0x7f6ebc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 -- 172.21.15.37:0/1758165448 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc19ef70 con 0x7f6ebc101440 2024-03-21T09:18:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.928+0000 7f6ec1e00640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc199f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ea60 tx=0x7f6eb000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.929+0000 7f6eb6a00640 1 -- 172.21.15.37:0/1758165448 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000cde0 con 0x7f6ebc101440 2024-03-21T09:18:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.929+0000 7f6eb6a00640 1 -- 172.21.15.37:0/1758165448 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eb00035f0 con 0x7f6ebc101440 2024-03-21T09:18:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.929+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc19f260 con 0x7f6ebc101440 2024-03-21T09:18:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.929+0000 7f6eb6a00640 1 -- 172.21.15.37:0/1758165448 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0010670 con 0x7f6ebc101440 2024-03-21T09:18:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.929+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc1a6af0 con 0x7f6ebc101440 2024-03-21T09:18:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.932+0000 7f6ec75b1740 1 Processor -- start 2024-03-21T09:18:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 -- start start 2024-03-21T09:18:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602af7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560602a50250 con 0x560602af5460 2024-03-21T09:18:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606029f8a30 con 0x560602aecdd0 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560602a4c2f0 con 0x560602aec0f0 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aebb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45774/0 (socket says 172.21.15.37:45774) 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec2800640 1 -- 172.21.15.37:0/721400413 learned_addr learned my addr 172.21.15.37:0/721400413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6eb6a00640 1 -- 172.21.15.37:0/1758165448 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6eb00037a0 con 0x7f6ebc101440 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.933+0000 7f6ec1400640 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602af7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6ec2800640 1 -- 172.21.15.37:0/721400413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 msgr2=0x560602aec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6ec2800640 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6ec2800640 1 -- 172.21.15.37:0/721400413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 msgr2=0x560602af7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6ec2800640 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602af7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6ec2800640 1 -- 172.21.15.37:0/721400413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606029e3690 con 0x560602aecdd0 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6ec2800640 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aebb20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80039f0 tx=0x7f6eb8003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28aef18b482ca035 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6eab000640 1 -- 172.21.15.37:0/721400413 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb801a470 con 0x560602aecdd0 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6eab000640 1 -- 172.21.15.37:0/721400413 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eb8015070 con 0x560602aecdd0 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.934+0000 7f6eab000640 1 -- 172.21.15.37:0/721400413 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8010900 con 0x560602aecdd0 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6eb6a00640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6ea40a3110 0x7f6ea40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6eb6a00640 1 -- 172.21.15.37:0/1758165448 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6eb00cfdb0 con 0x7f6ebc101440 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec1e00640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6ea40a3110 0x7f6ea40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 -- 172.21.15.37:0/721400413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 msgr2=0x560602aebb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aebb20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80039f0 tx=0x7f6eb8003eb0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 -- 172.21.15.37:0/721400413 shutdown_connections 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602af7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/721400413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aebb20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.935+0000 7f6ec75b1740 1 -- 172.21.15.37:0/721400413 >> 172.21.15.37:0/721400413 conn(0x560602ab3200 msgr2=0x560602ae6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.936+0000 7f6ec75b1740 1 -- 172.21.15.37:0/721400413 shutdown_connections 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.936+0000 7f6ec75b1740 1 -- 172.21.15.37:0/721400413 wait complete. 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.936+0000 7f6ec1e00640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6ea40a3110 0x7f6ea40a55d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x560602aed320 tx=0x7f6eb0009580 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.936+0000 7f6ec75b1740 1 -- start start 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 Processor -- start 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aecdd0 0x560602af1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602af5460 0x560602af23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560602af82d0 con 0x560602aec0f0 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560602a0be80 con 0x560602af5460 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.937+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560602af8450 con 0x560602aecdd0 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec75b1740 10 client.0 _ll_register_callbacks cb 0x560602a1e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aecdd0 0x560602af1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46642/0 (socket says 172.21.15.37:46642) 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec2800640 1 -- 172.21.15.37:0/3594084143 learned_addr learned my addr 172.21.15.37:0/3594084143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec2800640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602af5460 0x560602af23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec1400640 1 -- 172.21.15.37:0/3594084143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602af5460 msgr2=0x560602af23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec1400640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602af5460 0x560602af23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec1400640 1 -- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 msgr2=0x560602af4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec1400640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec1400640 1 -- 172.21.15.37:0/3594084143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56060282e7d0 con 0x560602aecdd0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.938+0000 7f6ec2800640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6ec1400640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aecdd0 0x560602af1e60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6eac02c170 tx=0x7f6eac02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1310c63f984c2210 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6e9f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6e9f400640 1 -- 172.21.15.37:0/3594084143 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac02ccc0 con 0x560602aecdd0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6e9f400640 1 -- 172.21.15.37:0/3594084143 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6eac0046c0 con 0x560602aecdd0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eb8015070 con 0x560602aecdd0 2024-03-21T09:18:06.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6e9f400640 1 -- 172.21.15.37:0/3594084143 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac0048c0 con 0x560602aecdd0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6ec75b1740 10 client.27055 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5606027643f0 con 0x560602aecdd0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.939+0000 7f6e9f400640 1 -- 172.21.15.37:0/3594084143 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eb8015070 con 0x560602aecdd0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6e9f400640 1 -- 172.21.15.37:0/3594084143 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6eac046070 con 0x560602aecdd0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec75b1740 10 client.27055 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec75b1740 10 client.27055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec75b1740 20 client.27055 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b052d0 0x560602b076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec75b1740 4 client.27055 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560602a5d790 con 0x560602b052d0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.940+0000 7f6ec2800640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b052d0 0x560602b076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.941+0000 7f6ec2800640 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b052d0 0x560602b076e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ebeb55f7a1d4c2 server_cookie=5304475aafc14b69 in_seq=0 out_seq=0 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.941+0000 7f6e9f400640 10 client.27055 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6e9f400640 1 -- 172.21.15.37:0/3594084143 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560602a5d790 con 0x560602b052d0 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6e9f400640 10 client.27055 handle_command_reply: tid=0 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 client.27055 shutdown 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aecdd0 msgr2=0x560602af1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aecdd0 0x560602af1e60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6eac02c170 tx=0x7f6eac02c7e0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 shutdown_connections 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b052d0 0x560602b076e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602af5460 0x560602af23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aecdd0 0x560602af1e60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/3594084143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 >> 172.21.15.37:0/3594084143 conn(0x560602ab3200 msgr2=0x560602ae8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6e9f400640 0 client.27055 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 shutdown_connections 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 1 -- 172.21.15.37:0/3594084143 wait complete. 2024-03-21T09:18:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.942+0000 7f6ec75b1740 20 client.27055 trim_cache size 0 max 16384 2024-03-21T09:18:06.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.960+0000 7f6ec75b1740 1 Processor -- start 2024-03-21T09:18:06.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.960+0000 7f6ec75b1740 1 -- start start 2024-03-21T09:18:06.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602af4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602af5460 0x560602af6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1b2f40 con 0x560602aec0f0 2024-03-21T09:18:06.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1b0a40 con 0x560602aecdd0 2024-03-21T09:18:06.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1aeae0 con 0x560602af5460 2024-03-21T09:18:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602af4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602af4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45792/0 (socket says 172.21.15.37:45792) 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 -- 172.21.15.37:0/2780329991 learned_addr learned my addr 172.21.15.37:0/2780329991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec2800640 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec2800640 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602af5460 0x560602af6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 -- 172.21.15.37:0/2780329991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602af5460 msgr2=0x560602af6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602af5460 0x560602af6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 -- 172.21.15.37:0/2780329991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 msgr2=0x560602af4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec1400640 1 -- 172.21.15.37:0/2780329991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606027240f0 con 0x560602aecdd0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.961+0000 7f6ec2800640 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.962+0000 7f6ec1400640 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602af4ad0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6eac006010 tx=0x7f6eac004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ce563183c0e8e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.962+0000 7f6eaa600640 1 -- 172.21.15.37:0/2780329991 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004a60 con 0x560602aecdd0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.962+0000 7f6eaa600640 1 -- 172.21.15.37:0/2780329991 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eac004c00 con 0x560602aecdd0 2024-03-21T09:18:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.962+0000 7f6eaa600640 1 -- 172.21.15.37:0/2780329991 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0051e0 con 0x560602aecdd0 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 -- 172.21.15.37:0/2780329991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 msgr2=0x560602af4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602af4ad0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6eac006010 tx=0x7f6eac004420 comp rx=0 tx=0).stop 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 -- 172.21.15.37:0/2780329991 shutdown_connections 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602af5460 0x560602af6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602af4ad0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/2780329991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602aec0f0 0x560602af4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 -- 172.21.15.37:0/2780329991 >> 172.21.15.37:0/2780329991 conn(0x560602ab3200 msgr2=0x560602af0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 -- 172.21.15.37:0/2780329991 shutdown_connections 2024-03-21T09:18:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.963+0000 7f6ec75b1740 1 -- 172.21.15.37:0/2780329991 wait complete. 2024-03-21T09:18:06.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.964+0000 7f6ec75b1740 1 -- start start 2024-03-21T09:18:06.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.964+0000 7f6ec75b1740 1 Processor -- start 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.964+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.964+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aeea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec75b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602aeef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1af0e0 con 0x560602af5460 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1b03e0 con 0x560602aecdd0 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec75b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1af720 con 0x560602aec0f0 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec75b1740 10 client.0 _ll_register_callbacks cb 0x7f6eac004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aeea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aeea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45798/0 (socket says 172.21.15.37:45798) 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 -- 172.21.15.37:0/219076857 learned_addr learned my addr 172.21.15.37:0/219076857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec2800640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec2800640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602aeef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 -- 172.21.15.37:0/219076857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 msgr2=0x560602aee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 -- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 msgr2=0x560602aeef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602aeef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 -- 172.21.15.37:0/219076857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560602b5fb70 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6ec1400640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aeea20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6eac02c170 tx=0x7f6eac005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21081932b171574 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.965+0000 7f6e9f400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6e9f400640 1 -- 172.21.15.37:0/219076857 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004a60 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6e9f400640 1 -- 172.21.15.37:0/219076857 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6eac0046c0 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eac004c00 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6e9f400640 1 -- 172.21.15.37:0/219076857 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac004f80 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6ec75b1740 10 client.27035 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560602b60d00 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6e9f400640 1 -- 172.21.15.37:0/219076857 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eac004c00 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6e9f400640 1 -- 172.21.15.37:0/219076857 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6eac054030 con 0x560602aecdd0 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.966+0000 7f6ec75b1740 10 client.27035 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:06.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec75b1740 10 client.27035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:06.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec75b1740 20 client.27035 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:06.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b61250 0x560602b63670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:06.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec75b1740 4 client.27035 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:18:06.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560602a57ae0 con 0x560602b61250 2024-03-21T09:18:06.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec2800640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b61250 0x560602b63670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6ec2800640 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b61250 0x560602b63670 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ecb2224b96a0c8 server_cookie=706c79238159ea53 in_seq=0 out_seq=0 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.967+0000 7f6e9f400640 10 client.27035 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6e9f400640 1 -- 172.21.15.37:0/219076857 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560602a57ae0 con 0x560602b61250 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6e9f400640 10 client.27035 handle_command_reply: tid=0 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 client.27035 shutdown 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 msgr2=0x560602aeea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aeea20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6eac02c170 tx=0x7f6eac005bd0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 shutdown_connections 2024-03-21T09:18:06.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560602af5460 0x560602aeef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560602b61250 0x560602b63670 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560602aecdd0 0x560602aeea20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 --2- 172.21.15.37:0/219076857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560602aec0f0 0x560602aee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 >> 172.21.15.37:0/219076857 conn(0x560602ab3200 msgr2=0x560602ae5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6e9f400640 0 client.27035 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 shutdown_connections 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 1 -- 172.21.15.37:0/219076857 wait complete. 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.968+0000 7f6ec75b1740 20 client.27035 trim_cache size 0 max 16384 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:18:06.967 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout: "cb7e0fb3-4664-4a5e-8ebd-027a7f72af54": { 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "cb7e0fb3-4664-4a5e-8ebd-027a7f72af54", 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:18:06.968 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6ea40a3110 msgr2=0x7f6ea40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6ea40a3110 0x7f6ea40a55d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x560602aed320 tx=0x7f6eb0009580 comp rx=0 tx=0).stop 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 msgr2=0x7f6ebc199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc199f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ea60 tx=0x7f6eb000ef20 comp rx=0 tx=0).stop 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 shutdown_connections 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ebc1074e0 0x7f6ebc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ebc101e00 0x7f6ebc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6ea40a3110 0x7f6ea40a55d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 --2- 172.21.15.37:0/1758165448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ebc101440 0x7f6ebc199f80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 >> 172.21.15.37:0/1758165448 conn(0x7f6ebc0fcf50 msgr2=0x7f6ebc10cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 shutdown_connections 2024-03-21T09:18:06.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:06.969+0000 7f6ec3200640 1 -- 172.21.15.37:0/1758165448 wait complete. 2024-03-21T09:18:06.990 INFO:tasks.cephfs.filesystem:scrub status for tag:cb7e0fb3-4664-4a5e-8ebd-027a7f72af54 - {'path': '/', 'tag': 'cb7e0fb3-4664-4a5e-8ebd-027a7f72af54', 'options': 'recursive,force'} 2024-03-21T09:18:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:08 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:18:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:08 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:08 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:18:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:08 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:08 smithi071 ceph-mon[36597]: pgmap v181: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:18:08.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:08 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:18:08.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:08 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:08.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:08 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:18:08.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:08 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:08.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:08 smithi037 ceph-mon[31830]: pgmap v181: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:18:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:08 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:18:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:08 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:08 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:18:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:08 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:08.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:08 smithi053 ceph-mon[36169]: pgmap v181: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:18:10.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:10 smithi071 ceph-mon[36597]: pgmap v182: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:18:10.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:10 smithi037 ceph-mon[31830]: pgmap v182: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:18:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:10 smithi053 ceph-mon[36169]: pgmap v182: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:18:12.353 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:12 smithi071 ceph-mon[36597]: pgmap v183: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:18:12.354 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:12 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:18:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:12 smithi037 ceph-mon[31830]: pgmap v183: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:18:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:12 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:18:12.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:12 smithi053 ceph-mon[36169]: pgmap v183: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-21T09:18:12.442 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:12 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:18:14.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:14 smithi071 ceph-mon[36597]: pgmap v184: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:18:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:14 smithi037 ceph-mon[31830]: pgmap v184: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:18:14.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:14 smithi053 ceph-mon[36169]: pgmap v184: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T09:18:16.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:16 smithi071 ceph-mon[36597]: pgmap v185: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-21T09:18:16.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:16 smithi037 ceph-mon[31830]: pgmap v185: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-21T09:18:16.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:16 smithi053 ceph-mon[36169]: pgmap v185: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-21T09:18:18.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:18 smithi071 ceph-mon[36597]: pgmap v186: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T09:18:18.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:18 smithi037 ceph-mon[31830]: pgmap v186: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T09:18:18.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:18 smithi053 ceph-mon[36169]: pgmap v186: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T09:18:20.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:20 smithi071 ceph-mon[36597]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:18:20.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:20 smithi037 ceph-mon[31830]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:18:20.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:20 smithi053 ceph-mon[36169]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:18:22.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:22 smithi071 ceph-mon[36597]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:18:22.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:22 smithi037 ceph-mon[31830]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:18:22.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:22 smithi053 ceph-mon[36169]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-21T09:18:24.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:24 smithi071 ceph-mon[36597]: pgmap v189: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:18:24.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:24 smithi037 ceph-mon[31830]: pgmap v189: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:18:24.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:24 smithi053 ceph-mon[36169]: pgmap v189: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-21T09:18:26.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:26 smithi071 ceph-mon[36597]: pgmap v190: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:26.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:26 smithi037 ceph-mon[31830]: pgmap v190: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:26.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:26 smithi053 ceph-mon[36169]: pgmap v190: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:28.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:28 smithi071 ceph-mon[36597]: pgmap v191: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:28.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:28 smithi037 ceph-mon[31830]: pgmap v191: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:28.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:28 smithi053 ceph-mon[36169]: pgmap v191: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:29.603 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:29 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:18:29.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:29 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:18:29.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:29 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:18:30.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:30 smithi071 ceph-mon[36597]: pgmap v192: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:30.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:30 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:18:30.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:30 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:18:30.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:30 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:18:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:30 smithi037 ceph-mon[31830]: pgmap v192: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:30 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:18:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:30 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:18:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:30 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:18:30.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:30 smithi053 ceph-mon[36169]: pgmap v192: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:30.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:30 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:18:30.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:30 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:18:30.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:30 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:18:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:32 smithi071 ceph-mon[36597]: pgmap v193: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:32 smithi037 ceph-mon[31830]: pgmap v193: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:32.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:32 smithi053 ceph-mon[36169]: pgmap v193: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:34.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:34 smithi071 ceph-mon[36597]: pgmap v194: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:34.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:34 smithi037 ceph-mon[31830]: pgmap v194: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:34.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:34 smithi053 ceph-mon[36169]: pgmap v194: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:36.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:36 smithi071 ceph-mon[36597]: pgmap v195: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:36.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:36 smithi037 ceph-mon[31830]: pgmap v195: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:36.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:36 smithi053 ceph-mon[36169]: pgmap v195: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:36.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:18:36.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:18:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.234+0000 7f6a0e600640 1 -- 172.21.15.37:0/2798634252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08109690 msgr2=0x7f6a08109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.234+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2798634252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08109690 0x7f6a08109a70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009ca0 tx=0x7f69f802c100 comp rx=0 tx=0).stop 2024-03-21T09:18:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 -- 172.21.15.37:0/2798634252 shutdown_connections 2024-03-21T09:18:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2798634252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08104040 0x7f6a08110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2798634252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08103680 0x7f6a08103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2798634252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08109690 0x7f6a08109a70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 -- 172.21.15.37:0/2798634252 >> 172.21.15.37:0/2798634252 conn(0x7f6a080ff110 msgr2=0x7f6a08101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 -- 172.21.15.37:0/2798634252 shutdown_connections 2024-03-21T09:18:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.235+0000 7f6a0e600640 1 -- 172.21.15.37:0/2798634252 wait complete. 2024-03-21T09:18:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.236+0000 7f6a0e600640 1 Processor -- start 2024-03-21T09:18:37.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.236+0000 7f6a0e600640 1 -- start start 2024-03-21T09:18:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08103680 0x7f6a08199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08104040 0x7f6a0819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08109690 0x7f6a0819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08113aa0 con 0x7f6a08103680 2024-03-21T09:18:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0e600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08113920 con 0x7f6a08104040 2024-03-21T09:18:37.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08113c20 con 0x7f6a08109690 2024-03-21T09:18:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08103680 0x7f6a08199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f69ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08104040 0x7f6a0819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08103680 0x7f6a08199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49540/0 (socket says 172.21.15.37:49540) 2024-03-21T09:18:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08109690 0x7f6a0819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.237+0000 7f6a0d200640 1 -- 172.21.15.37:0/2935204094 learned_addr learned my addr 172.21.15.37:0/2935204094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0dc00640 1 -- 172.21.15.37:0/2935204094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08104040 msgr2=0x7f6a0819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0dc00640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08104040 0x7f6a0819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0dc00640 1 -- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08103680 msgr2=0x7f6a08199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0dc00640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08103680 0x7f6a08199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0dc00640 1 -- 172.21.15.37:0/2935204094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0819eee0 con 0x7f6a08109690 2024-03-21T09:18:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0dc00640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08109690 0x7f6a0819e7b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c970 tx=0x7f6a0000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a06000640 1 -- 172.21.15.37:0/2935204094 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00007bb0 con 0x7f6a08109690 2024-03-21T09:18:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a06000640 1 -- 172.21.15.37:0/2935204094 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a00007d50 con 0x7f6a08109690 2024-03-21T09:18:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0819f1d0 con 0x7f6a08109690 2024-03-21T09:18:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a06000640 1 -- 172.21.15.37:0/2935204094 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00018770 con 0x7f6a08109690 2024-03-21T09:18:37.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.238+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a081a6a60 con 0x7f6a08109690 2024-03-21T09:18:37.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a06000640 1 -- 172.21.15.37:0/2935204094 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6a0000f450 con 0x7f6a08109690 2024-03-21T09:18:37.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 Processor -- start 2024-03-21T09:18:37.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 -- start start 2024-03-21T09:18:37.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed7cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 0x560bfed7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed88850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bfece1250 con 0x560bfed86460 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bfec89a30 con 0x560bfed7ddd0 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.242+0000 7f6a127f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bfecdd2f0 con 0x560bfed7d0f0 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 0x560bfed7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 0x560bfed7d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56942/0 (socket says 172.21.15.37:56942) 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 -- 172.21.15.37:0/2160637082 learned_addr learned my addr 172.21.15.37:0/2160637082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f69ffe00640 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed7cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f69ffe00640 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed88850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 -- 172.21.15.37:0/2160637082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 msgr2=0x560bfed7cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed7cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 -- 172.21.15.37:0/2160637082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 msgr2=0x560bfed88850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed88850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 -- 172.21.15.37:0/2160637082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bfec74690 con 0x560bfed7d0f0 2024-03-21T09:18:37.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f69ffe00640 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed7cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:37.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.243+0000 7f6a0d200640 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 0x560bfed7d570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006010 tx=0x7f69f8009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd0b502fef851ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f6a06000640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69ec0a3110 0x7f69ec0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f69fe000640 1 -- 172.21.15.37:0/2160637082 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8045020 con 0x560bfed7d0f0 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f69fe000640 1 -- 172.21.15.37:0/2160637082 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f803a040 con 0x560bfed7d0f0 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f69fe000640 1 -- 172.21.15.37:0/2160637082 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8039040 con 0x560bfed7d0f0 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f6a06000640 1 -- 172.21.15.37:0/2935204094 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6a000cfa90 con 0x7f6a08109690 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f69ffe00640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69ec0a3110 0x7f69ec0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f6a127f6740 1 -- 172.21.15.37:0/2160637082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 msgr2=0x560bfed7d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.244+0000 7f6a127f6740 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 0x560bfed7d570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006010 tx=0x7f69f8009a10 comp rx=0 tx=0).stop 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 -- 172.21.15.37:0/2160637082 shutdown_connections 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed88850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7d0f0 0x560bfed7d570 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 --2- 172.21.15.37:0/2160637082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed7cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 -- 172.21.15.37:0/2160637082 >> 172.21.15.37:0/2160637082 conn(0x560bfed44200 msgr2=0x560bfed77b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 -- 172.21.15.37:0/2160637082 shutdown_connections 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f69ffe00640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69ec0a3110 0x7f69ec0a55d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a08108bb0 tx=0x7f69f4004730 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 -- 172.21.15.37:0/2160637082 wait complete. 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.245+0000 7f6a127f6740 1 -- start start 2024-03-21T09:18:37.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.246+0000 7f6a127f6740 1 Processor -- start 2024-03-21T09:18:37.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.246+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed85a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.246+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7ddd0 0x560bfed82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed83350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a127f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bfed89270 con 0x560bfed86460 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a127f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bfec9ce80 con 0x560bfed7d0f0 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a127f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bfed893f0 con 0x560bfed7ddd0 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a127f6740 10 client.0 _ll_register_callbacks cb 0x560bfecaf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed85a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f69ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7ddd0 0x560bfed82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed85a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45604/0 (socket says 172.21.15.37:45604) 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 -- 172.21.15.37:0/4203730376 learned_addr learned my addr 172.21.15.37:0/4203730376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0dc00640 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed83350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 -- 172.21.15.37:0/4203730376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7ddd0 msgr2=0x560bfed82e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7ddd0 0x560bfed82e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 -- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 msgr2=0x560bfed83350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed83350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.247+0000 7f6a0d200640 1 -- 172.21.15.37:0/4203730376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bfeabf7d0 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f6a0d200640 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed85a80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005e30 tx=0x7f69f80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6467e22caec5d599 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f69f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f69f2a00640 1 -- 172.21.15.37:0/4203730376 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8051070 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f69f2a00640 1 -- 172.21.15.37:0/4203730376 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f69f8032790 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69f803a040 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f69f2a00640 1 -- 172.21.15.37:0/4203730376 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f8045020 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f6a127f6740 10 client.27047 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.248+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560bfe9f53f0 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f69f2a00640 1 -- 172.21.15.37:0/4203730376 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f803a040 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f69f2a00640 1 -- 172.21.15.37:0/4203730376 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f69f804a910 con 0x560bfed7d0f0 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f6a127f6740 10 client.27047 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f6a127f6740 10 client.27047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f6a127f6740 20 client.27047 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f6a127f6740 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfed96130 0x560bfed98520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f6a127f6740 4 client.27047 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bfecee790 con 0x560bfed96130 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.249+0000 7f69ffe00640 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfed96130 0x560bfed98520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.250+0000 7f69ffe00640 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfed96130 0x560bfed98520 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e108631257385e5 server_cookie=9c74e6f40bfc0a8f in_seq=0 out_seq=0 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.250+0000 7f69f2a00640 10 client.27047 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.250+0000 7f69f2a00640 1 -- 172.21.15.37:0/4203730376 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560bfecee790 con 0x560bfed96130 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.250+0000 7f69f2a00640 10 client.27047 handle_command_reply: tid=0 2024-03-21T09:18:37.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 client.27047 shutdown 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 msgr2=0x560bfed85a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed85a80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005e30 tx=0x7f69f80095c0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 shutdown_connections 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed86460 0x560bfed83350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed7ddd0 0x560bfed82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfed96130 0x560bfed98520 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 --2- 172.21.15.37:0/4203730376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed85a80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f69f2a00640 0 client.27047 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 >> 172.21.15.37:0/4203730376 conn(0x560bfed44200 msgr2=0x560bfed79890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 shutdown_connections 2024-03-21T09:18:37.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 1 -- 172.21.15.37:0/4203730376 wait complete. 2024-03-21T09:18:37.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.251+0000 7f6a127f6740 20 client.27047 trim_cache size 0 max 16384 2024-03-21T09:18:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 Processor -- start 2024-03-21T09:18:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 -- start start 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7ddd0 0x560bfed81df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed85540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081b2e40 con 0x560bfed7ddd0 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081b0940 con 0x560bfed7d0f0 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a127f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081ae9e0 con 0x560bfed86460 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f69ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7ddd0 0x560bfed81df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a0d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed926f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45610/0 (socket says 172.21.15.37:45610) 2024-03-21T09:18:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.272+0000 7f6a0d200640 1 -- 172.21.15.37:0/238938289 learned_addr learned my addr 172.21.15.37:0/238938289 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0dc00640 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed85540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0d200640 1 -- 172.21.15.37:0/238938289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 msgr2=0x560bfed85540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0d200640 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed85540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0d200640 1 -- 172.21.15.37:0/238938289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7ddd0 msgr2=0x560bfed81df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0d200640 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7ddd0 0x560bfed81df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0d200640 1 -- 172.21.15.37:0/238938289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bfe9b50f0 con 0x560bfed7d0f0 2024-03-21T09:18:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f6a0d200640 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed926f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f69f802c710 tx=0x7f69f8003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a83a87461426a5fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.273+0000 7f69fd600640 1 -- 172.21.15.37:0/238938289 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f804e070 con 0x560bfed7d0f0 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f69fd600640 1 -- 172.21.15.37:0/238938289 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f804a930 con 0x560bfed7d0f0 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f69fd600640 1 -- 172.21.15.37:0/238938289 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8051040 con 0x560bfed7d0f0 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 -- 172.21.15.37:0/238938289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 msgr2=0x560bfed926f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed926f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f69f802c710 tx=0x7f69f8003970 comp rx=0 tx=0).stop 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 -- 172.21.15.37:0/238938289 shutdown_connections 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed85540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7ddd0 0x560bfed81df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 --2- 172.21.15.37:0/238938289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7d0f0 0x560bfed926f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.274+0000 7f6a127f6740 1 -- 172.21.15.37:0/238938289 >> 172.21.15.37:0/238938289 conn(0x560bfed44200 msgr2=0x560bfed816e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.275+0000 7f6a127f6740 1 -- 172.21.15.37:0/238938289 shutdown_connections 2024-03-21T09:18:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.275+0000 7f6a127f6740 1 -- 172.21.15.37:0/238938289 wait complete. 2024-03-21T09:18:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.275+0000 7f6a127f6740 1 -- start start 2024-03-21T09:18:37.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.275+0000 7f6a127f6740 1 Processor -- start 2024-03-21T09:18:37.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7d0f0 0x560bfed96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed97610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081aefe0 con 0x560bfed7d0f0 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081b02e0 con 0x560bfed7ddd0 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081af620 con 0x560bfed86460 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a127f6740 10 client.0 _ll_register_callbacks cb 0x7f69f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f69ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f69ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45612/0 (socket says 172.21.15.37:45612) 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f69ffe00640 1 -- 172.21.15.37:0/94762153 learned_addr learned my addr 172.21.15.37:0/94762153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.276+0000 7f6a0dc00640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed97610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f6a0d200640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7d0f0 0x560bfed96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69ffe00640 1 -- 172.21.15.37:0/94762153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 msgr2=0x560bfed97610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69ffe00640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed97610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69ffe00640 1 -- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7d0f0 msgr2=0x560bfed96b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69ffe00640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7d0f0 0x560bfed96b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69ffe00640 1 -- 172.21.15.37:0/94762153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bfedf08a0 con 0x560bfed7ddd0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f6a0d200640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7d0f0 0x560bfed96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69ffe00640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed970d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x560bfed82670 tx=0x7f69f4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717e30eae1842197 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69f2a00640 1 -- 172.21.15.37:0/94762153 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4030070 con 0x560bfed7ddd0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69f804a930 con 0x560bfed7ddd0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.277+0000 7f69f2a00640 1 -- 172.21.15.37:0/94762153 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f69f402c680 con 0x560bfed7ddd0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f69f2a00640 1 -- 172.21.15.37:0/94762153 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f403c050 con 0x560bfed7ddd0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 10 client.27053 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560bfedf1bb0 con 0x560bfed7ddd0 2024-03-21T09:18:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f69f2a00640 1 -- 172.21.15.37:0/94762153 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f804a930 con 0x560bfed7ddd0 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f69f2a00640 1 -- 172.21.15.37:0/94762153 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f69f4020680 con 0x560bfed7ddd0 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 10 client.27053 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 20 client.27053 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfedf22c0 0x560bfedf46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 4 client.27053 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.278+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560bfece8ae0 con 0x560bfedf22c0 2024-03-21T09:18:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.279+0000 7f6a0d200640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfedf22c0 0x560bfedf46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.279+0000 7f6a0d200640 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfedf22c0 0x560bfedf46e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e185b2049ac17c44 server_cookie=8b6ac169edd78131 in_seq=0 out_seq=0 2024-03-21T09:18:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.279+0000 7f69f2a00640 10 client.27053 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.279+0000 7f69f2a00640 1 -- 172.21.15.37:0/94762153 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560bfece8ae0 con 0x560bfedf22c0 2024-03-21T09:18:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.279+0000 7f69f2a00640 10 client.27053 handle_command_reply: tid=0 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.279+0000 7f6a127f6740 1 client.27053 shutdown 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 msgr2=0x560bfed970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed970d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x560bfed82670 tx=0x7f69f4005f90 comp rx=0 tx=0).stop 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 shutdown_connections 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bfed86460 0x560bfed97610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x560bfedf22c0 0x560bfedf46e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bfed7ddd0 0x560bfed970d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 --2- 172.21.15.37:0/94762153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bfed7d0f0 0x560bfed96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f69f2a00640 0 client.27053 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 >> 172.21.15.37:0/94762153 conn(0x560bfed44200 msgr2=0x560bfed88850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 shutdown_connections 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 1 -- 172.21.15.37:0/94762153 wait complete. 2024-03-21T09:18:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.280+0000 7f6a127f6740 20 client.27053 trim_cache size 0 max 16384 2024-03-21T09:18:37.279 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:18:37.279 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:18:37.279 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:18:37.279 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:18:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69ec0a3110 msgr2=0x7f69ec0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69ec0a3110 0x7f69ec0a55d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a08108bb0 tx=0x7f69f4004730 comp rx=0 tx=0).stop 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08109690 msgr2=0x7f6a0819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08109690 0x7f6a0819e7b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c970 tx=0x7f6a0000ce30 comp rx=0 tx=0).stop 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 shutdown_connections 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a08109690 0x7f6a0819e7b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69ec0a3110 0x7f69ec0a55d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a08104040 0x7f6a0819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 --2- 172.21.15.37:0/2935204094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a08103680 0x7f6a08199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 >> 172.21.15.37:0/2935204094 conn(0x7f6a080ff110 msgr2=0x7f6a080ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 shutdown_connections 2024-03-21T09:18:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.281+0000 7f6a0e600640 1 -- 172.21.15.37:0/2935204094 wait complete. 2024-03-21T09:18:37.296 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:18:37.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:18:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 -- 172.21.15.37:0/1336871773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 msgr2=0x7f5fd0073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1336871773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd0073eb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009900 tx=0x7f5fc002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 -- 172.21.15.37:0/1336871773 shutdown_connections 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1336871773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 0x7f5fd01101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1336871773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd00743f0 0x7f5fd010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1336871773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd0073eb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 -- 172.21.15.37:0/1336871773 >> 172.21.15.37:0/1336871773 conn(0x7f5fd00fd050 msgr2=0x7f5fd00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 -- 172.21.15.37:0/1336871773 shutdown_connections 2024-03-21T09:18:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.518+0000 7f5fd6400640 1 -- 172.21.15.37:0/1336871773 wait complete. 2024-03-21T09:18:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.519+0000 7f5fd6400640 1 Processor -- start 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 -- start start 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd00743f0 0x7f5fd019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 0x7f5fd019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0113b10 con 0x7f5fd00743f0 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0113990 con 0x7f5fd0073ad0 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd6400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0113c90 con 0x7f5fd010dd60 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd5000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd5000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45624/0 (socket says 172.21.15.37:45624) 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd5000640 1 -- 172.21.15.37:0/1082963885 learned_addr learned my addr 172.21.15.37:0/1082963885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fd5a00640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 0x7f5fd019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.520+0000 7f5fc7e00640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd00743f0 0x7f5fd019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5000640 1 -- 172.21.15.37:0/1082963885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 msgr2=0x7f5fd019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5000640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 0x7f5fd019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5000640 1 -- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd00743f0 msgr2=0x7f5fd019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5000640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd00743f0 0x7f5fd019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5000640 1 -- 172.21.15.37:0/1082963885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd019f020 con 0x7f5fd0073ad0 2024-03-21T09:18:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5a00640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 0x7f5fd019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd5000640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd019a030 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0000c00 tx=0x7f5fc0004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0042020 con 0x7f5fd0073ad0 2024-03-21T09:18:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc00054e0 con 0x7f5fd0073ad0 2024-03-21T09:18:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd019f2b0 con 0x7f5fd0073ad0 2024-03-21T09:18:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.521+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0003af0 con 0x7f5fd0073ad0 2024-03-21T09:18:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.522+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd01a6ba0 con 0x7f5fd0073ad0 2024-03-21T09:18:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.523+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd004f470 con 0x7f5fd0073ad0 2024-03-21T09:18:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.524+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5fc0045020 con 0x7f5fd0073ad0 2024-03-21T09:18:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.526+0000 7f5fce000640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb00a3100 0x7f5fb00a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.526+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5fc00f16d0 con 0x7f5fd0073ad0 2024-03-21T09:18:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.526+0000 7f5fc7e00640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb00a3100 0x7f5fb00a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.527+0000 7f5fc7e00640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb00a3100 0x7f5fb00a55c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0100c50 tx=0x7f5fb8005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:37.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.530+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fc004c020 con 0x7f5fd0073ad0 2024-03-21T09:18:37.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.776+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd0108ee0 con 0x7f5fd0073ad0 2024-03-21T09:18:37.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.777+0000 7f5fce000640 1 -- 172.21.15.37:0/1082963885 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f5fc00d1e20 con 0x7f5fd0073ad0 2024-03-21T09:18:37.772 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:18:37.773 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:18:37.773 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:18:37.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb00a3100 msgr2=0x7f5fb00a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb00a3100 0x7f5fb00a55c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0100c50 tx=0x7f5fb8005f90 comp rx=0 tx=0).stop 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 msgr2=0x7f5fd019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd019a030 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0000c00 tx=0x7f5fc0004360 comp rx=0 tx=0).stop 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 shutdown_connections 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd010dd60 0x7f5fd019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd00743f0 0x7f5fd019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb00a3100 0x7f5fb00a55c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 --2- 172.21.15.37:0/1082963885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0073ad0 0x7f5fd019a030 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.780+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 >> 172.21.15.37:0/1082963885 conn(0x7f5fd00fd050 msgr2=0x7f5fd00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.781+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 shutdown_connections 2024-03-21T09:18:37.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:37.781+0000 7f5fd6400640 1 -- 172.21.15.37:0/1082963885 wait complete. 2024-03-21T09:18:37.800 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:18:37.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:18:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.043+0000 7fd723a00640 1 -- 172.21.15.37:0/2179948991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c1076f0 msgr2=0x7fd71c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.043+0000 7fd723a00640 1 --2- 172.21.15.37:0/2179948991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c1076f0 0x7fd71c107ad0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd710009930 tx=0x7fd71002bc30 comp rx=0 tx=0).stop 2024-03-21T09:18:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 -- 172.21.15.37:0/2179948991 shutdown_connections 2024-03-21T09:18:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 --2- 172.21.15.37:0/2179948991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd71c06a7a0 0x7fd71c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 --2- 172.21.15.37:0/2179948991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c069de0 0x7fd71c06a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 --2- 172.21.15.37:0/2179948991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c1076f0 0x7fd71c107ad0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 -- 172.21.15.37:0/2179948991 >> 172.21.15.37:0/2179948991 conn(0x7fd71c0776d0 msgr2=0x7fd71c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 -- 172.21.15.37:0/2179948991 shutdown_connections 2024-03-21T09:18:38.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.044+0000 7fd723a00640 1 -- 172.21.15.37:0/2179948991 wait complete. 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 Processor -- start 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 -- start start 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c069de0 0x7fd71c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 0x7fd71c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd71c1076f0 0x7fd71c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c111a40 con 0x7fd71c1076f0 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1118c0 con 0x7fd71c06a7a0 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.045+0000 7fd723a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c111bc0 con 0x7fd71c069de0 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd721a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd71c1076f0 0x7fd71c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd721000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c069de0 0x7fd71c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 0x7fd71c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 0x7fd71c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45632/0 (socket says 172.21.15.37:45632) 2024-03-21T09:18:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 -- 172.21.15.37:0/2581451328 learned_addr learned my addr 172.21.15.37:0/2581451328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd721000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c069de0 0x7fd71c19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57014/0 (socket says 172.21.15.37:57014) 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 -- 172.21.15.37:0/2581451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c069de0 msgr2=0x7fd71c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c069de0 0x7fd71c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 -- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd71c1076f0 msgr2=0x7fd71c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd71c1076f0 0x7fd71c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 -- 172.21.15.37:0/2581451328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c19f0f0 con 0x7fd71c06a7a0 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.046+0000 7fd71be00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 0x7fd71c19a640 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e9f0 tx=0x7fd70800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.047+0000 7fd719600640 1 -- 172.21.15.37:0/2581451328 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70800cd80 con 0x7fd71c06a7a0 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.047+0000 7fd719600640 1 -- 172.21.15.37:0/2581451328 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd708005500 con 0x7fd71c06a7a0 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.047+0000 7fd719600640 1 -- 172.21.15.37:0/2581451328 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708010690 con 0x7fd71c06a7a0 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.047+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c19f3e0 con 0x7fd71c06a7a0 2024-03-21T09:18:38.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.047+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c1a6c70 con 0x7fd71c06a7a0 2024-03-21T09:18:38.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.051+0000 7fd726674740 1 Processor -- start 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.051+0000 7fd726674740 1 -- start start 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd719600640 1 -- 172.21.15.37:0/2581451328 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd7080046c0 con 0x7fd71c06a7a0 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd726674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 0x555ef692bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd726674740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692c080 0x555ef6934630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd726674740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6937050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd726674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef688f2c0 con 0x555ef692b6d0 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd726674740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6837c90 con 0x555ef692c080 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd726674740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef688b360 con 0x555ef6934c00 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd71be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6937050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd721a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 0x555ef692bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd721000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692c080 0x555ef6934630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd721000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692c080 0x555ef6934630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45640/0 (socket says 172.21.15.37:45640) 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd71be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6937050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57022/0 (socket says 172.21.15.37:57022) 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd721000640 1 -- 172.21.15.37:0/122538060 learned_addr learned my addr 172.21.15.37:0/122538060 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.052+0000 7fd721a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 0x555ef692bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49620/0 (socket says 172.21.15.37:49620) 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd71be00640 1 -- 172.21.15.37:0/122538060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692c080 msgr2=0x555ef6934630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd71be00640 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692c080 0x555ef6934630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd71be00640 1 -- 172.21.15.37:0/122538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 msgr2=0x555ef692bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd71be00640 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 0x555ef692bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd71be00640 1 -- 172.21.15.37:0/122538060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ef685c080 con 0x555ef6934c00 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd721a00640 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 0x555ef692bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:38.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd71be00640 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6937050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd7080cd320 tx=0x7fd7080d81a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d44e987396e4488 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd719600640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd7040a3090 0x7fd7040a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd70e000640 1 -- 172.21.15.37:0/122538060 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7080e3070 con 0x555ef6934c00 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd70e000640 1 -- 172.21.15.37:0/122538060 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7080a49d0 con 0x555ef6934c00 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.053+0000 7fd70e000640 1 -- 172.21.15.37:0/122538060 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70809acc0 con 0x555ef6934c00 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd719600640 1 -- 172.21.15.37:0/2581451328 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd7080cff70 con 0x7fd71c06a7a0 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd721a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd7040a3090 0x7fd7040a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 -- 172.21.15.37:0/122538060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 msgr2=0x555ef6937050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6937050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd7080cd320 tx=0x7fd7080d81a0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 -- 172.21.15.37:0/122538060 shutdown_connections 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6937050 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692c080 0x555ef6934630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 --2- 172.21.15.37:0/122538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692b6d0 0x555ef692bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 -- 172.21.15.37:0/122538060 >> 172.21.15.37:0/122538060 conn(0x555ef6925da0 msgr2=0x555ef6926180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 -- 172.21.15.37:0/122538060 shutdown_connections 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd721a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd7040a3090 0x7fd7040a5550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd714005fd0 tx=0x7fd7140095f0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.054+0000 7fd726674740 1 -- 172.21.15.37:0/122538060 wait complete. 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.055+0000 7fd726674740 1 -- start start 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.055+0000 7fd726674740 1 Processor -- start 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef6931930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6931fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6937b70 con 0x555ef692c080 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef684b0d0 con 0x555ef692b6d0 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef6937cf0 con 0x555ef6934c00 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd726674740 10 client.0 _ll_register_callbacks cb 0x555ef684fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd71be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef6931930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45642/0 (socket says 172.21.15.37:45642) 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 -- 172.21.15.37:0/3236271639 learned_addr learned my addr 172.21.15.37:0/3236271639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721a00640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6931fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 -- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 msgr2=0x555ef6931fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6931fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 -- 172.21.15.37:0/3236271639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 msgr2=0x555ef6931930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef6931930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd721000640 1 -- 172.21.15.37:0/3236271639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ef65a33f0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.056+0000 7fd71be00640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef6931930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd721000640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd710005ab0 tx=0x7fd71002c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c710f5e738e286b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd702a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd702a00640 1 -- 172.21.15.37:0/3236271639 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71002ccc0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd702a00640 1 -- 172.21.15.37:0/3236271639 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd7100046c0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7080a49d0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd702a00640 1 -- 172.21.15.37:0/3236271639 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7100048c0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd726674740 10 client.27071 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555ef666d7d0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.057+0000 7fd702a00640 1 -- 172.21.15.37:0/3236271639 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7080a49d0 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd702a00640 1 -- 172.21.15.37:0/3236271639 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd710046070 con 0x555ef692b6d0 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd726674740 10 client.27071 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd726674740 10 client.27071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd726674740 20 client.27071 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd726674740 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef6944a40 0x555ef6946e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd726674740 4 client.27071 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ef689c800 con 0x555ef6944a40 2024-03-21T09:18:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd71be00640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef6944a40 0x555ef6946e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd71be00640 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef6944a40 0x555ef6946e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c6851b597f925ff server_cookie=59851b20fd5fad8 in_seq=0 out_seq=0 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.058+0000 7fd702a00640 10 client.27071 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.059+0000 7fd702a00640 1 -- 172.21.15.37:0/3236271639 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ef689c800 con 0x555ef6944a40 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.059+0000 7fd702a00640 10 client.27071 handle_command_reply: tid=0 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.059+0000 7fd726674740 1 client.27071 shutdown 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.059+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 msgr2=0x555ef6934530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.059+0000 7fd726674740 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd710005ab0 tx=0x7fd71002c7e0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.059+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 shutdown_connections 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef6934c00 0x555ef6931fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef6944a40 0x555ef6946e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef6931930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 --2- 172.21.15.37:0/3236271639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934530 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 >> 172.21.15.37:0/3236271639 conn(0x555ef6925da0 msgr2=0x555ef6927af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd702a00640 0 client.27071 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 shutdown_connections 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 1 -- 172.21.15.37:0/3236271639 wait complete. 2024-03-21T09:18:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.060+0000 7fd726674740 20 client.27071 trim_cache size 0 max 16384 2024-03-21T09:18:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.073+0000 7fd726674740 1 Processor -- start 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.073+0000 7fd726674740 1 -- start start 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd726674740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd726674740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692c080 0x555ef692eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd726674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef6934c00 0x555ef692f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd726674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1b3070 con 0x555ef6934c00 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd726674740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1b0b70 con 0x555ef692b6d0 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd726674740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1aec10 con 0x555ef692c080 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd71be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692c080 0x555ef692eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45656/0 (socket says 172.21.15.37:45656) 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 -- 172.21.15.37:0/4168673869 learned_addr learned my addr 172.21.15.37:0/4168673869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd71be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692c080 0x555ef692eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57030/0 (socket says 172.21.15.37:57030) 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721a00640 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef6934c00 0x555ef692f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 -- 172.21.15.37:0/4168673869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692c080 msgr2=0x555ef692eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692c080 0x555ef692eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 -- 172.21.15.37:0/4168673869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef6934c00 msgr2=0x555ef692f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef6934c00 0x555ef692f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721000640 1 -- 172.21.15.37:0/4168673869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ef6921700 con 0x555ef692b6d0 2024-03-21T09:18:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.074+0000 7fd721a00640 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef6934c00 0x555ef692f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.075+0000 7fd721000640 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd7100095a0 tx=0x7fd7100095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f279b93ee0fe9411 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.075+0000 7fd70d600640 1 -- 172.21.15.37:0/4168673869 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710005d50 con 0x555ef692b6d0 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.075+0000 7fd70d600640 1 -- 172.21.15.37:0/4168673869 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd710003e70 con 0x555ef692b6d0 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.075+0000 7fd70d600640 1 -- 172.21.15.37:0/4168673869 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710005170 con 0x555ef692b6d0 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.075+0000 7fd726674740 1 -- 172.21.15.37:0/4168673869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 msgr2=0x555ef6934060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd7100095a0 tx=0x7fd7100095d0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 -- 172.21.15.37:0/4168673869 shutdown_connections 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef6934c00 0x555ef692f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692c080 0x555ef692eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 --2- 172.21.15.37:0/4168673869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef692b6d0 0x555ef6934060 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 -- 172.21.15.37:0/4168673869 >> 172.21.15.37:0/4168673869 conn(0x555ef6925da0 msgr2=0x555ef6924eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 -- 172.21.15.37:0/4168673869 shutdown_connections 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 -- 172.21.15.37:0/4168673869 wait complete. 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 -- start start 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.076+0000 7fd726674740 1 Processor -- start 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 0x555ef693a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef693aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef6934c00 0x555ef693b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1af210 con 0x555ef692c080 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1b0510 con 0x555ef6934c00 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1af850 con 0x555ef692b6d0 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd726674740 10 client.0 _ll_register_callbacks cb 0x7fd710009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd721000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 0x555ef693a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd721000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 0x555ef693a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57034/0 (socket says 172.21.15.37:57034) 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd721000640 1 -- 172.21.15.37:0/4211536911 learned_addr learned my addr 172.21.15.37:0/4211536911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd71be00640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef693aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.077+0000 7fd721a00640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef6934c00 0x555ef693b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd721000640 1 -- 172.21.15.37:0/4211536911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef6934c00 msgr2=0x555ef693b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd721000640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef6934c00 0x555ef693b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd721000640 1 -- 172.21.15.37:0/4211536911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 msgr2=0x555ef693aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd721000640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef693aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd721000640 1 -- 172.21.15.37:0/4211536911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ef699f580 con 0x555ef692b6d0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd721000640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 0x555ef693a590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd710003820 tx=0x7fd710000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618faf2f3a2f5ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd702a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd702a00640 1 -- 172.21.15.37:0/4211536911 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710005d50 con 0x555ef692b6d0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd702a00640 1 -- 172.21.15.37:0/4211536911 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd710003b50 con 0x555ef692b6d0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd710003e70 con 0x555ef692b6d0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd702a00640 1 -- 172.21.15.37:0/4211536911 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd710004f10 con 0x555ef692b6d0 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd726674740 10 client.27103 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.078+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555ef69a0860 con 0x555ef692b6d0 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd702a00640 1 -- 172.21.15.37:0/4211536911 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd710003e70 con 0x555ef692b6d0 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd702a00640 1 -- 172.21.15.37:0/4211536911 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd710054030 con 0x555ef692b6d0 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd726674740 10 client.27103 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd726674740 10 client.27103 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd726674740 20 client.27103 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd726674740 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef69a0ce0 0x555ef69a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd726674740 4 client.27103 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ef6896b50 con 0x555ef69a0ce0 2024-03-21T09:18:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.079+0000 7fd71be00640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef69a0ce0 0x555ef69a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd71be00640 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef69a0ce0 0x555ef69a30d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afb6049aeefb673b server_cookie=9e5addfe45e2e8e in_seq=0 out_seq=0 2024-03-21T09:18:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd702a00640 10 client.27103 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd702a00640 1 -- 172.21.15.37:0/4211536911 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ef6896b50 con 0x555ef69a0ce0 2024-03-21T09:18:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd702a00640 10 client.27103 handle_command_reply: tid=0 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 client.27103 shutdown 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 msgr2=0x555ef693a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 0x555ef693a590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd710003820 tx=0x7fd710000bc0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 shutdown_connections 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ef6934c00 0x555ef693b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x555ef69a0ce0 0x555ef69a30d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ef692c080 0x555ef693aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 --2- 172.21.15.37:0/4211536911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555ef692b6d0 0x555ef693a590 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd702a00640 0 client.27103 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 >> 172.21.15.37:0/4211536911 conn(0x555ef6925da0 msgr2=0x555ef692c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 shutdown_connections 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.080+0000 7fd726674740 1 -- 172.21.15.37:0/4211536911 wait complete. 2024-03-21T09:18:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd726674740 20 client.27103 trim_cache size 0 max 16384 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd7040a3090 msgr2=0x7fd7040a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd7040a3090 0x7fd7040a5550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd714005fd0 tx=0x7fd7140095f0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 msgr2=0x7fd71c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 0x7fd71c19a640 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e9f0 tx=0x7fd70800edf0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 shutdown_connections 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd71c1076f0 0x7fd71c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd7040a3090 0x7fd7040a5550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd71c06a7a0 0x7fd71c19a640 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 --2- 172.21.15.37:0/2581451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd71c069de0 0x7fd71c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 >> 172.21.15.37:0/2581451328 conn(0x7fd71c0776d0 msgr2=0x7fd71c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 shutdown_connections 2024-03-21T09:18:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.081+0000 7fd723a00640 1 -- 172.21.15.37:0/2581451328 wait complete. 2024-03-21T09:18:38.094 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:18:38.094 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 -- 172.21.15.37:0/2866401743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 msgr2=0x7fec40102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 --2- 172.21.15.37:0/2866401743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40102af0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fec30009900 tx=0x7fec3002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 -- 172.21.15.37:0/2866401743 shutdown_connections 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 --2- 172.21.15.37:0/2866401743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4010ba50 0x7fec4010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 --2- 172.21.15.37:0/2866401743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec40103030 0x7fec4010b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 --2- 172.21.15.37:0/2866401743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40102af0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 -- 172.21.15.37:0/2866401743 >> 172.21.15.37:0/2866401743 conn(0x7fec40077130 msgr2=0x7fec400fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 -- 172.21.15.37:0/2866401743 shutdown_connections 2024-03-21T09:18:38.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.324+0000 7fec3f200640 1 -- 172.21.15.37:0/2866401743 wait complete. 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 Processor -- start 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 -- start start 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec40103030 0x7fec4019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4010ba50 0x7fec4019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40111870 con 0x7fec4010ba50 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401116f0 con 0x7fec40102710 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401119f0 con 0x7fec40103030 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45690/0 (socket says 172.21.15.37:45690) 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3de00640 1 -- 172.21.15.37:0/2632471351 learned_addr learned my addr 172.21.15.37:0/2632471351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3e800640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4010ba50 0x7fec4019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.326+0000 7fec3d400640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec40103030 0x7fec4019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3de00640 1 -- 172.21.15.37:0/2632471351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec40103030 msgr2=0x7fec4019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3de00640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec40103030 0x7fec4019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3de00640 1 -- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4010ba50 msgr2=0x7fec4019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3de00640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4010ba50 0x7fec4019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3de00640 1 -- 172.21.15.37:0/2632471351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4019ef00 con 0x7fec40102710 2024-03-21T09:18:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3de00640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40199f10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fec30000c00 tx=0x7fec300043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec36a00640 1 -- 172.21.15.37:0/2632471351 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30042020 con 0x7fec40102710 2024-03-21T09:18:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec36a00640 1 -- 172.21.15.37:0/2632471351 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec300054e0 con 0x7fec40102710 2024-03-21T09:18:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4019f190 con 0x7fec40102710 2024-03-21T09:18:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.327+0000 7fec36a00640 1 -- 172.21.15.37:0/2632471351 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30003af0 con 0x7fec40102710 2024-03-21T09:18:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.328+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec401a6a80 con 0x7fec40102710 2024-03-21T09:18:38.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.330+0000 7fec36a00640 1 -- 172.21.15.37:0/2632471351 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fec30045020 con 0x7fec40102710 2024-03-21T09:18:38.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.331+0000 7fec47d8c740 1 Processor -- start 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.331+0000 7fec47d8c740 1 -- start start 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec36a00640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fec240a3110 0x7fec240a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186847c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186850810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186853230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec47d8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611867ab2c0 con 0x5611868478b0 2024-03-21T09:18:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec47d8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561186753c90 con 0x561186848260 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec47d8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611867a7360 con 0x561186850de0 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec36a00640 1 -- 172.21.15.37:0/2632471351 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fec300f17a0 con 0x7fec40102710 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec3de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186847c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec3e800640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fec240a3110 0x7fec240a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec3d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186850810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186853230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec3de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186847c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49690/0 (socket says 172.21.15.37:49690) 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.332+0000 7fec3de00640 1 -- 172.21.15.37:0/3406033738 learned_addr learned my addr 172.21.15.37:0/3406033738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3d400640 1 -- 172.21.15.37:0/3406033738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 msgr2=0x561186853230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3d400640 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186853230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3d400640 1 -- 172.21.15.37:0/3406033738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 msgr2=0x561186847c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3d400640 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186847c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3d400640 1 -- 172.21.15.37:0/3406033738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561186778080 con 0x561186848260 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3d400640 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186850810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00bbc0 tx=0x7fec2c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ccb7ea9d4a5f71b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec3e800640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fec240a3110 0x7fec240a55d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec38009fd0 tx=0x7fec38009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec2b400640 1 -- 172.21.15.37:0/3406033738 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c004030 con 0x561186848260 2024-03-21T09:18:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec2b400640 1 -- 172.21.15.37:0/3406033738 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec2c0041d0 con 0x561186848260 2024-03-21T09:18:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.333+0000 7fec2b400640 1 -- 172.21.15.37:0/3406033738 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0116a0 con 0x561186848260 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 -- 172.21.15.37:0/3406033738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 msgr2=0x561186850810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186850810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00bbc0 tx=0x7fec2c0107a0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 -- 172.21.15.37:0/3406033738 shutdown_connections 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186853230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186850810 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 --2- 172.21.15.37:0/3406033738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186847c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.334+0000 7fec47d8c740 1 -- 172.21.15.37:0/3406033738 >> 172.21.15.37:0/3406033738 conn(0x561186841f80 msgr2=0x561186842360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.335+0000 7fec47d8c740 1 -- 172.21.15.37:0/3406033738 shutdown_connections 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.335+0000 7fec47d8c740 1 -- 172.21.15.37:0/3406033738 wait complete. 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.335+0000 7fec47d8c740 1 -- start start 2024-03-21T09:18:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 Processor -- start 2024-03-21T09:18:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186850720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 0x56118684dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186850de0 0x56118684e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561186853d50 con 0x5611868478b0 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611867670d0 con 0x561186850de0 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561186853ed0 con 0x561186848260 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.336+0000 7fec47d8c740 10 client.0 _ll_register_callbacks cb 0x56118676be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 0x56118684dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 0x56118684dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57076/0 (socket says 172.21.15.37:57076) 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 -- 172.21.15.37:0/1496763560 learned_addr learned my addr 172.21.15.37:0/1496763560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3de00640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186850720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3e800640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186850de0 0x56118684e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 -- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186850de0 msgr2=0x56118684e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186850de0 0x56118684e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 -- 172.21.15.37:0/1496763560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 msgr2=0x561186850720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186850720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 -- 172.21.15.37:0/1496763560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611864bf3f0 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec3d400640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 0x56118684dab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005890 tx=0x7fec2c004140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=986f696693265308 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.337+0000 7fec1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec1fe00640 1 -- 172.21.15.37:0/1496763560 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c02a060 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec2c0041d0 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec1fe00640 1 -- 172.21.15.37:0/1496763560 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fec2c0035f0 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec1fe00640 1 -- 172.21.15.37:0/1496763560 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2c031050 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec47d8c740 10 client.27115 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5611865897d0 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec1fe00640 1 -- 172.21.15.37:0/1496763560 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec2c0041d0 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.338+0000 7fec1fe00640 1 -- 172.21.15.37:0/1496763560 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fec2c0046c0 con 0x561186848260 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec47d8c740 10 client.27115 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec47d8c740 10 client.27115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:18:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec47d8c740 20 client.27115 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561186860b40 0x561186862f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec47d8c740 4 client.27115 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611867b8800 con 0x561186860b40 2024-03-21T09:18:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec3de00640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561186860b40 0x561186862f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec1fe00640 10 client.27115 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.339+0000 7fec3de00640 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561186860b40 0x561186862f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41754cc9f236e76 server_cookie=42d482fe10772292 in_seq=0 out_seq=0 2024-03-21T09:18:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec1fe00640 1 -- 172.21.15.37:0/1496763560 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611867b8800 con 0x561186860b40 2024-03-21T09:18:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec1fe00640 10 client.27115 handle_command_reply: tid=0 2024-03-21T09:18:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec47d8c740 1 client.27115 shutdown 2024-03-21T09:18:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 msgr2=0x56118684dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 0x56118684dab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005890 tx=0x7fec2c004140 comp rx=0 tx=0).stop 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 shutdown_connections 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.340+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x561186860b40 0x561186862f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186850de0 0x56118684e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186848260 0x56118684dab0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1496763560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186850720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec1fe00640 0 client.27115 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 >> 172.21.15.37:0/1496763560 conn(0x561186841f80 msgr2=0x561186843cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 shutdown_connections 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 1 -- 172.21.15.37:0/1496763560 wait complete. 2024-03-21T09:18:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.341+0000 7fec47d8c740 20 client.27115 trim_cache size 0 max 16384 2024-03-21T09:18:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.354+0000 7fec47d8c740 1 Processor -- start 2024-03-21T09:18:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 -- start start 2024-03-21T09:18:38.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 0x5611868501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561186848260 0x561186850720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x56118684ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401b2e60 con 0x561186848260 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401b0960 con 0x5611868478b0 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec47d8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401aea00 con 0x561186850de0 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec3de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 0x5611868501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x56118684ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec3de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 0x5611868501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45722/0 (socket says 172.21.15.37:45722) 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec3de00640 1 -- 172.21.15.37:0/1009216427 learned_addr learned my addr 172.21.15.37:0/1009216427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x56118684ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57078/0 (socket says 172.21.15.37:57078) 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.355+0000 7fec3d400640 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561186848260 0x561186850720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3de00640 1 -- 172.21.15.37:0/1009216427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 msgr2=0x56118684ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3de00640 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x56118684ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3de00640 1 -- 172.21.15.37:0/1009216427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561186848260 msgr2=0x561186850720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3de00640 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561186848260 0x561186850720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3de00640 1 -- 172.21.15.37:0/1009216427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118684e6c0 con 0x5611868478b0 2024-03-21T09:18:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3d400640 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561186848260 0x561186850720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3e800640 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x56118684ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec3de00640 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 0x5611868501e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fec300ba270 tx=0x7fec30009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2424c34d467dbb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec2aa00640 1 -- 172.21.15.37:0/1009216427 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300bdfa0 con 0x5611868478b0 2024-03-21T09:18:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec2aa00640 1 -- 172.21.15.37:0/1009216427 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec300bd8a0 con 0x5611868478b0 2024-03-21T09:18:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.356+0000 7fec2aa00640 1 -- 172.21.15.37:0/1009216427 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300b9f10 con 0x5611868478b0 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 -- 172.21.15.37:0/1009216427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 msgr2=0x5611868501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 0x5611868501e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fec300ba270 tx=0x7fec30009260 comp rx=0 tx=0).stop 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 -- 172.21.15.37:0/1009216427 shutdown_connections 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x56118684ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561186848260 0x561186850720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1009216427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611868478b0 0x5611868501e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 -- 172.21.15.37:0/1009216427 >> 172.21.15.37:0/1009216427 conn(0x561186841f80 msgr2=0x561186841160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 -- 172.21.15.37:0/1009216427 shutdown_connections 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 -- 172.21.15.37:0/1009216427 wait complete. 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.357+0000 7fec47d8c740 1 -- start start 2024-03-21T09:18:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 Processor -- start 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186865d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186864140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186864680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401af000 con 0x5611868478b0 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401b0300 con 0x561186848260 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401af640 con 0x561186850de0 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec47d8c740 10 client.0 _ll_register_callbacks cb 0x7fec30005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec3de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186865d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec3d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186864140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186864680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec3d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186864140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45734/0 (socket says 172.21.15.37:45734) 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.358+0000 7fec3d400640 1 -- 172.21.15.37:0/1952880231 learned_addr learned my addr 172.21.15.37:0/1952880231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec3d400640 1 -- 172.21.15.37:0/1952880231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 msgr2=0x561186864680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec3d400640 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186864680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec3d400640 1 -- 172.21.15.37:0/1952880231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 msgr2=0x561186865d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec3d400640 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186865d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec3d400640 1 -- 172.21.15.37:0/1952880231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561186864f00 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec3d400640 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186864140 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005940 tx=0x7fec2c00b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38b44baaa71195c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec1fe00640 1 -- 172.21.15.37:0/1952880231 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c039070 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec300bd8a0 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.359+0000 7fec1fe00640 1 -- 172.21.15.37:0/1952880231 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fec2c0053a0 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec1fe00640 1 -- 172.21.15.37:0/1952880231 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2c02a040 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 10 client.27095 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5611868bd100 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec1fe00640 1 -- 172.21.15.37:0/1952880231 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec300bd8a0 con 0x561186848260 2024-03-21T09:18:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec1fe00640 1 -- 172.21.15.37:0/1952880231 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fec2c03c020 con 0x561186848260 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 10 client.27095 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 10 client.27095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 20 client.27095 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5611868bd820 0x5611868bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 4 client.27095 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5611867b2b50 con 0x5611868bd820 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.360+0000 7fec3de00640 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5611868bd820 0x5611868bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec3de00640 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5611868bd820 0x5611868bfc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=430b34501d035a7e server_cookie=85a6534e3eb7532e in_seq=0 out_seq=0 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec1fe00640 10 client.27095 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec1fe00640 1 -- 172.21.15.37:0/1952880231 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5611867b2b50 con 0x5611868bd820 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec1fe00640 10 client.27095 handle_command_reply: tid=0 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 client.27095 shutdown 2024-03-21T09:18:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 msgr2=0x561186864140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186864140 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec2c005940 tx=0x7fec2c00b940 comp rx=0 tx=0).stop 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 shutdown_connections 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5611868bd820 0x5611868bfc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561186850de0 0x561186864680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561186848260 0x561186864140 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 --2- 172.21.15.37:0/1952880231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611868478b0 0x561186865d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec1fe00640 0 client.27095 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 >> 172.21.15.37:0/1952880231 conn(0x561186841f80 msgr2=0x5611868430d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 shutdown_connections 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.361+0000 7fec47d8c740 1 -- 172.21.15.37:0/1952880231 wait complete. 2024-03-21T09:18:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec47d8c740 20 client.27095 trim_cache size 0 max 16384 2024-03-21T09:18:38.362 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fec240a3110 msgr2=0x7fec240a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fec240a3110 0x7fec240a55d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec38009fd0 tx=0x7fec38009500 comp rx=0 tx=0).stop 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 msgr2=0x7fec40199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40199f10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fec30000c00 tx=0x7fec300043c0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 shutdown_connections 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4010ba50 0x7fec4019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec40103030 0x7fec4019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fec240a3110 0x7fec240a55d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 --2- 172.21.15.37:0/2632471351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40102710 0x7fec40199f10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 >> 172.21.15.37:0/2632471351 conn(0x7fec40077130 msgr2=0x7fec40100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.362+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 shutdown_connections 2024-03-21T09:18:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.363+0000 7fec3f200640 1 -- 172.21.15.37:0/2632471351 wait complete. 2024-03-21T09:18:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:38 smithi037 ceph-mon[31830]: pgmap v196: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1082963885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:18:38.375 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:18:38.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:18:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 -- 172.21.15.37:0/1501422704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 msgr2=0x7fb29010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 --2- 172.21.15.37:0/1501422704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29010ded0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb288009900 tx=0x7fb28802bcc0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 -- 172.21.15.37:0/1501422704 shutdown_connections 2024-03-21T09:18:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 --2- 172.21.15.37:0/1501422704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29010ded0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 --2- 172.21.15.37:0/1501422704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29010b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 --2- 172.21.15.37:0/1501422704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb290102630 0x7fb290102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 -- 172.21.15.37:0/1501422704 >> 172.21.15.37:0/1501422704 conn(0x7fb290077130 msgr2=0x7fb2900fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 -- 172.21.15.37:0/1501422704 shutdown_connections 2024-03-21T09:18:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.593+0000 7fb28f200640 1 -- 172.21.15.37:0/1501422704 wait complete. 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 Processor -- start 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 -- start start 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb290102630 0x7fb290199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290111860 con 0x7fb290102630 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901116e0 con 0x7fb290102f50 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.594+0000 7fb28f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901119e0 con 0x7fb29010ba50 2024-03-21T09:18:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb290102630 0x7fb290199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45750/0 (socket says 172.21.15.37:45750) 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57114/0 (socket says 172.21.15.37:57114) 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28d400640 1 -- 172.21.15.37:0/3163728901 learned_addr learned my addr 172.21.15.37:0/3163728901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 -- 172.21.15.37:0/3163728901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 msgr2=0x7fb29019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 -- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb290102630 msgr2=0x7fb290199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb290102630 0x7fb290199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 -- 172.21.15.37:0/3163728901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29019eee0 con 0x7fb29010ba50 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28d400640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.595+0000 7fb28e800640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29019e7b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb288036040 tx=0x7fb2880042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.596+0000 7fb286a00640 1 -- 172.21.15.37:0/3163728901 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288042020 con 0x7fb29010ba50 2024-03-21T09:18:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.596+0000 7fb286a00640 1 -- 172.21.15.37:0/3163728901 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2880044d0 con 0x7fb29010ba50 2024-03-21T09:18:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.596+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29019f170 con 0x7fb29010ba50 2024-03-21T09:18:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.596+0000 7fb286a00640 1 -- 172.21.15.37:0/3163728901 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288003af0 con 0x7fb29010ba50 2024-03-21T09:18:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.596+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901a6a60 con 0x7fb29010ba50 2024-03-21T09:18:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.599+0000 7fb286a00640 1 -- 172.21.15.37:0/3163728901 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb288045020 con 0x7fb29010ba50 2024-03-21T09:18:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.599+0000 7fb297b81740 1 Processor -- start 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.599+0000 7fb297b81740 1 -- start start 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb297b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa3cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb297b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb297b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa48230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb297b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174f9a02c0 con 0x56174fa3c8b0 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb297b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174f948c90 con 0x56174fa45de0 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb297b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174f99c360 con 0x56174fa3d260 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa3cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa45810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57116/0 (socket says 172.21.15.37:57116) 2024-03-21T09:18:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 -- 172.21.15.37:0/1905692833 learned_addr learned my addr 172.21.15.37:0/1905692833 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28d400640 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa48230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 -- 172.21.15.37:0/1905692833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 msgr2=0x56174fa48230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa48230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 -- 172.21.15.37:0/1905692833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 msgr2=0x56174fa3cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa3cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.600+0000 7fb28de00640 1 -- 172.21.15.37:0/1905692833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174f96d080 con 0x56174fa3d260 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb28d400640 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa48230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb28de00640 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa45810 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb2800114f0 tx=0x7fb2800119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594e735675b3fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb286a00640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb2740a3110 0x7fb2740a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb27b400640 1 -- 172.21.15.37:0/1905692833 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28001e070 con 0x56174fa3d260 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb27b400640 1 -- 172.21.15.37:0/1905692833 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb28000fb00 con 0x56174fa3d260 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb27b400640 1 -- 172.21.15.37:0/1905692833 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280019bf0 con 0x56174fa3d260 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb286a00640 1 -- 172.21.15.37:0/3163728901 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb2880f14c0 con 0x7fb29010ba50 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.601+0000 7fb28d400640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb2740a3110 0x7fb2740a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 -- 172.21.15.37:0/1905692833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 msgr2=0x56174fa45810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb28d400640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb2740a3110 0x7fb2740a55d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb27c005f60 tx=0x7fb27c009570 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa45810 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb2800114f0 tx=0x7fb2800119b0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 -- 172.21.15.37:0/1905692833 shutdown_connections 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa48230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa45810 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 --2- 172.21.15.37:0/1905692833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa3cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 -- 172.21.15.37:0/1905692833 >> 172.21.15.37:0/1905692833 conn(0x56174fa36f80 msgr2=0x56174fa37360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 -- 172.21.15.37:0/1905692833 shutdown_connections 2024-03-21T09:18:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.602+0000 7fb297b81740 1 -- 172.21.15.37:0/1905692833 wait complete. 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.603+0000 7fb297b81740 1 -- start start 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.603+0000 7fb297b81740 1 Processor -- start 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa42ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174fa48d50 con 0x56174fa3c8b0 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174f95c0d0 con 0x56174fa45de0 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174fa48ed0 con 0x56174fa3d260 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb297b81740 10 client.0 _ll_register_callbacks cb 0x56174f960e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb28d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa42ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb28d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa42ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57122/0 (socket says 172.21.15.37:57122) 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb28d400640 1 -- 172.21.15.37:0/3232512952 learned_addr learned my addr 172.21.15.37:0/3232512952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb28de00640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.604+0000 7fb28e800640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28d400640 1 -- 172.21.15.37:0/3232512952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 msgr2=0x56174fa430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28d400640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28d400640 1 -- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 msgr2=0x56174fa45750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28d400640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa45750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28d400640 1 -- 172.21.15.37:0/3232512952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56174f6b43f0 con 0x56174fa3d260 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28e800640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb28d400640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa42ae0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb27c021620 tx=0x7fb27c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=227d119a7829ccd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb26fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb26fe00640 1 -- 172.21.15.37:0/3232512952 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c02d480 con 0x56174fa3d260 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb28000fb00 con 0x56174fa3d260 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.605+0000 7fb26fe00640 1 -- 172.21.15.37:0/3232512952 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb27c02d620 con 0x56174fa3d260 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb26fe00640 1 -- 172.21.15.37:0/3232512952 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb27c02d860 con 0x56174fa3d260 2024-03-21T09:18:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 10 client.27133 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56174f77e7d0 con 0x56174fa3d260 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb26fe00640 1 -- 172.21.15.37:0/3232512952 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb28000fb00 con 0x56174fa3d260 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb26fe00640 1 -- 172.21.15.37:0/3232512952 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb27c01f5f0 con 0x56174fa3d260 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 10 client.27133 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 20 client.27133 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fa55ca0 0x56174fa58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 4 client.27133 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.606+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56174f9ad800 con 0x56174fa55ca0 2024-03-21T09:18:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.607+0000 7fb28de00640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fa55ca0 0x56174fa58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:38 smithi071 ceph-mon[36597]: pgmap v196: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:38.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1082963885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.607+0000 7fb28de00640 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fa55ca0 0x56174fa58090 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68562bf14166b9ee server_cookie=261227c7cb9b4084 in_seq=0 out_seq=0 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.607+0000 7fb26fe00640 10 client.27133 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.608+0000 7fb26fe00640 1 -- 172.21.15.37:0/3232512952 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56174f9ad800 con 0x56174fa55ca0 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.608+0000 7fb26fe00640 10 client.27133 handle_command_reply: tid=0 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.608+0000 7fb297b81740 1 client.27133 shutdown 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 msgr2=0x56174fa42ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa42ae0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb27c021620 tx=0x7fb27c005f90 comp rx=0 tx=0).stop 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 shutdown_connections 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fa55ca0 0x56174fa58090 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fa430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3d260 0x56174fa42ae0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 --2- 172.21.15.37:0/3232512952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3c8b0 0x56174fa45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 >> 172.21.15.37:0/3232512952 conn(0x56174fa36f80 msgr2=0x56174fa38cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb26fe00640 0 client.27133 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 shutdown_connections 2024-03-21T09:18:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 1 -- 172.21.15.37:0/3232512952 wait complete. 2024-03-21T09:18:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.609+0000 7fb297b81740 20 client.27133 trim_cache size 0 max 16384 2024-03-21T09:18:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.632+0000 7fb297b81740 1 Processor -- start 2024-03-21T09:18:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.632+0000 7fb297b81740 1 -- start start 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb297b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb297b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa3d260 0x56174fa45210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb297b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa45de0 0x56174fa45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb297b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901b2f00 con 0x56174fa45de0 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb297b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901b0a00 con 0x56174fa3d260 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb297b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901aeaa0 con 0x56174fa3c8b0 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa3d260 0x56174fa45210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa320a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57124/0 (socket says 172.21.15.37:57124) 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 -- 172.21.15.37:0/84878774 learned_addr learned my addr 172.21.15.37:0/84878774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28e800640 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa45de0 0x56174fa45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 -- 172.21.15.37:0/84878774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa3d260 msgr2=0x56174fa45210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa3d260 0x56174fa45210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 -- 172.21.15.37:0/84878774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa45de0 msgr2=0x56174fa45750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa45de0 0x56174fa45750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28de00640 1 -- 172.21.15.37:0/84878774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174fa58400 con 0x56174fa3c8b0 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.633+0000 7fb28e800640 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa45de0 0x56174fa45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.634+0000 7fb28de00640 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa320a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb280011eb0 tx=0x7fb280019750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54d2134404bebc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.634+0000 7fb27aa00640 1 -- 172.21.15.37:0/84878774 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280029070 con 0x56174fa3c8b0 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.634+0000 7fb27aa00640 1 -- 172.21.15.37:0/84878774 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb280003e50 con 0x56174fa3c8b0 2024-03-21T09:18:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.634+0000 7fb27aa00640 1 -- 172.21.15.37:0/84878774 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28001e040 con 0x56174fa3c8b0 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 -- 172.21.15.37:0/84878774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 msgr2=0x56174fa320a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa320a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb280011eb0 tx=0x7fb280019750 comp rx=0 tx=0).stop 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 -- 172.21.15.37:0/84878774 shutdown_connections 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa45de0 0x56174fa45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa3d260 0x56174fa45210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 --2- 172.21.15.37:0/84878774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa320a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 -- 172.21.15.37:0/84878774 >> 172.21.15.37:0/84878774 conn(0x56174fa36f80 msgr2=0x56174fa36160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 -- 172.21.15.37:0/84878774 shutdown_connections 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 -- 172.21.15.37:0/84878774 wait complete. 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.635+0000 7fb297b81740 1 -- start start 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 Processor -- start 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa4b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3d260 0x56174fa4bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fab0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901af0a0 con 0x56174fa3d260 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901b03a0 con 0x56174fa45de0 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901af6e0 con 0x56174fa3c8b0 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.636+0000 7fb297b81740 10 client.0 _ll_register_callbacks cb 0x7fb27c0135f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa4b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fab0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa4b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57130/0 (socket says 172.21.15.37:57130) 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28de00640 1 -- 172.21.15.37:0/3383020132 learned_addr learned my addr 172.21.15.37:0/3383020132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28d400640 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3d260 0x56174fa4bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 -- 172.21.15.37:0/3383020132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 msgr2=0x56174fa4b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa4b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 -- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3d260 msgr2=0x56174fa4bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3d260 0x56174fa4bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 -- 172.21.15.37:0/3383020132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56174fa4c6b0 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb28e800640 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fab0530 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb2880ee920 tx=0x7fb2880bd4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968a1f26b1e05c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb26fe00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb26fe00640 1 -- 172.21.15.37:0/3383020132 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880bd5c0 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb26fe00640 1 -- 172.21.15.37:0/3383020132 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb288005370 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.637+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb280003e50 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb26fe00640 1 -- 172.21.15.37:0/3383020132 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2880c1710 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 10 client.27113 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56174fab1bd0 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb26fe00640 1 -- 172.21.15.37:0/3383020132 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb280003e50 con 0x56174fa45de0 2024-03-21T09:18:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb26fe00640 1 -- 172.21.15.37:0/3383020132 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb288004f20 con 0x56174fa45de0 2024-03-21T09:18:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 10 client.27113 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 10 client.27113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 20 client.27113 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fab2250 0x56174fab4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 4 client.27113 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.638+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56174f9a7b50 con 0x56174fab2250 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.639+0000 7fb28de00640 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fab2250 0x56174fab4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.639+0000 7fb28de00640 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fab2250 0x56174fab4640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0d5d25068a4ff4 server_cookie=35362e279b4847cc in_seq=0 out_seq=0 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.639+0000 7fb26fe00640 10 client.27113 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.639+0000 7fb26fe00640 1 -- 172.21.15.37:0/3383020132 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56174f9a7b50 con 0x56174fab2250 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.639+0000 7fb26fe00640 10 client.27113 handle_command_reply: tid=0 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.639+0000 7fb297b81740 1 client.27113 shutdown 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 msgr2=0x56174fab0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fab0530 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb2880ee920 tx=0x7fb2880bd4a0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 shutdown_connections 2024-03-21T09:18:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56174fab2250 0x56174fab4640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56174fa45de0 0x56174fab0530 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174fa3d260 0x56174fa4bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb26fe00640 0 client.27113 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 --2- 172.21.15.37:0/3383020132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56174fa3c8b0 0x56174fa4b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 >> 172.21.15.37:0/3383020132 conn(0x56174fa36f80 msgr2=0x56174fa36160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 shutdown_connections 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 1 -- 172.21.15.37:0/3383020132 wait complete. 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.640+0000 7fb297b81740 20 client.27113 trim_cache size 0 max 16384 2024-03-21T09:18:38.638 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:18:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb2740a3110 msgr2=0x7fb2740a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb2740a3110 0x7fb2740a55d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb27c005f60 tx=0x7fb27c009570 comp rx=0 tx=0).stop 2024-03-21T09:18:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 msgr2=0x7fb29019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29019e7b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb288036040 tx=0x7fb2880042e0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 shutdown_connections 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb29010ba50 0x7fb29019e7b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb2740a3110 0x7fb2740a55d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290102f50 0x7fb29019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 --2- 172.21.15.37:0/3163728901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb290102630 0x7fb290199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 >> 172.21.15.37:0/3163728901 conn(0x7fb290077130 msgr2=0x7fb2901001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 shutdown_connections 2024-03-21T09:18:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:38.641+0000 7fb28f200640 1 -- 172.21.15.37:0/3163728901 wait complete. 2024-03-21T09:18:38.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:38 smithi053 ceph-mon[36169]: pgmap v196: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:38.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1082963885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:18:39.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:39 smithi071 ceph-mon[36597]: pgmap v197: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:39.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:18:39.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:18:39.654 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:18:39.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:39 smithi037 ceph-mon[31830]: pgmap v197: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:39 smithi053 ceph-mon[36169]: pgmap v197: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:39.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.882+0000 7f8046c00640 1 -- 172.21.15.37:0/1070109510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040109850 msgr2=0x7f8040109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.882+0000 7f8046c00640 1 --2- 172.21.15.37:0/1070109510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040109850 0x7f8040109c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8030009ca0 tx=0x7f803002c100 comp rx=0 tx=0).stop 2024-03-21T09:18:39.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 -- 172.21.15.37:0/1070109510 shutdown_connections 2024-03-21T09:18:39.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 --2- 172.21.15.37:0/1070109510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040104200 0x7f8040110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 --2- 172.21.15.37:0/1070109510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040103840 0x7f8040103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 --2- 172.21.15.37:0/1070109510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040109850 0x7f8040109c30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 -- 172.21.15.37:0/1070109510 >> 172.21.15.37:0/1070109510 conn(0x7f80400ff2f0 msgr2=0x7f8040101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:39.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 -- 172.21.15.37:0/1070109510 shutdown_connections 2024-03-21T09:18:39.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.883+0000 7f8046c00640 1 -- 172.21.15.37:0/1070109510 wait complete. 2024-03-21T09:18:39.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.884+0000 7f8046c00640 1 Processor -- start 2024-03-21T09:18:39.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.884+0000 7f8046c00640 1 -- start start 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8046c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 0x7f804019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8046c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040104200 0x7f804019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8046c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040109850 0x7f804019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8046c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040113c60 con 0x7f8040109850 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8046c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040113ae0 con 0x7f8040104200 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8046c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040113de0 con 0x7f8040103840 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f803fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 0x7f804019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f803f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040104200 0x7f804019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f803fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 0x7f804019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57156/0 (socket says 172.21.15.37:57156) 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f803fe00640 1 -- 172.21.15.37:0/3336947350 learned_addr learned my addr 172.21.15.37:0/3336947350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:39.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.885+0000 7f8044c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040109850 0x7f804019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803fe00640 1 -- 172.21.15.37:0/3336947350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040104200 msgr2=0x7f804019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803fe00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040104200 0x7f804019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803fe00640 1 -- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040109850 msgr2=0x7f804019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803fe00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040109850 0x7f804019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803fe00640 1 -- 172.21.15.37:0/3336947350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804019f170 con 0x7f8040103840 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f8044c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040109850 0x7f804019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803f400640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040104200 0x7f804019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:39.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803fe00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 0x7f804019a180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8030005e30 tx=0x7f80300346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:39.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803cc00640 1 -- 172.21.15.37:0/3336947350 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030039050 con 0x7f8040103840 2024-03-21T09:18:39.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804019f400 con 0x7f8040103840 2024-03-21T09:18:39.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803cc00640 1 -- 172.21.15.37:0/3336947350 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8030045070 con 0x7f8040103840 2024-03-21T09:18:39.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.886+0000 7f803cc00640 1 -- 172.21.15.37:0/3336947350 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300046c0 con 0x7f8040103840 2024-03-21T09:18:39.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.887+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80401a6cf0 con 0x7f8040103840 2024-03-21T09:18:39.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 Processor -- start 2024-03-21T09:18:39.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 -- start start 2024-03-21T09:18:39.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8f39a0 0x5580ad8fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8fc580 0x5580ad8fe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ad857eb0 con 0x5580ad8f39a0 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ad800a60 con 0x5580ad8f3080 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.890+0000 7f80499fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ad853f50 con 0x5580ad8fc580 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45794/0 (socket says 172.21.15.37:45794) 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 -- 172.21.15.37:0/2065096837 learned_addr learned my addr 172.21.15.37:0/2065096837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f803f400640 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8fc580 0x5580ad8fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 -- 172.21.15.37:0/2065096837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8fc580 msgr2=0x5580ad8fe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8fc580 0x5580ad8fe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 -- 172.21.15.37:0/2065096837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8f39a0 msgr2=0x5580ad8fc040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8f39a0 0x5580ad8fc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 -- 172.21.15.37:0/2065096837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ad6347e0 con 0x5580ad8f3080 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f803cc00640 1 -- 172.21.15.37:0/3336947350 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8030003970 con 0x7f8040103840 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.891+0000 7f8044c00640 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8f3460 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8038009c20 tx=0x7f80380117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283e37aff87bd574 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f8035200640 1 -- 172.21.15.37:0/2065096837 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038012450 con 0x5580ad8f3080 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f8035200640 1 -- 172.21.15.37:0/2065096837 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8038012ad0 con 0x5580ad8f3080 2024-03-21T09:18:39.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f8035200640 1 -- 172.21.15.37:0/2065096837 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800fc10 con 0x5580ad8f3080 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 -- 172.21.15.37:0/2065096837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 msgr2=0x5580ad8f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8f3460 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8038009c20 tx=0x7f80380117b0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 -- 172.21.15.37:0/2065096837 shutdown_connections 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8fc580 0x5580ad8fe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8f39a0 0x5580ad8fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 --2- 172.21.15.37:0/2065096837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8f3460 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.892+0000 7f80499fa740 1 -- 172.21.15.37:0/2065096837 >> 172.21.15.37:0/2065096837 conn(0x5580ad8ed7b0 msgr2=0x5580ad8edb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.893+0000 7f80499fa740 1 -- 172.21.15.37:0/2065096837 shutdown_connections 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.893+0000 7f80499fa740 1 -- 172.21.15.37:0/2065096837 wait complete. 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.893+0000 7f80499fa740 1 -- start start 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.893+0000 7f80499fa740 1 Processor -- start 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.893+0000 7f803cc00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f80240a2fa0 0x7f80240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f803cc00640 1 -- 172.21.15.37:0/3336947350 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f80300f2200 con 0x7f8040103840 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f8044c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f80240a2fa0 0x7f80240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ad8ff370 con 0x5580ad8fc580 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ad813eb0 con 0x5580ad8f3080 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ad8ff4f0 con 0x5580ad8f39a0 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f80499fa740 10 client.0 _ll_register_callbacks cb 0x5580ad808150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f803fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f803f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f803fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45808/0 (socket says 172.21.15.37:45808) 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f803fe00640 1 -- 172.21.15.37:0/231344262 learned_addr learned my addr 172.21.15.37:0/231344262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.894+0000 7f8044c00640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803fe00640 1 -- 172.21.15.37:0/231344262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 msgr2=0x5580ad8f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803fe00640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8f9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803fe00640 1 -- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 msgr2=0x5580ad8f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803fe00640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803fe00640 1 -- 172.21.15.37:0/231344262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580ad694f70 con 0x5580ad8f3080 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803f400640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f803fe00640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fbed0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f80300efbb0 tx=0x7f80300bdcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ce313a3ece3f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f802d600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f8044c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f80240a2fa0 0x7f80240a5460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f803800f670 tx=0x7f8038009f10 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f802d600640 1 -- 172.21.15.37:0/231344262 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030004040 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f802d600640 1 -- 172.21.15.37:0/231344262 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8030003c60 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8038012ad0 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.895+0000 7f802d600640 1 -- 172.21.15.37:0/231344262 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80300bea60 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 10 client.27125 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580ad52c170 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f802d600640 1 -- 172.21.15.37:0/231344262 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8038012ad0 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f802d600640 1 -- 172.21.15.37:0/231344262 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f80300ba9d0 con 0x5580ad8f3080 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 10 client.27125 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 10 client.27125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 20 client.27125 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad90c540 0x5580ad90e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 4 client.27125 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:39.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580ad8653f0 con 0x5580ad90c540 2024-03-21T09:18:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.896+0000 7f803f400640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad90c540 0x5580ad90e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.897+0000 7f803f400640 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad90c540 0x5580ad90e930 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c3fe1675ae442a server_cookie=1918d44375cc4168 in_seq=0 out_seq=0 2024-03-21T09:18:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.897+0000 7f802d600640 10 client.27125 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f802d600640 1 -- 172.21.15.37:0/231344262 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5580ad8653f0 con 0x5580ad90c540 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f802d600640 10 client.27125 handle_command_reply: tid=0 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 client.27125 shutdown 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 msgr2=0x5580ad8fbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fbed0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f80300efbb0 tx=0x7f80300bdcc0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 shutdown_connections 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad90c540 0x5580ad90e930 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 --2- 172.21.15.37:0/231344262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fbed0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 >> 172.21.15.37:0/231344262 conn(0x5580ad8ed7b0 msgr2=0x5580ad8ef500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f802d600640 0 client.27125 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 shutdown_connections 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.898+0000 7f80499fa740 1 -- 172.21.15.37:0/231344262 wait complete. 2024-03-21T09:18:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.899+0000 7f80499fa740 20 client.27125 trim_cache size 0 max 16384 2024-03-21T09:18:39.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.927+0000 7f80499fa740 1 Processor -- start 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.927+0000 7f80499fa740 1 -- start start 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f80499fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f80499fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f80499fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad907c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f80499fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401b2fc0 con 0x5580ad8fc580 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f80499fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401b0ac0 con 0x5580ad8f3080 2024-03-21T09:18:39.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f80499fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401aeb60 con 0x5580ad8f39a0 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45824/0 (socket says 172.21.15.37:45824) 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 -- 172.21.15.37:0/1839685381 learned_addr learned my addr 172.21.15.37:0/1839685381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f8044c00640 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad907c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 -- 172.21.15.37:0/1839685381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 msgr2=0x5580ad8fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8fbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 -- 172.21.15.37:0/1839685381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 msgr2=0x5580ad907c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad907c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.928+0000 7f803fe00640 1 -- 172.21.15.37:0/1839685381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ad8fece0 con 0x5580ad8f3080 2024-03-21T09:18:39.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.929+0000 7f803fe00640 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fb990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f80300ef770 tx=0x7f80300bc870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb78d2f5a50dc96 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.929+0000 7f802fe00640 1 -- 172.21.15.37:0/1839685381 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030004040 con 0x5580ad8f3080 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.929+0000 7f802fe00640 1 -- 172.21.15.37:0/1839685381 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f803003a040 con 0x5580ad8f3080 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.929+0000 7f802fe00640 1 -- 172.21.15.37:0/1839685381 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030003c60 con 0x5580ad8f3080 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 -- 172.21.15.37:0/1839685381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 msgr2=0x5580ad8fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fb990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f80300ef770 tx=0x7f80300bc870 comp rx=0 tx=0).stop 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 -- 172.21.15.37:0/1839685381 shutdown_connections 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad907c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f39a0 0x5580ad8fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 --2- 172.21.15.37:0/1839685381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f3080 0x5580ad8fb990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 -- 172.21.15.37:0/1839685381 >> 172.21.15.37:0/1839685381 conn(0x5580ad8ed7b0 msgr2=0x5580ad8ef470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 -- 172.21.15.37:0/1839685381 shutdown_connections 2024-03-21T09:18:39.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 -- 172.21.15.37:0/1839685381 wait complete. 2024-03-21T09:18:39.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.930+0000 7f80499fa740 1 -- start start 2024-03-21T09:18:39.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 Processor -- start 2024-03-21T09:18:39.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 0x5580ad9610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 0x5580ad961630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401af160 con 0x5580ad8fc580 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401b0460 con 0x5580ad8f39a0 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401af7a0 con 0x5580ad8f3080 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.931+0000 7f80499fa740 10 client.0 _ll_register_callbacks cb 0x7f80300be130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 0x5580ad961630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 0x5580ad9610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 0x5580ad961630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45826/0 (socket says 172.21.15.37:45826) 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 0x5580ad9610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57182/0 (socket says 172.21.15.37:57182) 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 -- 172.21.15.37:0/2032207935 learned_addr learned my addr 172.21.15.37:0/2032207935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f8044c00640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 -- 172.21.15.37:0/2032207935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 msgr2=0x5580ad9610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 0x5580ad9610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 -- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 msgr2=0x5580ad8f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 -- 172.21.15.37:0/2032207935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580ad8f6820 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803fe00640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 0x5580ad9610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f8044c00640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f803f400640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 0x5580ad961630 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8028005940 tx=0x7f8028011df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5997ba768a47ec40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f802d600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.932+0000 7f802d600640 1 -- 172.21.15.37:0/2032207935 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280202b0 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f803003a040 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f802d600640 1 -- 172.21.15.37:0/2032207935 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8028004020 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f802d600640 1 -- 172.21.15.37:0/2032207935 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8028030050 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 10 client.27131 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580ad968910 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f802d600640 1 -- 172.21.15.37:0/2032207935 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f803003a040 con 0x5580ad8f39a0 2024-03-21T09:18:39.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f802d600640 1 -- 172.21.15.37:0/2032207935 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f802801d3f0 con 0x5580ad8f39a0 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 10 client.27131 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 20 client.27131 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad969020 0x5580ad96b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 4 client.27131 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.933+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5580ad85f740 con 0x5580ad969020 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.934+0000 7f803fe00640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad969020 0x5580ad96b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.934+0000 7f803fe00640 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad969020 0x5580ad96b440 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2366c955d4bd9ad3 server_cookie=7cbb56c4eb8fac22 in_seq=0 out_seq=0 2024-03-21T09:18:39.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.934+0000 7f802d600640 10 client.27131 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:39.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.935+0000 7f802d600640 1 -- 172.21.15.37:0/2032207935 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580ad85f740 con 0x5580ad969020 2024-03-21T09:18:39.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.935+0000 7f802d600640 10 client.27131 handle_command_reply: tid=0 2024-03-21T09:18:39.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.935+0000 7f80499fa740 1 client.27131 shutdown 2024-03-21T09:18:39.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 msgr2=0x5580ad961630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 0x5580ad961630 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8028005940 tx=0x7f8028011df0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 shutdown_connections 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580ad8fc580 0x5580ad8f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580ad969020 0x5580ad96b440 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580ad8f39a0 0x5580ad961630 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f802d600640 0 client.27131 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 --2- 172.21.15.37:0/2032207935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5580ad8f3080 0x5580ad9610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 >> 172.21.15.37:0/2032207935 conn(0x5580ad8ed7b0 msgr2=0x5580ad8f3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.936+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 shutdown_connections 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.937+0000 7f80499fa740 1 -- 172.21.15.37:0/2032207935 wait complete. 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.937+0000 7f80499fa740 20 client.27131 trim_cache size 0 max 16384 2024-03-21T09:18:39.934 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:18:39.935 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:18:39.935 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "51cbb626-c858-4d9a-89f5-f9bd57fc9a9d", 2024-03-21T09:18:39.935 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:18:39.935 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:18:39.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f80240a2fa0 msgr2=0x7f80240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f80240a2fa0 0x7f80240a5460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f803800f670 tx=0x7f8038009f10 comp rx=0 tx=0).stop 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 msgr2=0x7f804019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 0x7f804019a180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8030005e30 tx=0x7f80300346f0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 shutdown_connections 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8040109850 0x7f804019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f80240a2fa0 0x7f80240a5460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8040104200 0x7f804019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 --2- 172.21.15.37:0/3336947350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8040103840 0x7f804019a180 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 >> 172.21.15.37:0/3336947350 conn(0x7f80400ff2f0 msgr2=0x7f80400ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 shutdown_connections 2024-03-21T09:18:39.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:39.938+0000 7f8046c00640 1 -- 172.21.15.37:0/3336947350 wait complete. 2024-03-21T09:18:39.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:18:39.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.222+0000 7fae3cc00640 1 -- 172.21.15.37:0/2398884233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38104020 msgr2=0x7fae38110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.222+0000 7fae3cc00640 1 --2- 172.21.15.37:0/2398884233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38104020 0x7fae38110530 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae30009cd0 tx=0x7fae3002c0f0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 -- 172.21.15.37:0/2398884233 shutdown_connections 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 --2- 172.21.15.37:0/2398884233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38104020 0x7fae38110530 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 --2- 172.21.15.37:0/2398884233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103660 0x7fae38103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 --2- 172.21.15.37:0/2398884233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38109670 0x7fae38109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 -- 172.21.15.37:0/2398884233 >> 172.21.15.37:0/2398884233 conn(0x7fae380ff150 msgr2=0x7fae38101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 -- 172.21.15.37:0/2398884233 shutdown_connections 2024-03-21T09:18:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.223+0000 7fae3cc00640 1 -- 172.21.15.37:0/2398884233 wait complete. 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 Processor -- start 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 -- start start 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38103660 0x7fae38199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38104020 0x7fae3819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38109670 0x7fae3819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38113a80 con 0x7fae38104020 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38113900 con 0x7fae38109670 2024-03-21T09:18:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.224+0000 7fae3cc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38113c00 con 0x7fae38103660 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38103660 0x7fae38199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38109670 0x7fae3819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38103660 0x7fae38199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57200/0 (socket says 172.21.15.37:57200) 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37400640 1 -- 172.21.15.37:0/3163010666 learned_addr learned my addr 172.21.15.37:0/3163010666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae36a00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38104020 0x7fae3819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 -- 172.21.15.37:0/3163010666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38103660 msgr2=0x7fae38199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38103660 0x7fae38199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 -- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38104020 msgr2=0x7fae3819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38104020 0x7fae3819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 -- 172.21.15.37:0/3163010666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3819ef30 con 0x7fae38109670 2024-03-21T09:18:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae37e00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38109670 0x7fae3819e800 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fae30037040 tx=0x7fae30032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.225+0000 7fae2fe00640 1 -- 172.21.15.37:0/3163010666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30041020 con 0x7fae38109670 2024-03-21T09:18:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.226+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3819f1c0 con 0x7fae38109670 2024-03-21T09:18:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.226+0000 7fae2fe00640 1 -- 172.21.15.37:0/3163010666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae30034cd0 con 0x7fae38109670 2024-03-21T09:18:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.226+0000 7fae2fe00640 1 -- 172.21.15.37:0/3163010666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30039040 con 0x7fae38109670 2024-03-21T09:18:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.226+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae381a6ab0 con 0x7fae38109670 2024-03-21T09:18:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.229+0000 7fae2fe00640 1 -- 172.21.15.37:0/3163010666 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fae30044020 con 0x7fae38109670 2024-03-21T09:18:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.229+0000 7fae41012740 1 Processor -- start 2024-03-21T09:18:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.229+0000 7fae41012740 1 -- start start 2024-03-21T09:18:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae41012740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb909bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae41012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909c0f0 0x562fb909c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae41012740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb90a5460 0x562fb90a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae41012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9000250 con 0x562fb909c0f0 2024-03-21T09:18:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae41012740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb8fa8a30 con 0x562fb909cdd0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae41012740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb8ffc2f0 con 0x562fb90a5460 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae36a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb909bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae36a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb909bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45858/0 (socket says 172.21.15.37:45858) 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae36a00640 1 -- 172.21.15.37:0/1542003354 learned_addr learned my addr 172.21.15.37:0/1542003354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae37400640 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909c0f0 0x562fb909c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae36a00640 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb90a5460 0x562fb90a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.230+0000 7fae2fe00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae1c0a3110 0x7fae1c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae36a00640 1 -- 172.21.15.37:0/1542003354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb90a5460 msgr2=0x562fb90a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae36a00640 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb90a5460 0x562fb90a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae36a00640 1 -- 172.21.15.37:0/1542003354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909c0f0 msgr2=0x562fb909c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae36a00640 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909c0f0 0x562fb909c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae36a00640 1 -- 172.21.15.37:0/1542003354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fb8f93690 con 0x562fb909cdd0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae2fe00640 1 -- 172.21.15.37:0/3163010666 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fae300f1720 con 0x7fae38109670 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae36a00640 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb909bb20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fae28012790 tx=0x7fae28009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38f50f705dbdc5be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae37e00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae1c0a3110 0x7fae1c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae27e00640 1 -- 172.21.15.37:0/1542003354 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28012d10 con 0x562fb909cdd0 2024-03-21T09:18:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae27e00640 1 -- 172.21.15.37:0/1542003354 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae280107d0 con 0x562fb909cdd0 2024-03-21T09:18:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.231+0000 7fae27e00640 1 -- 172.21.15.37:0/1542003354 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28003850 con 0x562fb909cdd0 2024-03-21T09:18:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 -- 172.21.15.37:0/1542003354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 msgr2=0x562fb909bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb909bb20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fae28012790 tx=0x7fae28009700 comp rx=0 tx=0).stop 2024-03-21T09:18:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae37e00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae1c0a3110 0x7fae1c0a55d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae300bd7b0 tx=0x7fae300095c0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 -- 172.21.15.37:0/1542003354 shutdown_connections 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb90a5460 0x562fb90a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909c0f0 0x562fb909c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 --2- 172.21.15.37:0/1542003354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb909bb20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 -- 172.21.15.37:0/1542003354 >> 172.21.15.37:0/1542003354 conn(0x562fb9063200 msgr2=0x562fb9096b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 -- 172.21.15.37:0/1542003354 shutdown_connections 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.232+0000 7fae41012740 1 -- 172.21.15.37:0/1542003354 wait complete. 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 -- start start 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 Processor -- start 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb90a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb90a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb90a82d0 con 0x562fb90a5460 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb8fbbe80 con 0x562fb909cdd0 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb90a8450 con 0x562fb909c0f0 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.233+0000 7fae41012740 10 client.0 _ll_register_callbacks cb 0x562fb8fce130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb90a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb90a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:45868/0 (socket says 172.21.15.37:45868) 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 -- 172.21.15.37:0/1842918747 learned_addr learned my addr 172.21.15.37:0/1842918747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae36a00640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb90a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae36a00640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 -- 172.21.15.37:0/1842918747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 msgr2=0x562fb90a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb90a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 -- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 msgr2=0x562fb90a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 -- 172.21.15.37:0/1842918747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fb8dde7d0 con 0x562fb909cdd0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae36a00640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb90a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae37400640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb90a1f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae2000ba90 tx=0x7fae2000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f2d5355b4cbc9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae24c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.234+0000 7fae24c00640 1 -- 172.21.15.37:0/1842918747 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20011440 con 0x562fb909cdd0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae24c00640 1 -- 172.21.15.37:0/1842918747 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fae20003ee0 con 0x562fb909cdd0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae280107d0 con 0x562fb909cdd0 2024-03-21T09:18:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae24c00640 1 -- 172.21.15.37:0/1842918747 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae200115e0 con 0x562fb909cdd0 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 10 client.27143 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562fb8d143f0 con 0x562fb909cdd0 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae24c00640 1 -- 172.21.15.37:0/1842918747 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae280107d0 con 0x562fb909cdd0 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae24c00640 1 -- 172.21.15.37:0/1842918747 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae2001bb50 con 0x562fb909cdd0 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 10 client.27143 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 10 client.27143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 20 client.27143 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb90b5460 0x562fb90b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 4 client.27143 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.235+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fb900d790 con 0x562fb90b5460 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.236+0000 7fae36a00640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb90b5460 0x562fb90b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.236+0000 7fae36a00640 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb90b5460 0x562fb90b7880 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f346600955806aec server_cookie=cff696704c4673d4 in_seq=0 out_seq=0 2024-03-21T09:18:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.236+0000 7fae24c00640 10 client.27143 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae24c00640 1 -- 172.21.15.37:0/1842918747 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562fb900d790 con 0x562fb90b5460 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae24c00640 10 client.27143 handle_command_reply: tid=0 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 client.27143 shutdown 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 msgr2=0x562fb90a1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb90a1f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae2000ba90 tx=0x7fae2000bf50 comp rx=0 tx=0).stop 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 shutdown_connections 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb90b5460 0x562fb90b7880 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909cdd0 0x562fb90a1f20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 --2- 172.21.15.37:0/1842918747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb90a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae24c00640 0 client.27143 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.237+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 >> 172.21.15.37:0/1842918747 conn(0x562fb9063200 msgr2=0x562fb9098890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.238+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 shutdown_connections 2024-03-21T09:18:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.238+0000 7fae41012740 1 -- 172.21.15.37:0/1842918747 wait complete. 2024-03-21T09:18:40.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.238+0000 7fae41012740 20 client.27143 trim_cache size 0 max 16384 2024-03-21T09:18:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 Processor -- start 2024-03-21T09:18:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 -- start start 2024-03-21T09:18:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909c0f0 0x562fb90a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 0x562fb90a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381b2ed0 con 0x562fb90a5460 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381b09d0 con 0x562fb909c0f0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae41012740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381aea70 con 0x562fb909cdd0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae37400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 0x562fb90a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae36a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909c0f0 0x562fb90a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae37400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 0x562fb90a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57224/0 (socket says 172.21.15.37:57224) 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae37400640 1 -- 172.21.15.37:0/2762982631 learned_addr learned my addr 172.21.15.37:0/2762982631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.256+0000 7fae36a00640 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae37400640 1 -- 172.21.15.37:0/2762982631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909c0f0 msgr2=0x562fb90a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae37400640 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909c0f0 0x562fb90a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae37400640 1 -- 172.21.15.37:0/2762982631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 msgr2=0x562fb90a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae37400640 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae37400640 1 -- 172.21.15.37:0/2762982631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fb8cd40f0 con 0x562fb909cdd0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae36a00640 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909c0f0 0x562fb90a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae37400640 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 0x562fb90a4650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae20001eb0 tx=0x7fae20003b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdcea6739ced5003 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae27400640 1 -- 172.21.15.37:0/2762982631 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2002c070 con 0x562fb909cdd0 2024-03-21T09:18:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae27400640 1 -- 172.21.15.37:0/2762982631 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae20010e80 con 0x562fb909cdd0 2024-03-21T09:18:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.257+0000 7fae27400640 1 -- 172.21.15.37:0/2762982631 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20011780 con 0x562fb909cdd0 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 -- 172.21.15.37:0/2762982631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 msgr2=0x562fb90a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 0x562fb90a4650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae20001eb0 tx=0x7fae20003b30 comp rx=0 tx=0).stop 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 -- 172.21.15.37:0/2762982631 shutdown_connections 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb90a5460 0x562fb90a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909cdd0 0x562fb90a4650 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 --2- 172.21.15.37:0/2762982631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb909c0f0 0x562fb90a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 -- 172.21.15.37:0/2762982631 >> 172.21.15.37:0/2762982631 conn(0x562fb9063200 msgr2=0x562fb90a0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 -- 172.21.15.37:0/2762982631 shutdown_connections 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 -- 172.21.15.37:0/2762982631 wait complete. 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.258+0000 7fae41012740 1 -- start start 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 Processor -- start 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb909f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 0x562fb90b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb90a5460 0x562fb90b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381af070 con 0x562fb909cdd0 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381b0370 con 0x562fb90a5460 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381af6b0 con 0x562fb909c0f0 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae41012740 10 client.0 _ll_register_callbacks cb 0x7fae20004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae37400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 0x562fb90b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae36a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb909f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae37400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 0x562fb90b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49866/0 (socket says 172.21.15.37:49866) 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.259+0000 7fae37400640 1 -- 172.21.15.37:0/1298851715 learned_addr learned my addr 172.21.15.37:0/1298851715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb90a5460 0x562fb90b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 -- 172.21.15.37:0/1298851715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb90a5460 msgr2=0x562fb90b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb90a5460 0x562fb90b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 -- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 msgr2=0x562fb90b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 0x562fb90b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 -- 172.21.15.37:0/1298851715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fb90b1270 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae37400640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 0x562fb90b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae36a00640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb909f890 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fae28011010 tx=0x7fae28009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c56d4f92030887 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae24c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae24c00640 1 -- 172.21.15.37:0/1298851715 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28033020 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae24c00640 1 -- 172.21.15.37:0/1298851715 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fae280050d0 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.260+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fb9007ae0 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae24c00640 1 -- 172.21.15.37:0/1298851715 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2803b050 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 10 client.27175 fetch_fsmap learned FSMap version 30 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562fb9111b70 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae24c00640 1 -- 172.21.15.37:0/1298851715 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562fb9007ae0 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae24c00640 1 -- 172.21.15.37:0/1298851715 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae280035f0 con 0x562fb909c0f0 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 10 client.27175 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:18:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 10 client.27175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 20 client.27175 populate_metadata read hostname 'smithi037' 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb9111ea0 0x562fb9114290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 4 client.27175 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562fb8f853a0 con 0x562fb9111ea0 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.261+0000 7fae37400640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb9111ea0 0x562fb9114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.262+0000 7fae37400640 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb9111ea0 0x562fb9114290 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba151d62982d9e9 server_cookie=b03e57cc58f5c896 in_seq=0 out_seq=0 2024-03-21T09:18:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.262+0000 7fae24c00640 10 client.27175 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.262+0000 7fae24c00640 1 -- 172.21.15.37:0/1298851715 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562fb8f853a0 con 0x562fb9111ea0 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.262+0000 7fae24c00640 10 client.27175 handle_command_reply: tid=0 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.262+0000 7fae41012740 1 client.27175 shutdown 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 msgr2=0x562fb909f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb909f890 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fae28011010 tx=0x7fae28009700 comp rx=0 tx=0).stop 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 shutdown_connections 2024-03-21T09:18:40.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562fb9111ea0 0x562fb9114290 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fb90a5460 0x562fb90b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fb909cdd0 0x562fb90b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae24c00640 0 client.27175 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 --2- 172.21.15.37:0/1298851715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562fb909c0f0 0x562fb909f890 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 >> 172.21.15.37:0/1298851715 conn(0x562fb9063200 msgr2=0x562fb90a10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 shutdown_connections 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 1 -- 172.21.15.37:0/1298851715 wait complete. 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.263+0000 7fae41012740 20 client.27175 trim_cache size 0 max 16384 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:18:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "51cbb626-c858-4d9a-89f5-f9bd57fc9a9d": { 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "51cbb626-c858-4d9a-89f5-f9bd57fc9a9d", 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:18:40.263 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae1c0a3110 msgr2=0x7fae1c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae1c0a3110 0x7fae1c0a55d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae300bd7b0 tx=0x7fae300095c0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38109670 msgr2=0x7fae3819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38109670 0x7fae3819e800 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fae30037040 tx=0x7fae30032ea0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 shutdown_connections 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fae1c0a3110 0x7fae1c0a55d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38109670 0x7fae3819e800 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38104020 0x7fae3819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 --2- 172.21.15.37:0/3163010666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae38103660 0x7fae38199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 >> 172.21.15.37:0/3163010666 conn(0x7fae380ff150 msgr2=0x7fae380ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:18:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 shutdown_connections 2024-03-21T09:18:40.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:18:40.264+0000 7fae3cc00640 1 -- 172.21.15.37:0/3163010666 wait complete. 2024-03-21T09:18:40.286 INFO:tasks.cephfs.filesystem:scrub status for tag:51cbb626-c858-4d9a-89f5-f9bd57fc9a9d - {'path': '/', 'tag': '51cbb626-c858-4d9a-89f5-f9bd57fc9a9d', 'options': 'recursive,force'} 2024-03-21T09:18:40.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:40 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:18:40.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:40 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:40.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:40 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:18:40.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:40 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:40.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:40 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:18:40.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:40 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:40.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:40 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:18:40.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:40 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:41.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:40 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:18:41.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:40 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:41.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:40 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:18:41.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:40 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:18:41.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:41 smithi037 ceph-mon[31830]: pgmap v198: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:41.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:41 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:18:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:41 smithi053 ceph-mon[36169]: pgmap v198: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:41.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:41 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:18:42.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:41 smithi071 ceph-mon[36597]: pgmap v198: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:42.104 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:41 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:18:44.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:44 smithi071 ceph-mon[36597]: pgmap v199: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:44.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:44 smithi037 ceph-mon[31830]: pgmap v199: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:44.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:44 smithi053 ceph-mon[36169]: pgmap v199: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:46.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:46 smithi071 ceph-mon[36597]: pgmap v200: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:46.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:46 smithi037 ceph-mon[31830]: pgmap v200: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:46.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:46 smithi053 ceph-mon[36169]: pgmap v200: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:48.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:48 smithi071 ceph-mon[36597]: pgmap v201: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:48.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:48 smithi037 ceph-mon[31830]: pgmap v201: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:48.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:48 smithi053 ceph-mon[36169]: pgmap v201: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:50.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:50 smithi071 ceph-mon[36597]: pgmap v202: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:50.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:50 smithi037 ceph-mon[31830]: pgmap v202: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:50.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:50 smithi053 ceph-mon[36169]: pgmap v202: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:52.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:52 smithi071 ceph-mon[36597]: pgmap v203: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:52.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:52 smithi037 ceph-mon[31830]: pgmap v203: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:52.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:52 smithi053 ceph-mon[36169]: pgmap v203: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:18:54.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:54 smithi071 ceph-mon[36597]: pgmap v204: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:54.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:54 smithi037 ceph-mon[31830]: pgmap v204: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:54.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:54 smithi053 ceph-mon[36169]: pgmap v204: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:18:56.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:56 smithi071 ceph-mon[36597]: pgmap v205: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:56.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:56 smithi037 ceph-mon[31830]: pgmap v205: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:56.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:56 smithi053 ceph-mon[36169]: pgmap v205: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:18:58.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:18:58 smithi071 ceph-mon[36597]: pgmap v206: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:58.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:18:58 smithi037 ceph-mon[31830]: pgmap v206: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:18:58.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:18:58 smithi053 ceph-mon[36169]: pgmap v206: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:00.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:00 smithi071 ceph-mon[36597]: pgmap v207: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:00.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:00 smithi037 ceph-mon[31830]: pgmap v207: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:00.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:00 smithi053 ceph-mon[36169]: pgmap v207: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:02.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:02 smithi071 ceph-mon[36597]: pgmap v208: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:02.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:02 smithi037 ceph-mon[31830]: pgmap v208: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:02.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:02 smithi053 ceph-mon[36169]: pgmap v208: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:03.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:03 smithi071 ceph-mon[36597]: pgmap v209: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:19:03.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:03 smithi037 ceph-mon[31830]: pgmap v209: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:19:03.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:03 smithi053 ceph-mon[36169]: pgmap v209: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:19:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:06 smithi071 ceph-mon[36597]: pgmap v210: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:19:06.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:06 smithi037 ceph-mon[31830]: pgmap v210: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:19:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:06 smithi053 ceph-mon[36169]: pgmap v210: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:19:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:08 smithi071 ceph-mon[36597]: pgmap v211: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T09:19:08.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:08 smithi071 ceph-mon[36597]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T09:19:08.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:08 smithi037 ceph-mon[31830]: pgmap v211: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T09:19:08.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:08 smithi037 ceph-mon[31830]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T09:19:08.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:08 smithi053 ceph-mon[36169]: pgmap v211: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-21T09:19:08.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:08 smithi053 ceph-mon[36169]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T09:19:10.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:19:10.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:19:10.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 -- 172.21.15.37:0/3806942464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c102630 msgr2=0x7eff2c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 --2- 172.21.15.37:0/3806942464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c102630 0x7eff2c102a10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009900 tx=0x7eff1c02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 -- 172.21.15.37:0/3806942464 shutdown_connections 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 --2- 172.21.15.37:0/3806942464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c10ba50 0x7eff2c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 --2- 172.21.15.37:0/3806942464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 0x7eff2c10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 --2- 172.21.15.37:0/3806942464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c102630 0x7eff2c102a10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 -- 172.21.15.37:0/3806942464 >> 172.21.15.37:0/3806942464 conn(0x7eff2c077130 msgr2=0x7eff2c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 -- 172.21.15.37:0/3806942464 shutdown_connections 2024-03-21T09:19:10.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.536+0000 7eff30a00640 1 -- 172.21.15.37:0/3806942464 wait complete. 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 Processor -- start 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 -- start start 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 0x7eff2c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 0x7eff2c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c10ba50 0x7eff2c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c111820 con 0x7eff2c102f50 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1116a0 con 0x7eff2c10ba50 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.538+0000 7eff30a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1119a0 con 0x7eff2c102630 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 0x7eff2c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c10ba50 0x7eff2c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 0x7eff2c197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58478/0 (socket says 172.21.15.37:58478) 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c10ba50 0x7eff2c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47294/0 (socket says 172.21.15.37:47294) 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 -- 172.21.15.37:0/3308945715 learned_addr learned my addr 172.21.15.37:0/3308945715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff22a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 0x7eff2c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 -- 172.21.15.37:0/3308945715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c10ba50 msgr2=0x7eff2c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c10ba50 0x7eff2c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 -- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 msgr2=0x7eff2c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 0x7eff2c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 -- 172.21.15.37:0/3308945715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c19cd50 con 0x7eff2c102630 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff22a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 0x7eff2c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.539+0000 7eff2b400640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 0x7eff2c197d60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff1c036040 tx=0x7eff1c02cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.540+0000 7eff28c00640 1 -- 172.21.15.37:0/3308945715 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c043020 con 0x7eff2c102630 2024-03-21T09:19:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.540+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c19cfe0 con 0x7eff2c102630 2024-03-21T09:19:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.540+0000 7eff28c00640 1 -- 172.21.15.37:0/3308945715 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff1c0054e0 con 0x7eff2c102630 2024-03-21T09:19:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.540+0000 7eff28c00640 1 -- 172.21.15.37:0/3308945715 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c003c20 con 0x7eff2c102630 2024-03-21T09:19:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.540+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1a48d0 con 0x7eff2c102630 2024-03-21T09:19:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.543+0000 7eff28c00640 1 -- 172.21.15.37:0/3308945715 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7eff1c046020 con 0x7eff2c102630 2024-03-21T09:19:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.544+0000 7eff28c00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7eff100a3090 0x7eff100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.544+0000 7eff28c00640 1 -- 172.21.15.37:0/3308945715 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7eff1c0f1d10 con 0x7eff2c102630 2024-03-21T09:19:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.544+0000 7eff22a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7eff100a3090 0x7eff100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.545+0000 7eff34c7d740 1 Processor -- start 2024-03-21T09:19:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.545+0000 7eff22a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7eff100a3090 0x7eff100a5550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0fee30 tx=0x7eff14005e50 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.545+0000 7eff34c7d740 1 -- start start 2024-03-21T09:19:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea89f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea8a500 0x557d9ea8a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 0x557d9ea95c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff34c7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9e9ea250 con 0x557d9ea8a500 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff34c7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9e992a30 con 0x557d9ea8b1e0 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff34c7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9e9e62f0 con 0x557d9ea93870 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea89f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 0x557d9ea95c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 0x557d9ea95c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58486/0 (socket says 172.21.15.37:58486) 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea89f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47302/0 (socket says 172.21.15.37:47302) 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff2be00640 1 -- 172.21.15.37:0/2779161769 learned_addr learned my addr 172.21.15.37:0/2779161769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.546+0000 7eff22a00640 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea8a500 0x557d9ea8a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2be00640 1 -- 172.21.15.37:0/2779161769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 msgr2=0x557d9ea89f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2be00640 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea89f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2be00640 1 -- 172.21.15.37:0/2779161769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea8a500 msgr2=0x557d9ea8a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2be00640 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea8a500 0x557d9ea8a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2be00640 1 -- 172.21.15.37:0/2779161769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d9e97d690 con 0x557d9ea93870 2024-03-21T09:19:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2b400640 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea89f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff2be00640 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 0x557d9ea95c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff24009f50 tx=0x7eff240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2fd8444d805049 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff1be00640 1 -- 172.21.15.37:0/2779161769 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24004d20 con 0x557d9ea93870 2024-03-21T09:19:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff1be00640 1 -- 172.21.15.37:0/2779161769 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff24004050 con 0x557d9ea93870 2024-03-21T09:19:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.547+0000 7eff1be00640 1 -- 172.21.15.37:0/2779161769 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24005240 con 0x557d9ea93870 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 -- 172.21.15.37:0/2779161769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 msgr2=0x557d9ea95c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 0x557d9ea95c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff24009f50 tx=0x7eff240118d0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 -- 172.21.15.37:0/2779161769 shutdown_connections 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea93870 0x557d9ea95c60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea8a500 0x557d9ea8a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2779161769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea89f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 -- 172.21.15.37:0/2779161769 >> 172.21.15.37:0/2779161769 conn(0x557d9ea4d200 msgr2=0x557d9ea84f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 -- 172.21.15.37:0/2779161769 shutdown_connections 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.548+0000 7eff34c7d740 1 -- 172.21.15.37:0/2779161769 wait complete. 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 -- start start 2024-03-21T09:19:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 Processor -- start 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea92f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ea966e0 con 0x557d9ea93870 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9e9a5e80 con 0x557d9ea8a500 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.549+0000 7eff34c7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ea96860 con 0x557d9ea8b1e0 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff34c7d740 10 client.0 _ll_register_callbacks cb 0x557d9e9b8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea92f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff22a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea92f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47316/0 (socket says 172.21.15.37:47316) 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 -- 172.21.15.37:0/1988811932 learned_addr learned my addr 172.21.15.37:0/1988811932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff22a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea902a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58500/0 (socket says 172.21.15.37:58500) 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2be00640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 -- 172.21.15.37:0/1988811932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 msgr2=0x557d9ea902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 -- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 msgr2=0x557d9ea907e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea907e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 -- 172.21.15.37:0/1988811932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d9e7c87d0 con 0x557d9ea8a500 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff22a00640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff2b400640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea92f10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000c00 tx=0x7eff1c0bd6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21af112fd266ec14 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff18c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.550+0000 7eff18c00640 1 -- 172.21.15.37:0/1988811932 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0be460 con 0x557d9ea8a500 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff18c00640 1 -- 172.21.15.37:0/1988811932 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7eff1c0ba3d0 con 0x557d9ea8a500 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff24004050 con 0x557d9ea8a500 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff18c00640 1 -- 172.21.15.37:0/1988811932 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1c0bdd60 con 0x557d9ea8a500 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 10 client.27161 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557d9e6fe3f0 con 0x557d9ea8a500 2024-03-21T09:19:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff18c00640 1 -- 172.21.15.37:0/1988811932 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff24004050 con 0x557d9ea8a500 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff18c00640 1 -- 172.21.15.37:0/1988811932 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7eff1c09cb80 con 0x557d9ea8a500 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 10 client.27161 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 10 client.27161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 20 client.27161 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaa36b0 0x557d9eaa5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 4 client.27161 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.551+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d9e9f7790 con 0x557d9eaa36b0 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.552+0000 7eff22a00640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaa36b0 0x557d9eaa5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.552+0000 7eff22a00640 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaa36b0 0x557d9eaa5b50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca766a0c1e822ba2 server_cookie=430faeb7e8f1b2ed in_seq=0 out_seq=0 2024-03-21T09:19:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.552+0000 7eff18c00640 10 client.27161 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff18c00640 1 -- 172.21.15.37:0/1988811932 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557d9e9f7790 con 0x557d9eaa36b0 2024-03-21T09:19:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff18c00640 10 client.27161 handle_command_reply: tid=0 2024-03-21T09:19:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff34c7d740 1 client.27161 shutdown 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 msgr2=0x557d9ea92f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff34c7d740 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea92f10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000c00 tx=0x7eff1c0bd6c0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 shutdown_connections 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff34c7d740 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.553+0000 7eff34c7d740 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff34c7d740 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaa36b0 0x557d9eaa5b50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff34c7d740 1 --2- 172.21.15.37:0/1988811932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea92f10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff18c00640 0 client.27161 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 >> 172.21.15.37:0/1988811932 conn(0x557d9ea4d200 msgr2=0x557d9ea86ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 shutdown_connections 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff34c7d740 1 -- 172.21.15.37:0/1988811932 wait complete. 2024-03-21T09:19:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.554+0000 7eff34c7d740 20 client.27161 trim_cache size 0 max 16384 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.571+0000 7eff34c7d740 1 Processor -- start 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.571+0000 7eff34c7d740 1 -- start start 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea92f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea94c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff34c7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1aed00 con 0x557d9ea93870 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff34c7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ac800 con 0x557d9ea8a500 2024-03-21T09:19:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff34c7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1aa8c0 con 0x557d9ea8b1e0 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea929d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47318/0 (socket says 172.21.15.37:47318) 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 -- 172.21.15.37:0/2331548785 learned_addr learned my addr 172.21.15.37:0/2331548785 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff22a00640 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea92f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2be00640 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea94c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 -- 172.21.15.37:0/2331548785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 msgr2=0x557d9ea92f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea92f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 -- 172.21.15.37:0/2331548785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 msgr2=0x557d9ea94c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea94c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2b400640 1 -- 172.21.15.37:0/2331548785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d9e6be0f0 con 0x557d9ea8a500 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.572+0000 7eff2be00640 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea94c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.573+0000 7eff2b400640 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea929d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0c6240 tx=0x7eff1c0bc270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158e2800f9906b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.573+0000 7eff1b400640 1 -- 172.21.15.37:0/2331548785 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0da030 con 0x557d9ea8a500 2024-03-21T09:19:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.573+0000 7eff1b400640 1 -- 172.21.15.37:0/2331548785 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff1c004c80 con 0x557d9ea8a500 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.573+0000 7eff1b400640 1 -- 172.21.15.37:0/2331548785 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0ba3d0 con 0x557d9ea8a500 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.573+0000 7eff34c7d740 1 -- 172.21.15.37:0/2331548785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 msgr2=0x557d9ea929d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.573+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea929d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0c6240 tx=0x7eff1c0bc270 comp rx=0 tx=0).stop 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 -- 172.21.15.37:0/2331548785 shutdown_connections 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea94c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8b1e0 0x557d9ea92f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 --2- 172.21.15.37:0/2331548785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8a500 0x557d9ea929d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 -- 172.21.15.37:0/2331548785 >> 172.21.15.37:0/2331548785 conn(0x557d9ea4d200 msgr2=0x557d9ea8ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 -- 172.21.15.37:0/2331548785 shutdown_connections 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 -- 172.21.15.37:0/2331548785 wait complete. 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 -- start start 2024-03-21T09:19:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.574+0000 7eff34c7d740 1 Processor -- start 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 0x557d9ea8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea8d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea8d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1aaec0 con 0x557d9ea93870 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ac1a0 con 0x557d9ea8b1e0 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ab4e0 con 0x557d9ea8a500 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff34c7d740 10 client.0 _ll_register_callbacks cb 0x7eff1c0bdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 0x557d9ea8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 0x557d9ea8cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58520/0 (socket says 172.21.15.37:58520) 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 -- 172.21.15.37:0/697693277 learned_addr learned my addr 172.21.15.37:0/697693277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff22a00640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea8d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2be00640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea8d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 -- 172.21.15.37:0/697693277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 msgr2=0x557d9ea8d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea8d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 -- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 msgr2=0x557d9ea8d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea8d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2b400640 1 -- 172.21.15.37:0/697693277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d9eafe280 con 0x557d9ea8a500 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.575+0000 7eff2be00640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff2b400640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 0x557d9ea8cc70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0bda60 tx=0x7eff1c0ef3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e1c867a8d43dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff18c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff18c00640 1 -- 172.21.15.37:0/697693277 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0da030 con 0x557d9ea8a500 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff18c00640 1 -- 172.21.15.37:0/697693277 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7eff1c0a2040 con 0x557d9ea8a500 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff1c004c80 con 0x557d9ea8a500 2024-03-21T09:19:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff18c00640 1 -- 172.21.15.37:0/697693277 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff1c09cd90 con 0x557d9ea8a500 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff34c7d740 10 client.27193 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557d9eafe280 con 0x557d9ea8a500 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.576+0000 7eff18c00640 1 -- 172.21.15.37:0/697693277 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff1c004c80 con 0x557d9ea8a500 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff18c00640 1 -- 172.21.15.37:0/697693277 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7eff1c0bd290 con 0x557d9ea8a500 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff34c7d740 10 client.27193 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff34c7d740 10 client.27193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff34c7d740 20 client.27193 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff34c7d740 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaff790 0x557d9eb01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff34c7d740 4 client.27193 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d9e9f1ae0 con 0x557d9eaff790 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.577+0000 7eff22a00640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaff790 0x557d9eb01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff22a00640 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaff790 0x557d9eb01b80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e822183f0c46ff75 server_cookie=8af63b3f2b80b74b in_seq=0 out_seq=0 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff18c00640 10 client.27193 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff18c00640 1 -- 172.21.15.37:0/697693277 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557d9e9f1ae0 con 0x557d9eaff790 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff18c00640 10 client.27193 handle_command_reply: tid=0 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 client.27193 shutdown 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 msgr2=0x557d9ea8cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 0x557d9ea8cc70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0bda60 tx=0x7eff1c0ef3b0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 shutdown_connections 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d9ea93870 0x557d9ea8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557d9eaff790 0x557d9eb01b80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d9ea8b1e0 0x557d9ea8d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 --2- 172.21.15.37:0/697693277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d9ea8a500 0x557d9ea8cc70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 >> 172.21.15.37:0/697693277 conn(0x557d9ea4d200 msgr2=0x557d9ea7f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff18c00640 0 client.27193 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 shutdown_connections 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.578+0000 7eff34c7d740 1 -- 172.21.15.37:0/697693277 wait complete. 2024-03-21T09:19:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff34c7d740 20 client.27193 trim_cache size 0 max 16384 2024-03-21T09:19:10.580 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:19:10.580 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:19:10.580 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:19:10.580 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7eff100a3090 msgr2=0x7eff100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7eff100a3090 0x7eff100a5550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0fee30 tx=0x7eff14005e50 comp rx=0 tx=0).stop 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 msgr2=0x7eff2c197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 0x7eff2c197d60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff1c036040 tx=0x7eff1c02cbc0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 shutdown_connections 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7eff100a3090 0x7eff100a5550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff2c10ba50 0x7eff2c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c102f50 0x7eff2c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 --2- 172.21.15.37:0/3308945715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2c102630 0x7eff2c197d60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 >> 172.21.15.37:0/3308945715 conn(0x7eff2c077130 msgr2=0x7eff2c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.579+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 shutdown_connections 2024-03-21T09:19:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.580+0000 7eff30a00640 1 -- 172.21.15.37:0/3308945715 wait complete. 2024-03-21T09:19:10.591 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:10 smithi037 ceph-mon[31830]: pgmap v213: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:19:10.594 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:19:10.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:19:10.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:10 smithi071 ceph-mon[36597]: pgmap v213: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:19:10.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:10 smithi053 ceph-mon[36169]: pgmap v213: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 -- 172.21.15.37:0/2847245497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 msgr2=0x7f4e8c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 --2- 172.21.15.37:0/2847245497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c1006e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009900 tx=0x7f4e7c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 -- 172.21.15.37:0/2847245497 shutdown_connections 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 --2- 172.21.15.37:0/2847245497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e8c10deb0 0x7f4e8c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 --2- 172.21.15.37:0/2847245497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c100cb0 0x7f4e8c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 --2- 172.21.15.37:0/2847245497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c1006e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 -- 172.21.15.37:0/2847245497 >> 172.21.15.37:0/2847245497 conn(0x7f4e8c0fbf00 msgr2=0x7f4e8c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 -- 172.21.15.37:0/2847245497 shutdown_connections 2024-03-21T09:19:10.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.817+0000 7f4e8be00640 1 -- 172.21.15.37:0/2847245497 wait complete. 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.818+0000 7f4e8be00640 1 Processor -- start 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.818+0000 7f4e8be00640 1 -- start start 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e8c100cb0 0x7f4e8c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c10deb0 0x7f4e8c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c113ba0 con 0x7f4e8c10deb0 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c113a20 con 0x7f4e8c100cb0 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c113d20 con 0x7f4e8c100300 2024-03-21T09:19:10.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58540/0 (socket says 172.21.15.37:58540) 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 -- 172.21.15.37:0/4169464248 learned_addr learned my addr 172.21.15.37:0/4169464248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8a000640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e8c100cb0 0x7f4e8c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8b400640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c10deb0 0x7f4e8c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 -- 172.21.15.37:0/4169464248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e8c100cb0 msgr2=0x7f4e8c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e8c100cb0 0x7f4e8c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 -- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c10deb0 msgr2=0x7f4e8c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.819+0000 7f4e8aa00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c10deb0 0x7f4e8c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e8aa00640 1 -- 172.21.15.37:0/4169464248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c19f070 con 0x7f4e8c100300 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e8b400640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c10deb0 0x7f4e8c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e8aa00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c19a080 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c02c240 tx=0x7f4e7c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c043020 con 0x7f4e8c100300 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e7c0054e0 con 0x7f4e8c100300 2024-03-21T09:19:10.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c19f300 con 0x7f4e8c100300 2024-03-21T09:19:10.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c003b50 con 0x7f4e8c100300 2024-03-21T09:19:10.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.820+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c1a6bf0 con 0x7f4e8c100300 2024-03-21T09:19:10.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.821+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c04f470 con 0x7f4e8c100300 2024-03-21T09:19:10.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.823+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4e7c046020 con 0x7f4e8c100300 2024-03-21T09:19:10.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.825+0000 7f4e90a00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4e700a3080 0x7f4e700a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:10.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.825+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f4e7c0f1ca0 con 0x7f4e8c100300 2024-03-21T09:19:10.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.825+0000 7f4e8a000640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4e700a3080 0x7f4e700a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:10.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.826+0000 7f4e8a000640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4e700a3080 0x7f4e700a5540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c101ce0 tx=0x7f4e7800b390 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:10.829+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e7c04d020 con 0x7f4e8c100300 2024-03-21T09:19:11.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.096+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e8c109040 con 0x7f4e8c100300 2024-03-21T09:19:11.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.097+0000 7f4e90a00640 1 -- 172.21.15.37:0/4169464248 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f4e7c0be3f0 con 0x7f4e8c100300 2024-03-21T09:19:11.094 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:19:11.094 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:19:11.095 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:19:11.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4e700a3080 msgr2=0x7f4e700a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4e700a3080 0x7f4e700a5540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c101ce0 tx=0x7f4e7800b390 comp rx=0 tx=0).stop 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 msgr2=0x7f4e8c19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c19a080 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c02c240 tx=0x7f4e7c0042f0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 shutdown_connections 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c10deb0 0x7f4e8c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f4e700a3080 0x7f4e700a5540 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e8c100cb0 0x7f4e8c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 --2- 172.21.15.37:0/4169464248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e8c100300 0x7f4e8c19a080 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 >> 172.21.15.37:0/4169464248 conn(0x7f4e8c0fbf00 msgr2=0x7f4e8c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 shutdown_connections 2024-03-21T09:19:11.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.101+0000 7f4e8be00640 1 -- 172.21.15.37:0/4169464248 wait complete. 2024-03-21T09:19:11.117 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:19:11.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:19:11.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 -- 172.21.15.37:0/3540564286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c10ba50 msgr2=0x7fc72c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 --2- 172.21.15.37:0/3540564286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c10ba50 0x7fc72c10ded0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc724009ca0 tx=0x7fc72402c190 comp rx=0 tx=0).stop 2024-03-21T09:19:11.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 -- 172.21.15.37:0/3540564286 shutdown_connections 2024-03-21T09:19:11.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 --2- 172.21.15.37:0/3540564286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c10ba50 0x7fc72c10ded0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 --2- 172.21.15.37:0/3540564286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc72c102f50 0x7fc72c10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 --2- 172.21.15.37:0/3540564286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc72c102630 0x7fc72c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.364+0000 7fc72be00640 1 -- 172.21.15.37:0/3540564286 >> 172.21.15.37:0/3540564286 conn(0x7fc72c077130 msgr2=0x7fc72c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.365+0000 7fc72be00640 1 -- 172.21.15.37:0/3540564286 shutdown_connections 2024-03-21T09:19:11.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.365+0000 7fc72be00640 1 -- 172.21.15.37:0/3540564286 wait complete. 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 Processor -- start 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 -- start start 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 0x7fc72c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc72c102f50 0x7fc72c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc72c10ba50 0x7fc72c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1117f0 con 0x7fc72c102f50 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c111670 con 0x7fc72c10ba50 2024-03-21T09:19:11.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c111970 con 0x7fc72c102630 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 0x7fc72c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 0x7fc72c199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58558/0 (socket says 172.21.15.37:58558) 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.366+0000 7fc72aa00640 1 -- 172.21.15.37:0/1591667503 learned_addr learned my addr 172.21.15.37:0/1591667503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72a000640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc72c102f50 0x7fc72c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72b400640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc72c10ba50 0x7fc72c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72aa00640 1 -- 172.21.15.37:0/1591667503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc72c10ba50 msgr2=0x7fc72c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72aa00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc72c10ba50 0x7fc72c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72aa00640 1 -- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc72c102f50 msgr2=0x7fc72c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72aa00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc72c102f50 0x7fc72c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72aa00640 1 -- 172.21.15.37:0/1591667503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c19eeb0 con 0x7fc72c102630 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.367+0000 7fc72aa00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 0x7fc72c199ec0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00e9f0 tx=0x7fc71c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.368+0000 7fc730a00640 1 -- 172.21.15.37:0/1591667503 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00cd80 con 0x7fc72c102630 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.368+0000 7fc730a00640 1 -- 172.21.15.37:0/1591667503 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc71c010910 con 0x7fc72c102630 2024-03-21T09:19:11.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.368+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c19f1a0 con 0x7fc72c102630 2024-03-21T09:19:11.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.368+0000 7fc730a00640 1 -- 172.21.15.37:0/1591667503 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c018990 con 0x7fc72c102630 2024-03-21T09:19:11.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.368+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72c1a6a30 con 0x7fc72c102630 2024-03-21T09:19:11.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.371+0000 7fc730a00640 1 -- 172.21.15.37:0/1591667503 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc71c018b30 con 0x7fc72c102630 2024-03-21T09:19:11.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.371+0000 7fc734b94740 1 Processor -- start 2024-03-21T09:19:11.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.371+0000 7fc734b94740 1 -- start start 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc734b94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef974c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc734b94740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 0x5581ef97d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc734b94740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef97dde0 0x5581ef980230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc734b94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ef8d82c0 con 0x5581ef9748b0 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc734b94740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ef880c90 con 0x5581ef975260 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc734b94740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ef8d4360 con 0x5581ef97dde0 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 0x5581ef97d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef974c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 0x5581ef97d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47364/0 (socket says 172.21.15.37:47364) 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 -- 172.21.15.37:0/3882726652 learned_addr learned my addr 172.21.15.37:0/3882726652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef97dde0 0x5581ef980230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 -- 172.21.15.37:0/3882726652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef97dde0 msgr2=0x5581ef980230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef97dde0 0x5581ef980230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 -- 172.21.15.37:0/3882726652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 msgr2=0x5581ef974c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef974c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.372+0000 7fc72aa00640 1 -- 172.21.15.37:0/3882726652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ef8a5080 con 0x5581ef975260 2024-03-21T09:19:11.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.373+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 0x5581ef97d810 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0cd150 tx=0x7fc71c0d85f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8740d6654675fe79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.373+0000 7fc717e00640 1 -- 172.21.15.37:0/3882726652 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0e3070 con 0x5581ef975260 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.373+0000 7fc717e00640 1 -- 172.21.15.37:0/3882726652 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc71c0deae0 con 0x5581ef975260 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.373+0000 7fc717e00640 1 -- 172.21.15.37:0/3882726652 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0ddbd0 con 0x5581ef975260 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.373+0000 7fc730a00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7100a3110 0x7fc7100a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc72b400640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7100a3110 0x7fc7100a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc730a00640 1 -- 172.21.15.37:0/1591667503 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc71c0cff70 con 0x7fc72c102630 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 -- 172.21.15.37:0/3882726652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 msgr2=0x5581ef97d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 0x5581ef97d810 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0cd150 tx=0x7fc71c0d85f0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 -- 172.21.15.37:0/3882726652 shutdown_connections 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef97dde0 0x5581ef980230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef975260 0x5581ef97d810 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 --2- 172.21.15.37:0/3882726652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef974c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.374+0000 7fc734b94740 1 -- 172.21.15.37:0/3882726652 >> 172.21.15.37:0/3882726652 conn(0x5581ef96ef80 msgr2=0x5581ef96f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.375+0000 7fc734b94740 1 -- 172.21.15.37:0/3882726652 shutdown_connections 2024-03-21T09:19:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.375+0000 7fc72b400640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7100a3110 0x7fc7100a55d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc724005e30 tx=0x7fc724005d80 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.375+0000 7fc734b94740 1 -- 172.21.15.37:0/3882726652 wait complete. 2024-03-21T09:19:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.375+0000 7fc734b94740 1 -- start start 2024-03-21T09:19:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.375+0000 7fc734b94740 1 Processor -- start 2024-03-21T09:19:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef97d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ef980d50 con 0x5581ef9748b0 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ef8940d0 con 0x5581ef97dde0 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ef980ed0 con 0x5581ef975260 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc734b94740 10 client.0 _ll_register_callbacks cb 0x5581ef898e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc72b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc72a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc72b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47370/0 (socket says 172.21.15.37:47370) 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc72b400640 1 -- 172.21.15.37:0/3463647795 learned_addr learned my addr 172.21.15.37:0/3463647795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc72a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58576/0 (socket says 172.21.15.37:58576) 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.376+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef97d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc72b400640 1 -- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 msgr2=0x5581ef97aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc72b400640 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc72b400640 1 -- 172.21.15.37:0/3463647795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 msgr2=0x5581ef97d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc72b400640 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef97d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc72b400640 1 -- 172.21.15.37:0/3463647795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581ef5ec3f0 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc72b400640 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97afa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc724004af0 tx=0x7fc7240095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb49e394f2e3bba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc714c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc714c00640 1 -- 172.21.15.37:0/3463647795 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724051020 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.377+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc71c0deae0 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc714c00640 1 -- 172.21.15.37:0/3463647795 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc724032670 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc714c00640 1 -- 172.21.15.37:0/3463647795 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc72405c050 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 10 client.27185 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581ef6b67d0 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc714c00640 1 -- 172.21.15.37:0/3463647795 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc71c0deae0 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc714c00640 1 -- 172.21.15.37:0/3463647795 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc72404d830 con 0x5581ef97dde0 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 10 client.27185 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 10 client.27185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 20 client.27185 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef98dca0 0x5581ef990100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.378+0000 7fc734b94740 4 client.27185 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.379+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581ef8e5800 con 0x5581ef98dca0 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.379+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef98dca0 0x5581ef990100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.379+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef98dca0 0x5581ef990100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b34cebbdb0e0e262 server_cookie=bfdb57be3350aa8e in_seq=0 out_seq=0 2024-03-21T09:19:11.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.379+0000 7fc714c00640 10 client.27185 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc714c00640 1 -- 172.21.15.37:0/3463647795 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5581ef8e5800 con 0x5581ef98dca0 2024-03-21T09:19:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc714c00640 10 client.27185 handle_command_reply: tid=0 2024-03-21T09:19:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 client.27185 shutdown 2024-03-21T09:19:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 msgr2=0x5581ef97afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97afa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc724004af0 tx=0x7fc7240095c0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 shutdown_connections 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97afa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef98dca0 0x5581ef990100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 --2- 172.21.15.37:0/3463647795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef97d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc714c00640 0 client.27185 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.380+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 >> 172.21.15.37:0/3463647795 conn(0x5581ef96ef80 msgr2=0x5581ef970cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.381+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 shutdown_connections 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.381+0000 7fc734b94740 1 -- 172.21.15.37:0/3463647795 wait complete. 2024-03-21T09:19:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.381+0000 7fc734b94740 20 client.27185 trim_cache size 0 max 16384 2024-03-21T09:19:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.402+0000 7fc734b94740 1 Processor -- start 2024-03-21T09:19:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 -- start start 2024-03-21T09:19:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef96dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1b3100 con 0x5581ef9748b0 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1b0c00 con 0x5581ef97dde0 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc734b94740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1aecc0 con 0x5581ef975260 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc72a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc72b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc72a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58578/0 (socket says 172.21.15.37:58578) 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc72a000640 1 -- 172.21.15.37:0/3864252637 learned_addr learned my addr 172.21.15.37:0/3864252637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.403+0000 7fc72aa00640 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef96dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72b400640 1 -- 172.21.15.37:0/3864252637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 msgr2=0x5581ef97d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72b400640 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72b400640 1 -- 172.21.15.37:0/3864252637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 msgr2=0x5581ef96dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72b400640 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef96dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72b400640 1 -- 172.21.15.37:0/3864252637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ef974470 con 0x5581ef97dde0 2024-03-21T09:19:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72a000640 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc72b400640 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97d6b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc724009ca0 tx=0x7fc724003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edcf6c8ae2844740 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.404+0000 7fc717400640 1 -- 172.21.15.37:0/3864252637 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724051030 con 0x5581ef97dde0 2024-03-21T09:19:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc717400640 1 -- 172.21.15.37:0/3864252637 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc724004850 con 0x5581ef97dde0 2024-03-21T09:19:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc717400640 1 -- 172.21.15.37:0/3864252637 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724039040 con 0x5581ef97dde0 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 -- 172.21.15.37:0/3864252637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 msgr2=0x5581ef97d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97d6b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc724009ca0 tx=0x7fc724003f50 comp rx=0 tx=0).stop 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 -- 172.21.15.37:0/3864252637 shutdown_connections 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef97d6b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef975260 0x5581ef97d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 --2- 172.21.15.37:0/3864252637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef9748b0 0x5581ef96dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 -- 172.21.15.37:0/3864252637 >> 172.21.15.37:0/3864252637 conn(0x5581ef96ef80 msgr2=0x5581ef96d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 -- 172.21.15.37:0/3864252637 shutdown_connections 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.405+0000 7fc734b94740 1 -- 172.21.15.37:0/3864252637 wait complete. 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 -- start start 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 Processor -- start 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 0x5581ef981ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef975260 0x5581ef9821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef9e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1af2c0 con 0x5581ef975260 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1b05a0 con 0x5581ef97dde0 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.406+0000 7fc734b94740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c1af8e0 con 0x5581ef9748b0 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc734b94740 10 client.0 _ll_register_callbacks cb 0x7fc724044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 0x5581ef981ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef9e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 0x5581ef981ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58588/0 (socket says 172.21.15.37:58588) 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 -- 172.21.15.37:0/1098938762 learned_addr learned my addr 172.21.15.37:0/1098938762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72a000640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef975260 0x5581ef9821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 -- 172.21.15.37:0/1098938762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 msgr2=0x5581ef9e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef9e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 -- 172.21.15.37:0/1098938762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef975260 msgr2=0x5581ef9821e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef975260 0x5581ef9821e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 -- 172.21.15.37:0/1098938762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581ef9e8b00 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72b400640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef9e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72a000640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef975260 0x5581ef9821e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc72aa00640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 0x5581ef981ca0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc71c09bc60 tx=0x7fc71c0dd450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=328af81bb922c33d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.407+0000 7fc714c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc714c00640 1 -- 172.21.15.37:0/1098938762 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c098200 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc714c00640 1 -- 172.21.15.37:0/1098938762 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc71c010ac0 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc724004850 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc714c00640 1 -- 172.21.15.37:0/1098938762 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc71c0d38f0 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 10 client.27217 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581ef9e9e00 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc714c00640 1 -- 172.21.15.37:0/1098938762 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc724004850 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc714c00640 1 -- 172.21.15.37:0/1098938762 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc71c0985d0 con 0x5581ef9748b0 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 10 client.27217 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 10 client.27217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:19:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 20 client.27217 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef9ea080 0x5581ef9ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 4 client.27217 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.408+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581ef8dfb50 con 0x5581ef9ea080 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.409+0000 7fc72a000640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef9ea080 0x5581ef9ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.409+0000 7fc72a000640 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef9ea080 0x5581ef9ec470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2a3f34768e56222 server_cookie=fce562557ba402c2 in_seq=0 out_seq=0 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.409+0000 7fc714c00640 10 client.27217 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.409+0000 7fc714c00640 1 -- 172.21.15.37:0/1098938762 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581ef8dfb50 con 0x5581ef9ea080 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.409+0000 7fc714c00640 10 client.27217 handle_command_reply: tid=0 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.409+0000 7fc734b94740 1 client.27217 shutdown 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 msgr2=0x5581ef981ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 0x5581ef981ca0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc71c09bc60 tx=0x7fc71c0dd450 comp rx=0 tx=0).stop 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 shutdown_connections 2024-03-21T09:19:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5581ef97dde0 0x5581ef9e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5581ef9ea080 0x5581ef9ec470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581ef975260 0x5581ef9821e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 --2- 172.21.15.37:0/1098938762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581ef9748b0 0x5581ef981ca0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc714c00640 0 client.27217 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 >> 172.21.15.37:0/1098938762 conn(0x5581ef96ef80 msgr2=0x5581ef96e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 shutdown_connections 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 1 -- 172.21.15.37:0/1098938762 wait complete. 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.410+0000 7fc734b94740 20 client.27217 trim_cache size 0 max 16384 2024-03-21T09:19:11.411 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:19:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7100a3110 msgr2=0x7fc7100a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7100a3110 0x7fc7100a55d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc724005e30 tx=0x7fc724005d80 comp rx=0 tx=0).stop 2024-03-21T09:19:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 msgr2=0x7fc72c199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 0x7fc72c199ec0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00e9f0 tx=0x7fc71c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 shutdown_connections 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc7100a3110 0x7fc7100a55d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc72c10ba50 0x7fc72c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc72c102f50 0x7fc72c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 --2- 172.21.15.37:0/1591667503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc72c102630 0x7fc72c199ec0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 >> 172.21.15.37:0/1591667503 conn(0x7fc72c077130 msgr2=0x7fc72c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 shutdown_connections 2024-03-21T09:19:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.411+0000 7fc72be00640 1 -- 172.21.15.37:0/1591667503 wait complete. 2024-03-21T09:19:11.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:11 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4169464248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:19:11.426 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:19:11.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:19:11.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:11 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4169464248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:19:11.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 -- 172.21.15.37:0/2922555310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 msgr2=0x7f8fcc110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 --2- 172.21.15.37:0/2922555310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc110130 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009ca0 tx=0x7f8fc402c190 comp rx=0 tx=0).stop 2024-03-21T09:19:11.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 -- 172.21.15.37:0/2922555310 shutdown_connections 2024-03-21T09:19:11.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 --2- 172.21.15.37:0/2922555310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc110130 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 --2- 172.21.15.37:0/2922555310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fcc074390 0x7f8fcc10d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 --2- 172.21.15.37:0/2922555310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 0x7f8fcc073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.692+0000 7f8fd2600640 1 -- 172.21.15.37:0/2922555310 >> 172.21.15.37:0/2922555310 conn(0x7f8fcc0fcfb0 msgr2=0x7f8fcc0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.693+0000 7f8fd2600640 1 -- 172.21.15.37:0/2922555310 shutdown_connections 2024-03-21T09:19:11.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.693+0000 7f8fd2600640 1 -- 172.21.15.37:0/2922555310 wait complete. 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 Processor -- start 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 -- start start 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 0x7f8fcc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fcc074390 0x7f8fcc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc113a80 con 0x7f8fcc073a70 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc113900 con 0x7f8fcc074390 2024-03-21T09:19:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.694+0000 7f8fd2600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc113c00 con 0x7f8fcc10dd20 2024-03-21T09:19:11.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 0x7f8fcc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fcc074390 0x7f8fcc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 0x7f8fcc199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34328/0 (socket says 172.21.15.37:34328) 2024-03-21T09:19:11.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1200640 1 -- 172.21.15.37:0/239923920 learned_addr learned my addr 172.21.15.37:0/239923920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 -- 172.21.15.37:0/239923920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fcc074390 msgr2=0x7f8fcc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fcc074390 0x7f8fcc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 -- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 msgr2=0x7f8fcc199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 0x7f8fcc199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 -- 172.21.15.37:0/239923920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc19ef10 con 0x7f8fcc10dd20 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.695+0000 7f8fd1c00640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc19e7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8fc402c7a0 tx=0x7f8fc4035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.696+0000 7f8fc9600640 1 -- 172.21.15.37:0/239923920 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4039040 con 0x7f8fcc10dd20 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.696+0000 7f8fc9600640 1 -- 172.21.15.37:0/239923920 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4044070 con 0x7f8fcc10dd20 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.696+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc19f1a0 con 0x7f8fcc10dd20 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.696+0000 7f8fc9600640 1 -- 172.21.15.37:0/239923920 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40328d0 con 0x7f8fcc10dd20 2024-03-21T09:19:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.696+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc1a6a90 con 0x7f8fcc10dd20 2024-03-21T09:19:11.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:11 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4169464248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:19:11.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.699+0000 7f8fc9600640 1 -- 172.21.15.37:0/239923920 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8fc4032a70 con 0x7f8fcc10dd20 2024-03-21T09:19:11.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.699+0000 7f8fd69b4740 1 Processor -- start 2024-03-21T09:19:11.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.699+0000 7f8fd69b4740 1 -- start start 2024-03-21T09:19:11.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c93961c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c938c52c0 con 0x562c9396ade0 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9386dc90 con 0x562c939618b0 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c938c1360 con 0x562c93962260 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c93961c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c93961c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47410/0 (socket says 172.21.15.37:47410) 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fcbe00640 1 -- 172.21.15.37:0/1434978572 learned_addr learned my addr 172.21.15.37:0/1434978572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd1200640 1 -- 172.21.15.37:0/1434978572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 msgr2=0x562c93961c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd1200640 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c93961c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd1200640 1 -- 172.21.15.37:0/1434978572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 msgr2=0x562c9396d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd1200640 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fd1200640 1 -- 172.21.15.37:0/1434978572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c93892080 con 0x562c93962260 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.700+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c93961c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fc9600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8fb40a3110 0x7f8fb40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fd1200640 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a810 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009c80 tx=0x7f8fc0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6110874103ff0799 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fc9600640 1 -- 172.21.15.37:0/239923920 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f8fc40f1f10 con 0x7f8fcc10dd20 2024-03-21T09:19:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8fb40a3110 0x7f8fb40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fbe000640 1 -- 172.21.15.37:0/1434978572 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004230 con 0x562c93962260 2024-03-21T09:19:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fbe000640 1 -- 172.21.15.37:0/1434978572 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc00043d0 con 0x562c93962260 2024-03-21T09:19:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.701+0000 7f8fbe000640 1 -- 172.21.15.37:0/1434978572 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004cd0 con 0x562c93962260 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8fb40a3110 0x7f8fb40a55d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800b6c0 tx=0x7f8fb800b410 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 -- 172.21.15.37:0/1434978572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 msgr2=0x562c9396a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a810 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009c80 tx=0x7f8fc0011810 comp rx=0 tx=0).stop 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 -- 172.21.15.37:0/1434978572 shutdown_connections 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a810 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/1434978572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c93961c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 -- 172.21.15.37:0/1434978572 >> 172.21.15.37:0/1434978572 conn(0x562c9395bf80 msgr2=0x562c9395c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 -- 172.21.15.37:0/1434978572 shutdown_connections 2024-03-21T09:19:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.702+0000 7f8fd69b4740 1 -- 172.21.15.37:0/1434978572 wait complete. 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.703+0000 7f8fd69b4740 1 -- start start 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.703+0000 7f8fd69b4740 1 Processor -- start 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c939618b0 0x562c9396a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 0x562c93967a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c9396ade0 0x562c93967fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9396dd50 con 0x562c939618b0 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c938810d0 con 0x562c93962260 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9396ded0 con 0x562c9396ade0 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 0x562c93967a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c9396ade0 0x562c93967fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 0x562c93967a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47416/0 (socket says 172.21.15.37:47416) 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fcbe00640 1 -- 172.21.15.37:0/3614162295 learned_addr learned my addr 172.21.15.37:0/3614162295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd1200640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c939618b0 0x562c9396a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.704+0000 7f8fd69b4740 10 client.0 _ll_register_callbacks cb 0x562c93885e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fcbe00640 1 -- 172.21.15.37:0/3614162295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c9396ade0 msgr2=0x562c93967fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c9396ade0 0x562c93967fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fcbe00640 1 -- 172.21.15.37:0/3614162295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c939618b0 msgr2=0x562c9396a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c939618b0 0x562c9396a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fcbe00640 1 -- 172.21.15.37:0/3614162295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c935d93f0 con 0x562c93962260 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc00043d0 con 0x562c93962260 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fd1c00640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c9396ade0 0x562c93967fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 0x562c93967a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80129d0 tx=0x7f8fb8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0979bb3e088223 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fb2a00640 10 client.27203 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3614162295 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004020 con 0x562c93962260 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.705+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3614162295 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f8fb80041c0 con 0x562c93962260 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3614162295 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fb80043c0 con 0x562c93962260 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 10 client.27203 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x562c936a37d0 con 0x562c93962260 2024-03-21T09:19:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3614162295 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8fb8027030 con 0x562c93962260 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 10 client.27203 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 10 client.27203 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 20 client.27203 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c9397aa60 0x562c9397ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 4 client.27203 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c938d2800 con 0x562c9397aa60 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.706+0000 7f8fd1200640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c9397aa60 0x562c9397ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.707+0000 7f8fd1200640 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c9397aa60 0x562c9397ce70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=516dde7dc29e248c server_cookie=4e3668d5f36126af in_seq=0 out_seq=0 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.707+0000 7f8fb2a00640 10 client.27203 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3614162295 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562c938d2800 con 0x562c9397aa60 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fb2a00640 10 client.27203 handle_command_reply: tid=0 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 client.27203 shutdown 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 msgr2=0x562c93967a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 0x562c93967a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80129d0 tx=0x7f8fb8004730 comp rx=0 tx=0).stop 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 shutdown_connections 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c9397aa60 0x562c9397ce70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c9396ade0 0x562c93967fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c93962260 0x562c93967a40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3614162295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c939618b0 0x562c9396a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 >> 172.21.15.37:0/3614162295 conn(0x562c9395bf80 msgr2=0x562c9395dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.708+0000 7f8fb2a00640 0 client.27203 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.709+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 shutdown_connections 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.709+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3614162295 wait complete. 2024-03-21T09:19:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.709+0000 7f8fd69b4740 20 client.27203 trim_cache size 0 max 16384 2024-03-21T09:19:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 Processor -- start 2024-03-21T09:19:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 -- start start 2024-03-21T09:19:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c93964ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1b3070 con 0x562c9396ade0 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1b0b70 con 0x562c939618b0 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1aec30 con 0x562c93962260 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58628/0 (socket says 172.21.15.37:58628) 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fd1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c93964ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.723+0000 7f8fcbe00640 1 -- 172.21.15.37:0/2864256036 learned_addr learned my addr 172.21.15.37:0/2864256036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fcbe00640 1 -- 172.21.15.37:0/2864256036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 msgr2=0x562c9396a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fcbe00640 1 -- 172.21.15.37:0/2864256036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 msgr2=0x562c93964ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c93964ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fcbe00640 1 -- 172.21.15.37:0/2864256036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c93962000 con 0x562c93962260 2024-03-21T09:19:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fd1200640 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a6b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800bf40 tx=0x7f8fb8022040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71d4d390cfca17ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fbd600640 1 -- 172.21.15.37:0/2864256036 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80125b0 con 0x562c93962260 2024-03-21T09:19:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fbd600640 1 -- 172.21.15.37:0/2864256036 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc00043d0 con 0x562c93962260 2024-03-21T09:19:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.724+0000 7f8fbd600640 1 -- 172.21.15.37:0/2864256036 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004990 con 0x562c93962260 2024-03-21T09:19:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 -- 172.21.15.37:0/2864256036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 msgr2=0x562c9396a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a6b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800bf40 tx=0x7f8fb8022040 comp rx=0 tx=0).stop 2024-03-21T09:19:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 -- 172.21.15.37:0/2864256036 shutdown_connections 2024-03-21T09:19:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c93964ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396a6b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/2864256036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 -- 172.21.15.37:0/2864256036 >> 172.21.15.37:0/2864256036 conn(0x562c9395bf80 msgr2=0x562c93963040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 -- 172.21.15.37:0/2864256036 shutdown_connections 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.725+0000 7f8fd69b4740 1 -- 172.21.15.37:0/2864256036 wait complete. 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.726+0000 7f8fd69b4740 1 -- start start 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.726+0000 7f8fd69b4740 1 Processor -- start 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1af230 con 0x562c9396ade0 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1b0510 con 0x562c939618b0 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1af850 con 0x562c93962260 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd69b4740 10 client.0 _ll_register_callbacks cb 0x7f8fb800bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47442/0 (socket says 172.21.15.37:47442) 2024-03-21T09:19:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 -- 172.21.15.37:0/3084118476 learned_addr learned my addr 172.21.15.37:0/3084118476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1c00640 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 -- 172.21.15.37:0/3084118476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 msgr2=0x562c9396ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 -- 172.21.15.37:0/3084118476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 msgr2=0x562c9396ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.727+0000 7f8fd1200640 1 -- 172.21.15.37:0/3084118476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c939ddae0 con 0x562c939618b0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fd1200640 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396e4c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009bd0 tx=0x7f8fc0004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66b419c7d97cb3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3084118476 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001e040 con 0x562c939618b0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc00043d0 con 0x562c939618b0 2024-03-21T09:19:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3084118476 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f8fc002bab0 con 0x562c939618b0 2024-03-21T09:19:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3084118476 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc0039050 con 0x562c939618b0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fd69b4740 10 client.27209 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.728+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562c939ded10 con 0x562c939618b0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3084118476 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc00043d0 con 0x562c939618b0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3084118476 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8fc0024040 con 0x562c939618b0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fd69b4740 10 client.27209 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fd69b4740 10 client.27209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fd69b4740 20 client.27209 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c939df2a0 0x562c939e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fd69b4740 4 client.27209 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c938ccb50 con 0x562c939df2a0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.729+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c939df2a0 0x562c939e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fcbe00640 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c939df2a0 0x562c939e1690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ad28d09ff0b4d3e server_cookie=8811091d8c991224 in_seq=0 out_seq=0 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fb2a00640 10 client.27209 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fb2a00640 1 -- 172.21.15.37:0/3084118476 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c938ccb50 con 0x562c939df2a0 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fb2a00640 10 client.27209 handle_command_reply: tid=0 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 client.27209 shutdown 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 msgr2=0x562c9396e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396e4c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009bd0 tx=0x7f8fc0004580 comp rx=0 tx=0).stop 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 shutdown_connections 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x562c939df2a0 0x562c939e1690 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c9396ade0 0x562c9396ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c93962260 0x562c9396ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 --2- 172.21.15.37:0/3084118476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c939618b0 0x562c9396e4c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 >> 172.21.15.37:0/3084118476 conn(0x562c9395bf80 msgr2=0x562c93962690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fb2a00640 0 client.27209 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 shutdown_connections 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 1 -- 172.21.15.37:0/3084118476 wait complete. 2024-03-21T09:19:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.730+0000 7f8fd69b4740 20 client.27209 trim_cache size 0 max 16384 2024-03-21T09:19:11.731 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:19:11.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8fb40a3110 msgr2=0x7f8fb40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8fb40a3110 0x7f8fb40a55d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800b6c0 tx=0x7f8fb800b410 comp rx=0 tx=0).stop 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 msgr2=0x7f8fcc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc19e7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8fc402c7a0 tx=0x7f8fc4035d10 comp rx=0 tx=0).stop 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 shutdown_connections 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc10dd20 0x7f8fcc19e7e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8fb40a3110 0x7f8fb40a55d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fcc074390 0x7f8fcc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 --2- 172.21.15.37:0/239923920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fcc073a70 0x7f8fcc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 >> 172.21.15.37:0/239923920 conn(0x7f8fcc0fcfb0 msgr2=0x7f8fcc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.731+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 shutdown_connections 2024-03-21T09:19:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.732+0000 7f8fd2600640 1 -- 172.21.15.37:0/239923920 wait complete. 2024-03-21T09:19:11.749 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:19:11.750 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 -- 172.21.15.37:0/2997965462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c109670 msgr2=0x7f7c5c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 --2- 172.21.15.37:0/2997965462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c109670 0x7f7c5c109a50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009900 tx=0x7f7c4c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 -- 172.21.15.37:0/2997965462 shutdown_connections 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 --2- 172.21.15.37:0/2997965462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c104020 0x7f7c5c110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 --2- 172.21.15.37:0/2997965462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c103660 0x7f7c5c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 --2- 172.21.15.37:0/2997965462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c109670 0x7f7c5c109a50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 -- 172.21.15.37:0/2997965462 >> 172.21.15.37:0/2997965462 conn(0x7f7c5c0ff110 msgr2=0x7f7c5c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 -- 172.21.15.37:0/2997965462 shutdown_connections 2024-03-21T09:19:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.985+0000 7f7c5b200640 1 -- 172.21.15.37:0/2997965462 wait complete. 2024-03-21T09:19:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.987+0000 7f7c5b200640 1 Processor -- start 2024-03-21T09:19:11.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 -- start start 2024-03-21T09:19:11.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c103660 0x7f7c5c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 0x7f7c5c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c109670 0x7f7c5c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c113a90 con 0x7f7c5c103660 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c113910 con 0x7f7c5c104020 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.988+0000 7f7c5b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c113c10 con 0x7f7c5c109670 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 0x7f7c5c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c103660 0x7f7c5c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 0x7f7c5c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47458/0 (socket says 172.21.15.37:47458) 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 -- 172.21.15.37:0/1544395525 learned_addr learned my addr 172.21.15.37:0/1544395525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c5a800640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c109670 0x7f7c5c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 -- 172.21.15.37:0/1544395525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c109670 msgr2=0x7f7c5c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c109670 0x7f7c5c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 -- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c103660 msgr2=0x7f7c5c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c103660 0x7f7c5c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 -- 172.21.15.37:0/1544395525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c19ef80 con 0x7f7c5c104020 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c5a800640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c109670 0x7f7c5c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.989+0000 7f7c53e00640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 0x7f7c5c19a4d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400e9f0 tx=0x7f7c4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.990+0000 7f7c52a00640 1 -- 172.21.15.37:0/1544395525 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400cd80 con 0x7f7c5c104020 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.990+0000 7f7c52a00640 1 -- 172.21.15.37:0/1544395525 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c44010910 con 0x7f7c5c104020 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.990+0000 7f7c52a00640 1 -- 172.21.15.37:0/1544395525 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44018990 con 0x7f7c5c104020 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.990+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c19f270 con 0x7f7c5c104020 2024-03-21T09:19:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.990+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c1a6b00 con 0x7f7c5c104020 2024-03-21T09:19:11.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.994+0000 7f7c63c35740 1 Processor -- start 2024-03-21T09:19:11.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.994+0000 7f7c63c35740 1 -- start start 2024-03-21T09:19:11.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c52a00640 1 -- 172.21.15.37:0/1544395525 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7c44010ac0 con 0x7f7c5c104020 2024-03-21T09:19:11.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824e6de0 0x55f6824e9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c63c35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6824412c0 con 0x55f6824dd8b0 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c63c35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6823e9c90 con 0x55f6824e6de0 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c63c35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f68243d360 con 0x55f6824de260 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58664/0 (socket says 172.21.15.37:58664) 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c59e00640 1 -- 172.21.15.37:0/2716746573 learned_addr learned my addr 172.21.15.37:0/2716746573 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c53e00640 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824e6de0 0x55f6824e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.995+0000 7f7c5a800640 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c59e00640 1 -- 172.21.15.37:0/2716746573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824e6de0 msgr2=0x55f6824e9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c59e00640 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824e6de0 0x55f6824e9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c59e00640 1 -- 172.21.15.37:0/2716746573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 msgr2=0x55f6824ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:11.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c59e00640 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c59e00640 1 -- 172.21.15.37:0/2716746573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f68240e080 con 0x55f6824de260 2024-03-21T09:19:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c59e00640 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e6810 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009230 tx=0x7f7c4c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43b7fbafb8fadb4a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c4b400640 1 -- 172.21.15.37:0/2716746573 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c042070 con 0x55f6824de260 2024-03-21T09:19:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c4b400640 1 -- 172.21.15.37:0/2716746573 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02c690 con 0x55f6824de260 2024-03-21T09:19:11.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.996+0000 7f7c4b400640 1 -- 172.21.15.37:0/2716746573 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c004c80 con 0x55f6824de260 2024-03-21T09:19:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c52a00640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7c400a3110 0x7f7c400a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c52a00640 1 -- 172.21.15.37:0/1544395525 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f7c440cff90 con 0x7f7c5c104020 2024-03-21T09:19:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c5a800640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7c400a3110 0x7f7c400a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 -- 172.21.15.37:0/2716746573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 msgr2=0x55f6824e6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e6810 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009230 tx=0x7f7c4c005df0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 -- 172.21.15.37:0/2716746573 shutdown_connections 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824e6de0 0x55f6824e9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e6810 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2716746573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 -- 172.21.15.37:0/2716746573 >> 172.21.15.37:0/2716746573 conn(0x55f6824d7f80 msgr2=0x55f6824d8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.997+0000 7f7c63c35740 1 -- 172.21.15.37:0/2716746573 shutdown_connections 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.998+0000 7f7c63c35740 1 -- 172.21.15.37:0/2716746573 wait complete. 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.998+0000 7f7c5a800640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7c400a3110 0x7f7c400a55d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c540099a0 tx=0x7f7c54009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.998+0000 7f7c63c35740 1 -- start start 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 Processor -- start 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6824e9cf0 con 0x55f6824e6de0 2024-03-21T09:19:11.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6823fd0d0 con 0x55f6824dd8b0 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6824e9e70 con 0x55f6824de260 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c63c35740 10 client.0 _ll_register_callbacks cb 0x55f682401e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:11.999+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47478/0 (socket says 172.21.15.37:47478) 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c59e00640 1 -- 172.21.15.37:0/2892878219 learned_addr learned my addr 172.21.15.37:0/2892878219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c5a800640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c53e00640 1 -- 172.21.15.37:0/2892878219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 msgr2=0x55f6824e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c53e00640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c53e00640 1 -- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 msgr2=0x55f6824e3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c53e00640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c53e00640 1 -- 172.21.15.37:0/2892878219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6821553f0 con 0x55f6824de260 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c59e00640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c5a800640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c53e00640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e3a70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7c440cd270 tx=0x7f7c4409bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800f2a6cc71a17d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c3fe00640 1 -- 172.21.15.37:0/2892878219 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440cd6f0 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.000+0000 7f7c3fe00640 1 -- 172.21.15.37:0/2892878219 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f7c440a4490 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c4c02c690 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c3fe00640 1 -- 172.21.15.37:0/2892878219 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c4409c610 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 10 client.27247 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f68221f7d0 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c3fe00640 1 -- 172.21.15.37:0/2892878219 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02c690 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c3fe00640 1 -- 172.21.15.37:0/2892878219 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7c44018c50 con 0x55f6824de260 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 10 client.27247 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 20 client.27247 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6824f6c70 0x55f6824f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 4 client.27247 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.001+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f68244e800 con 0x55f6824f6c70 2024-03-21T09:19:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.002+0000 7f7c59e00640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6824f6c70 0x55f6824f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.002+0000 7f7c3fe00640 10 client.27247 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.002+0000 7f7c59e00640 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6824f6c70 0x55f6824f9060 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2541297d1d720b server_cookie=b0127d7adb90f85a in_seq=0 out_seq=0 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.003+0000 7f7c3fe00640 1 -- 172.21.15.37:0/2892878219 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f68244e800 con 0x55f6824f6c70 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.003+0000 7f7c3fe00640 10 client.27247 handle_command_reply: tid=0 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.003+0000 7f7c63c35740 1 client.27247 shutdown 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.003+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 msgr2=0x55f6824e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.003+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e3a70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7c440cd270 tx=0x7f7c4409bce0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 shutdown_connections 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e3a70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f6824f6c70 0x55f6824f9060 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 --2- 172.21.15.37:0/2892878219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c3fe00640 0 client.27247 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 >> 172.21.15.37:0/2892878219 conn(0x55f6824d7f80 msgr2=0x55f6824d9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 shutdown_connections 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 1 -- 172.21.15.37:0/2892878219 wait complete. 2024-03-21T09:19:12.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.004+0000 7f7c63c35740 20 client.27247 trim_cache size 0 max 16384 2024-03-21T09:19:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 Processor -- start 2024-03-21T09:19:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 -- start start 2024-03-21T09:19:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1b31b0 con 0x55f6824e6de0 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1b0cb0 con 0x55f6824dd8b0 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c63c35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1aed70 con 0x55f6824de260 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47490/0 (socket says 172.21.15.37:47490) 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58682/0 (socket says 172.21.15.37:58682) 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c59e00640 1 -- 172.21.15.37:0/108684939 learned_addr learned my addr 172.21.15.37:0/108684939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.018+0000 7f7c5a800640 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c53e00640 1 -- 172.21.15.37:0/108684939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 msgr2=0x55f6824e61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c53e00640 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c53e00640 1 -- 172.21.15.37:0/108684939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 msgr2=0x55f6824e0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c53e00640 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c53e00640 1 -- 172.21.15.37:0/108684939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6824d8b50 con 0x55f6824de260 2024-03-21T09:19:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c59e00640 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c53e00640 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e66e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7c4409bc10 tx=0x7f7c4409be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450a6b43c9e7d96d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c4aa00640 1 -- 172.21.15.37:0/108684939 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4409bf10 con 0x55f6824de260 2024-03-21T09:19:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c4aa00640 1 -- 172.21.15.37:0/108684939 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c440a4920 con 0x55f6824de260 2024-03-21T09:19:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.019+0000 7f7c4aa00640 1 -- 172.21.15.37:0/108684939 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44018d90 con 0x55f6824de260 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 -- 172.21.15.37:0/108684939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 msgr2=0x55f6824e66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e66e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7c4409bc10 tx=0x7f7c4409be10 comp rx=0 tx=0).stop 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 -- 172.21.15.37:0/108684939 shutdown_connections 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824e6de0 0x55f6824e0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824de260 0x55f6824e66e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 --2- 172.21.15.37:0/108684939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824dd8b0 0x55f6824e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 -- 172.21.15.37:0/108684939 >> 172.21.15.37:0/108684939 conn(0x55f6824d7f80 msgr2=0x55f6824d7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 -- 172.21.15.37:0/108684939 shutdown_connections 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 -- 172.21.15.37:0/108684939 wait complete. 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 -- start start 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.020+0000 7f7c63c35740 1 Processor -- start 2024-03-21T09:19:12.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 0x55f6824ed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 0x55f6825514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1af370 con 0x55f6824dd8b0 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1b0650 con 0x55f6824de260 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1af990 con 0x55f6824e6de0 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c63c35740 10 client.0 _ll_register_callbacks cb 0x7f7c440d3770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 0x55f6825514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 0x55f6824ed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 0x55f6825514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58696/0 (socket says 172.21.15.37:58696) 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 0x55f6824ed1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47500/0 (socket says 172.21.15.37:47500) 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c5a800640 1 -- 172.21.15.37:0/1803095752 learned_addr learned my addr 172.21.15.37:0/1803095752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:12.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.021+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ecc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34394/0 (socket says 172.21.15.37:34394) 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c53e00640 1 -- 172.21.15.37:0/1803095752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 msgr2=0x55f6825514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c53e00640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 0x55f6825514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c53e00640 1 -- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 msgr2=0x55f6824ecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c53e00640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c53e00640 1 -- 172.21.15.37:0/1803095752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f682551a70 con 0x55f6824de260 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c59e00640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c5a800640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 0x55f6825514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c53e00640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 0x55f6824ed1b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7c4409bc10 tx=0x7f7c440d36a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc3bf73c18dc3b43 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c3fe00640 1 -- 172.21.15.37:0/1803095752 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44018d30 con 0x55f6824de260 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c3fe00640 1 -- 172.21.15.37:0/1803095752 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f7c44010430 con 0x55f6824de260 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c440a4920 con 0x55f6824de260 2024-03-21T09:19:12.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c3fe00640 1 -- 172.21.15.37:0/1803095752 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c440a46a0 con 0x55f6824de260 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c63c35740 10 client.27227 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.022+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f682552d70 con 0x55f6824de260 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c3fe00640 1 -- 172.21.15.37:0/1803095752 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c440a4920 con 0x55f6824de260 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c3fe00640 1 -- 172.21.15.37:0/1803095752 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7c440b8070 con 0x55f6824de260 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c63c35740 10 client.27227 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c63c35740 10 client.27227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c63c35740 20 client.27227 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c63c35740 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f682553390 0x55f6825557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c63c35740 4 client.27227 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f682448b50 con 0x55f682553390 2024-03-21T09:19:12.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.023+0000 7f7c59e00640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f682553390 0x55f6825557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:12.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c59e00640 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f682553390 0x55f6825557b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b76ea233a2fe4861 server_cookie=86d13af589a9014c in_seq=0 out_seq=0 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c3fe00640 10 client.27227 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c3fe00640 1 -- 172.21.15.37:0/1803095752 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f682448b50 con 0x55f682553390 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c3fe00640 10 client.27227 handle_command_reply: tid=0 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 client.27227 shutdown 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 msgr2=0x55f6824ed1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 0x55f6824ed1b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7c4409bc10 tx=0x7f7c440d36a0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 shutdown_connections 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6824e6de0 0x55f6825514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55f682553390 0x55f6825557b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6824de260 0x55f6824ed1b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 --2- 172.21.15.37:0/1803095752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6824dd8b0 0x55f6824ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 >> 172.21.15.37:0/1803095752 conn(0x55f6824d7f80 msgr2=0x55f6824d68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.024+0000 7f7c3fe00640 0 client.27227 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:12.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.025+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 shutdown_connections 2024-03-21T09:19:12.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.025+0000 7f7c63c35740 1 -- 172.21.15.37:0/1803095752 wait complete. 2024-03-21T09:19:12.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.025+0000 7f7c63c35740 20 client.27227 trim_cache size 0 max 16384 2024-03-21T09:19:12.026 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.025+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7c400a3110 msgr2=0x7f7c400a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.025+0000 7f7c5b200640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7c400a3110 0x7f7c400a55d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c540099a0 tx=0x7f7c54009500 comp rx=0 tx=0).stop 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 msgr2=0x7f7c5c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 0x7f7c5c19a4d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400e9f0 tx=0x7f7c4400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 shutdown_connections 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5c109670 0x7f7c5c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f7c400a3110 0x7f7c400a55d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c5c104020 0x7f7c5c19a4d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 --2- 172.21.15.37:0/1544395525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c5c103660 0x7f7c5c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 >> 172.21.15.37:0/1544395525 conn(0x7f7c5c0ff110 msgr2=0x7f7c5c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 shutdown_connections 2024-03-21T09:19:12.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:12.026+0000 7f7c5b200640 1 -- 172.21.15.37:0/1544395525 wait complete. 2024-03-21T09:19:12.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:12 smithi071 ceph-mon[36597]: pgmap v214: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:19:12.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:12 smithi037 ceph-mon[31830]: pgmap v214: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:19:12.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:12 smithi053 ceph-mon[36169]: pgmap v214: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:19:13.042 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:19:13.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:19:13.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:19:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 -- 172.21.15.37:0/3983016564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100fc0 msgr2=0x7fa8c4104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 --2- 172.21.15.37:0/3983016564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100fc0 0x7fa8c4104fb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009900 tx=0x7fa8b002bc30 comp rx=0 tx=0).stop 2024-03-21T09:19:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 -- 172.21.15.37:0/3983016564 shutdown_connections 2024-03-21T09:19:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 --2- 172.21.15.37:0/3983016564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 --2- 172.21.15.37:0/3983016564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100fc0 0x7fa8c4104fb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 --2- 172.21.15.37:0/3983016564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4100610 0x7fa8c41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 -- 172.21.15.37:0/3983016564 >> 172.21.15.37:0/3983016564 conn(0x7fa8c4077130 msgr2=0x7fa8c40fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 -- 172.21.15.37:0/3983016564 shutdown_connections 2024-03-21T09:19:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.291+0000 7fa8c3200640 1 -- 172.21.15.37:0/3983016564 wait complete. 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.292+0000 7fa8c3200640 1 Processor -- start 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 -- start start 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100610 0x7fa8c410dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c410e670 0x7fa8c41a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4111820 con 0x7fa8c410e670 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41116a0 con 0x7fa8c4105690 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41119a0 con 0x7fa8c4100610 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100610 0x7fa8c410dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47512/0 (socket says 172.21.15.37:47512) 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c1400640 1 -- 172.21.15.37:0/224479921 learned_addr learned my addr 172.21.15.37:0/224479921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.293+0000 7fa8c2800640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c410e670 0x7fa8c41a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1e00640 1 -- 172.21.15.37:0/224479921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 msgr2=0x7fa8c410e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1e00640 1 -- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c410e670 msgr2=0x7fa8c41a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c410e670 0x7fa8c41a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1e00640 1 -- 172.21.15.37:0/224479921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c41a6670 con 0x7fa8c4100610 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1400640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100610 0x7fa8c410dbf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400d6d0 tx=0x7fa8b400dad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8baa00640 1 -- 172.21.15.37:0/224479921 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4017070 con 0x7fa8c4100610 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8baa00640 1 -- 172.21.15.37:0/224479921 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b40047a0 con 0x7fa8c4100610 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c41a6960 con 0x7fa8c4100610 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8baa00640 1 -- 172.21.15.37:0/224479921 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4009c80 con 0x7fa8c4100610 2024-03-21T09:19:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.294+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c41a6f00 con 0x7fa8c4100610 2024-03-21T09:19:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8cbfea740 1 Processor -- start 2024-03-21T09:19:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8cbfea740 1 -- start start 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074351670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x55807435cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8baa00640 1 -- 172.21.15.37:0/224479921 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa8b4004950 con 0x7fa8c4100610 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.299+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580742b5eb0 con 0x558074351290 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807425ea60 con 0x558074351bb0 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580742b1f50 con 0x55807435a790 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x55807435cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x55807435cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58728/0 (socket says 172.21.15.37:58728) 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 -- 172.21.15.37:0/699644329 learned_addr learned my addr 172.21.15.37:0/699644329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c2800640 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074351670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 -- 172.21.15.37:0/699644329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 msgr2=0x55807435a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 -- 172.21.15.37:0/699644329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 msgr2=0x558074351670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074351670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.300+0000 7fa8c1400640 1 -- 172.21.15.37:0/699644329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580740927e0 con 0x55807435a790 2024-03-21T09:19:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8c1400640 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x55807435cb80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0036040 tx=0x7fa8b0009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e725b7f31fbeef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8af400640 1 -- 172.21.15.37:0/699644329 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0042070 con 0x55807435a790 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8af400640 1 -- 172.21.15.37:0/699644329 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b0004c70 con 0x55807435a790 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8af400640 1 -- 172.21.15.37:0/699644329 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0005320 con 0x55807435a790 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8baa00640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa8a80a3140 0x7fa8a80a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8baa00640 1 -- 172.21.15.37:0/224479921 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa8b40cf400 con 0x7fa8c4100610 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8cbfea740 1 -- 172.21.15.37:0/699644329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 msgr2=0x55807435cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.301+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x55807435cb80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0036040 tx=0x7fa8b0009610 comp rx=0 tx=0).stop 2024-03-21T09:19:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8c2800640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa8a80a3140 0x7fa8a80a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 -- 172.21.15.37:0/699644329 shutdown_connections 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x55807435cb80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/699644329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074351670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 -- 172.21.15.37:0/699644329 >> 172.21.15.37:0/699644329 conn(0x55807434b9c0 msgr2=0x55807434bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 -- 172.21.15.37:0/699644329 shutdown_connections 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 -- 172.21.15.37:0/699644329 wait complete. 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 -- start start 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8c2800640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa8a80a3140 0x7fa8a80a5600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc005fd0 tx=0x7fa8bc009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.302+0000 7fa8cbfea740 1 Processor -- start 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 0x55807435a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558074351bb0 0x558074357470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807435a790 0x5580743579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807435d520 con 0x55807435a790 2024-03-21T09:19:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558074271eb0 con 0x558074351290 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807435d6a0 con 0x558074351bb0 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8cbfea740 10 client.0 _ll_register_callbacks cb 0x558074266150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 0x55807435a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558074351bb0 0x558074357470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 0x55807435a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47530/0 (socket says 172.21.15.37:47530) 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8c1e00640 1 -- 172.21.15.37:0/123476701 learned_addr learned my addr 172.21.15.37:0/123476701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.303+0000 7fa8c2800640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807435a790 0x5580743579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1e00640 1 -- 172.21.15.37:0/123476701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558074351bb0 msgr2=0x558074357470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558074351bb0 0x558074357470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1e00640 1 -- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807435a790 msgr2=0x5580743579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807435a790 0x5580743579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1e00640 1 -- 172.21.15.37:0/123476701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580740f2f70 con 0x558074351290 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1400640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558074351bb0 0x558074357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c2800640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807435a790 0x5580743579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:13.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 0x55807435a0e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa8b409b040 tx=0x7fa8b409b110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3286d8239bcd84f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8a3e00640 1 -- 172.21.15.37:0/123476701 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4020070 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8a3e00640 1 -- 172.21.15.37:0/123476701 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa8b40979b0 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580742c33f0 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8a3e00640 1 -- 172.21.15.37:0/123476701 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8b4009e20 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8cbfea740 10 client.27239 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.304+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558073f8a170 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8a3e00640 1 -- 172.21.15.37:0/123476701 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5580742c33f0 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8a3e00640 1 -- 172.21.15.37:0/123476701 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa8b40cc920 con 0x558074351290 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8cbfea740 10 client.27239 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8cbfea740 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8cbfea740 20 client.27239 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55807436a800 0x55807436cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8cbfea740 4 client.27239 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580742bd740 con 0x55807436a800 2024-03-21T09:19:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.305+0000 7fa8c1400640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55807436a800 0x55807436cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.306+0000 7fa8c1400640 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55807436a800 0x55807436cbf0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92d423bcffcf68b server_cookie=8d7aa7bea950691 in_seq=0 out_seq=0 2024-03-21T09:19:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.306+0000 7fa8a3e00640 10 client.27239 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8a3e00640 1 -- 172.21.15.37:0/123476701 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5580742bd740 con 0x55807436a800 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8a3e00640 10 client.27239 handle_command_reply: tid=0 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 client.27239 shutdown 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 msgr2=0x55807435a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 0x55807435a0e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa8b409b040 tx=0x7fa8b409b110 comp rx=0 tx=0).stop 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 shutdown_connections 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807435a790 0x5580743579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558074351bb0 0x558074357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55807436a800 0x55807436cbf0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/123476701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351290 0x55807435a0e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 >> 172.21.15.37:0/123476701 conn(0x55807434b9c0 msgr2=0x55807434d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.307+0000 7fa8a3e00640 0 client.27239 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.308+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 shutdown_connections 2024-03-21T09:19:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.308+0000 7fa8cbfea740 1 -- 172.21.15.37:0/123476701 wait complete. 2024-03-21T09:19:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.308+0000 7fa8cbfea740 20 client.27239 trim_cache size 0 max 16384 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 Processor -- start 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 -- start start 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074359ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074365de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41b3110 con 0x558074351290 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41b0c10 con 0x558074351bb0 2024-03-21T09:19:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41aecd0 con 0x55807435a790 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074359ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47538/0 (socket says 172.21.15.37:47538) 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8c1400640 1 -- 172.21.15.37:0/1966851548 learned_addr learned my addr 172.21.15.37:0/1966851548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.326+0000 7fa8c2800640 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074365de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1400640 1 -- 172.21.15.37:0/1966851548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 msgr2=0x558074365de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1400640 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074365de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1400640 1 -- 172.21.15.37:0/1966851548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 msgr2=0x558074359ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1400640 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074359ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1400640 1 -- 172.21.15.37:0/1966851548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55807434b0a0 con 0x558074351bb0 2024-03-21T09:19:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074359ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8c1400640 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a0e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0004630 tx=0x7fa8b0004fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e980e3a8f3c50b81 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8aea00640 1 -- 172.21.15.37:0/1966851548 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0056070 con 0x558074351bb0 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8aea00640 1 -- 172.21.15.37:0/1966851548 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b002c650 con 0x558074351bb0 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.327+0000 7fa8aea00640 1 -- 172.21.15.37:0/1966851548 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0042040 con 0x558074351bb0 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 -- 172.21.15.37:0/1966851548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 msgr2=0x55807435a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a0e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0004630 tx=0x7fa8b0004fc0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 -- 172.21.15.37:0/1966851548 shutdown_connections 2024-03-21T09:19:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074365de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435a0e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/1966851548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x558074359ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 -- 172.21.15.37:0/1966851548 >> 172.21.15.37:0/1966851548 conn(0x55807434b9c0 msgr2=0x55807434d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 -- 172.21.15.37:0/1966851548 shutdown_connections 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.328+0000 7fa8cbfea740 1 -- 172.21.15.37:0/1966851548 wait complete. 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 -- start start 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 Processor -- start 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x55807435f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074360370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41af2d0 con 0x558074351290 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41b05b0 con 0x558074351bb0 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41af8f0 con 0x55807435a790 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.329+0000 7fa8cbfea740 10 client.0 _ll_register_callbacks cb 0x7fa8b40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074360370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47546/0 (socket says 172.21.15.37:47546) 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 -- 172.21.15.37:0/793546521 learned_addr learned my addr 172.21.15.37:0/793546521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x55807435f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 -- 172.21.15.37:0/793546521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 msgr2=0x558074360370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074360370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 -- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 msgr2=0x55807435f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x55807435f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 -- 172.21.15.37:0/793546521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580743c60d0 con 0x558074351bb0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8c1400640 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435fe30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0000c00 tx=0x7fa8b002cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dcc2c44f7ae77f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.330+0000 7fa8a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8a3e00640 1 -- 172.21.15.37:0/793546521 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0056030 con 0x558074351bb0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55807423ba40 con 0x558074351bb0 2024-03-21T09:19:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8a3e00640 1 -- 172.21.15.37:0/793546521 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa8b0005030 con 0x558074351bb0 2024-03-21T09:19:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8a3e00640 1 -- 172.21.15.37:0/793546521 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8b0042040 con 0x558074351bb0 2024-03-21T09:19:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8cbfea740 10 client.27245 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580743c7410 con 0x558074351bb0 2024-03-21T09:19:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.331+0000 7fa8a3e00640 1 -- 172.21.15.37:0/793546521 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55807423ba40 con 0x558074351bb0 2024-03-21T09:19:13.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8a3e00640 1 -- 172.21.15.37:0/793546521 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa8b004e070 con 0x558074351bb0 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8cbfea740 10 client.27245 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8cbfea740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8cbfea740 20 client.27245 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580743c76f0 0x5580743c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8cbfea740 4 client.27245 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5580742878e0 con 0x5580743c76f0 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.332+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580743c76f0 0x5580743c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.333+0000 7fa8c1e00640 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580743c76f0 0x5580743c9ae0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b293d0d542ae219 server_cookie=60ee1484d9536778 in_seq=0 out_seq=0 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.333+0000 7fa8a3e00640 10 client.27245 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8a3e00640 1 -- 172.21.15.37:0/793546521 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580742878e0 con 0x5580743c76f0 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8a3e00640 10 client.27245 handle_command_reply: tid=0 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 client.27245 shutdown 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 msgr2=0x55807435fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435fe30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0000c00 tx=0x7fa8b002cc10 comp rx=0 tx=0).stop 2024-03-21T09:19:13.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 shutdown_connections 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55807435a790 0x558074360370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5580743c76f0 0x5580743c9ae0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558074351bb0 0x55807435fe30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 --2- 172.21.15.37:0/793546521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558074351290 0x55807435f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.334+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 >> 172.21.15.37:0/793546521 conn(0x55807434b9c0 msgr2=0x5580743520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.335+0000 7fa8a3e00640 0 client.27245 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.335+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 shutdown_connections 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.335+0000 7fa8cbfea740 1 -- 172.21.15.37:0/793546521 wait complete. 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.335+0000 7fa8cbfea740 20 client.27245 trim_cache size 0 max 16384 2024-03-21T09:19:13.333 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:19:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:19:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "a297735b-1988-4896-adb8-b350fcb72e45", 2024-03-21T09:19:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:19:13.334 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:19:13.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa8a80a3140 msgr2=0x7fa8a80a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa8a80a3140 0x7fa8a80a5600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc005fd0 tx=0x7fa8bc009500 comp rx=0 tx=0).stop 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100610 msgr2=0x7fa8c410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100610 0x7fa8c410dbf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400d6d0 tx=0x7fa8b400dad0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 shutdown_connections 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c410e670 0x7fa8c41a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa8a80a3140 0x7fa8a80a5600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8c4105690 0x7fa8c410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 --2- 172.21.15.37:0/224479921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c4100610 0x7fa8c410dbf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 >> 172.21.15.37:0/224479921 conn(0x7fa8c4077130 msgr2=0x7fa8c40fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 shutdown_connections 2024-03-21T09:19:13.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.336+0000 7fa8c3200640 1 -- 172.21.15.37:0/224479921 wait complete. 2024-03-21T09:19:13.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:19:13.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:19:13.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 -- 172.21.15.37:0/1457420380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 msgr2=0x7fc914103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 --2- 172.21.15.37:0/1457420380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 0x7fc914103cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc900009930 tx=0x7fc90002bc30 comp rx=0 tx=0).stop 2024-03-21T09:19:13.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 -- 172.21.15.37:0/1457420380 shutdown_connections 2024-03-21T09:19:13.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 --2- 172.21.15.37:0/1457420380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914104200 0x7fc914110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 --2- 172.21.15.37:0/1457420380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 0x7fc914103cc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 --2- 172.21.15.37:0/1457420380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914109850 0x7fc914109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 -- 172.21.15.37:0/1457420380 >> 172.21.15.37:0/1457420380 conn(0x7fc9140ff2f0 msgr2=0x7fc914101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.584+0000 7fc919e00640 1 -- 172.21.15.37:0/1457420380 shutdown_connections 2024-03-21T09:19:13.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.585+0000 7fc919e00640 1 -- 172.21.15.37:0/1457420380 wait complete. 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 Processor -- start 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 -- start start 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 0x7fc91419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 0x7fc91419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914109850 0x7fc91419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914113c60 con 0x7fc914103840 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914113ae0 con 0x7fc914109850 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc919e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914113de0 con 0x7fc914104200 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc912000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 0x7fc91419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc913400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914109850 0x7fc91419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc912000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 0x7fc91419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58762/0 (socket says 172.21.15.37:58762) 2024-03-21T09:19:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc912a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 0x7fc91419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.586+0000 7fc912000640 1 -- 172.21.15.37:0/1008088803 learned_addr learned my addr 172.21.15.37:0/1008088803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc912000640 1 -- 172.21.15.37:0/1008088803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914109850 msgr2=0x7fc91419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc912000640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914109850 0x7fc91419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc912000640 1 -- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 msgr2=0x7fc91419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc912000640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 0x7fc91419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc912000640 1 -- 172.21.15.37:0/1008088803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91419f190 con 0x7fc914104200 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc913400640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914109850 0x7fc91419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.587+0000 7fc912000640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 0x7fc91419a6e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc900009230 tx=0x7fc90002cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.588+0000 7fc918a00640 1 -- 172.21.15.37:0/1008088803 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900041070 con 0x7fc914104200 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.588+0000 7fc918a00640 1 -- 172.21.15.37:0/1008088803 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc900004450 con 0x7fc914104200 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.588+0000 7fc918a00640 1 -- 172.21.15.37:0/1008088803 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900003de0 con 0x7fc914104200 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.588+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91419f420 con 0x7fc914104200 2024-03-21T09:19:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.588+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9141a6d10 con 0x7fc914104200 2024-03-21T09:19:13.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.592+0000 7fc918a00640 1 -- 172.21.15.37:0/1008088803 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc900044070 con 0x7fc914104200 2024-03-21T09:19:13.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.592+0000 7fc91cbe2740 1 Processor -- start 2024-03-21T09:19:13.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.592+0000 7fc91cbe2740 1 -- start start 2024-03-21T09:19:13.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x56289240f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 0x5628924103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892419290 0x56289241b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562892374250 con 0x562892410c00 2024-03-21T09:19:13.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56289231ca30 con 0x56289240ff20 2024-03-21T09:19:13.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628923702f0 con 0x562892419290 2024-03-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 0x5628924103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892419290 0x56289241b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc913400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x56289240f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 0x5628924103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:47564/0 (socket says 172.21.15.37:47564) 2024-03-21T09:19:13.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 -- 172.21.15.37:0/2240848122 learned_addr learned my addr 172.21.15.37:0/2240848122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 -- 172.21.15.37:0/2240848122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892419290 msgr2=0x56289241b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892419290 0x56289241b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 -- 172.21.15.37:0/2240848122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 msgr2=0x56289240f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x56289240f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912a00640 1 -- 172.21.15.37:0/2240848122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562892307690 con 0x56289240ff20 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc912000640 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892419290 0x56289241b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:13.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.593+0000 7fc918a00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc8f80a3090 0x7fc8f80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.594+0000 7fc918a00640 1 -- 172.21.15.37:0/1008088803 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc9000f1500 con 0x7fc914104200 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.594+0000 7fc912a00640 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 0x5628924103a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc904009ca0 tx=0x7fc904011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba6b0e492b448a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.594+0000 7fc913400640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc8f80a3090 0x7fc8f80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.594+0000 7fc8ffe00640 1 -- 172.21.15.37:0/2240848122 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90401e070 con 0x56289240ff20 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.594+0000 7fc8ffe00640 1 -- 172.21.15.37:0/2240848122 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc904019b10 con 0x56289240ff20 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.594+0000 7fc8ffe00640 1 -- 172.21.15.37:0/2240848122 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90400eae0 con 0x56289240ff20 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc913400640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc8f80a3090 0x7fc8f80a5550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005fd0 tx=0x7fc90c009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 -- 172.21.15.37:0/2240848122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 msgr2=0x5628924103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 0x5628924103a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc904009ca0 tx=0x7fc904011810 comp rx=0 tx=0).stop 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 -- 172.21.15.37:0/2240848122 shutdown_connections 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892419290 0x56289241b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56289240ff20 0x5628924103a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/2240848122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x56289240f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 -- 172.21.15.37:0/2240848122 >> 172.21.15.37:0/2240848122 conn(0x5628923d7200 msgr2=0x56289240a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 -- 172.21.15.37:0/2240848122 shutdown_connections 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.595+0000 7fc91cbe2740 1 -- 172.21.15.37:0/2240848122 wait complete. 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.596+0000 7fc91cbe2740 1 -- start start 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.596+0000 7fc91cbe2740 1 Processor -- start 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892418960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892415cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892416230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56289241c100 con 0x562892410c00 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56289232fe80 con 0x562892419290 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56289241c280 con 0x56289240ff20 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc91cbe2740 10 client.0 _ll_register_callbacks cb 0x562892342130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc912a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892418960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc912a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892418960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58790/0 (socket says 172.21.15.37:58790) 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc912a00640 1 -- 172.21.15.37:0/411294320 learned_addr learned my addr 172.21.15.37:0/411294320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc913400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892416230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc912000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892415cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.597+0000 7fc912a00640 1 -- 172.21.15.37:0/411294320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 msgr2=0x562892416230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc912a00640 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892416230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc912a00640 1 -- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 msgr2=0x562892415cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc912a00640 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892415cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc912a00640 1 -- 172.21.15.37:0/411294320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628921527d0 con 0x56289240ff20 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc912000640 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892415cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc912a00640 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892418960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc904009870 tx=0x7fc9040098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c673924d01490d4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc8fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc8fcc00640 1 -- 172.21.15.37:0/411294320 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904026030 con 0x56289240ff20 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc8fcc00640 1 -- 172.21.15.37:0/411294320 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc904019640 con 0x56289240ff20 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.598+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc904019b10 con 0x56289240ff20 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc8fcc00640 1 -- 172.21.15.37:0/411294320 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc904030050 con 0x56289240ff20 2024-03-21T09:19:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 10 client.27283 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5628920883f0 con 0x56289240ff20 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc8fcc00640 1 -- 172.21.15.37:0/411294320 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc904019b10 con 0x56289240ff20 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc8fcc00640 1 -- 172.21.15.37:0/411294320 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc90400e7e0 con 0x56289240ff20 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 10 client.27283 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 20 client.27283 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562892429100 0x56289242b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 4 client.27283 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.599+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562892381790 con 0x562892429100 2024-03-21T09:19:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.600+0000 7fc912000640 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562892429100 0x56289242b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.600+0000 7fc912000640 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562892429100 0x56289242b4f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31a90963dae4872 server_cookie=7558c116d6785643 in_seq=0 out_seq=0 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.600+0000 7fc8fcc00640 10 client.27283 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc8fcc00640 1 -- 172.21.15.37:0/411294320 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562892381790 con 0x562892429100 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc8fcc00640 10 client.27283 handle_command_reply: tid=0 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 client.27283 shutdown 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 msgr2=0x562892418960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892418960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc904009870 tx=0x7fc9040098d0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 shutdown_connections 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562892429100 0x56289242b4f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892416230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892415cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc8fcc00640 0 client.27283 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/411294320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892418960 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 >> 172.21.15.37:0/411294320 conn(0x5628923d7200 msgr2=0x56289240c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 shutdown_connections 2024-03-21T09:19:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.601+0000 7fc91cbe2740 1 -- 172.21.15.37:0/411294320 wait complete. 2024-03-21T09:19:13.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.602+0000 7fc91cbe2740 20 client.27283 trim_cache size 0 max 16384 2024-03-21T09:19:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 Processor -- start 2024-03-21T09:19:13.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 -- start start 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56289240ff20 0x56289241a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 0x56289241abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892418420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141b3360 con 0x56289240ff20 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141b0e60 con 0x562892419290 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.625+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141aef20 con 0x562892410c00 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 0x56289241abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc913400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892418420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 0x56289241abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58794/0 (socket says 172.21.15.37:58794) 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 -- 172.21.15.37:0/1293026950 learned_addr learned my addr 172.21.15.37:0/1293026950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912a00640 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56289240ff20 0x56289241a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 -- 172.21.15.37:0/1293026950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 msgr2=0x562892418420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892418420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 -- 172.21.15.37:0/1293026950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56289240ff20 msgr2=0x56289241a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56289240ff20 0x56289241a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 -- 172.21.15.37:0/1293026950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628920480f0 con 0x562892410c00 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.626+0000 7fc912000640 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 0x56289241abc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc90002cfd0 tx=0x7fc9000f3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23445d882da048c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc8ff400640 1 -- 172.21.15.37:0/1293026950 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90004b040 con 0x562892410c00 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc8ff400640 1 -- 172.21.15.37:0/1293026950 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9000bdc50 con 0x562892410c00 2024-03-21T09:19:13.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc8ff400640 1 -- 172.21.15.37:0/1293026950 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000bd550 con 0x562892410c00 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 -- 172.21.15.37:0/1293026950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 msgr2=0x56289241abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 0x56289241abc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc90002cfd0 tx=0x7fc9000f3040 comp rx=0 tx=0).stop 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 -- 172.21.15.37:0/1293026950 shutdown_connections 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892418420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562892410c00 0x56289241abc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/1293026950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56289240ff20 0x56289241a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.627+0000 7fc91cbe2740 1 -- 172.21.15.37:0/1293026950 >> 172.21.15.37:0/1293026950 conn(0x5628923d7200 msgr2=0x56289240a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.628+0000 7fc91cbe2740 1 -- 172.21.15.37:0/1293026950 shutdown_connections 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.628+0000 7fc91cbe2740 1 -- 172.21.15.37:0/1293026950 wait complete. 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.628+0000 7fc91cbe2740 1 -- start start 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.628+0000 7fc91cbe2740 1 Processor -- start 2024-03-21T09:19:13.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892412560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892412aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892412fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141af520 con 0x562892410c00 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141b0800 con 0x562892419290 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141afb40 con 0x56289240ff20 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc91cbe2740 10 client.0 _ll_register_callbacks cb 0x7fc90401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc912a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892412560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc912a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892412560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58810/0 (socket says 172.21.15.37:58810) 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc912a00640 1 -- 172.21.15.37:0/3399759984 learned_addr learned my addr 172.21.15.37:0/3399759984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc913400640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892412fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.629+0000 7fc912000640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892412aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc912a00640 1 -- 172.21.15.37:0/3399759984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 msgr2=0x562892412fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc912a00640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892412fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc912a00640 1 -- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 msgr2=0x562892412aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:13.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc912a00640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892412aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc912a00640 1 -- 172.21.15.37:0/3399759984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56289248c110 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc912a00640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892412560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc904009250 tx=0x7fc904011eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46440cada229c153 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc8fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc8fcc00640 1 -- 172.21.15.37:0/3399759984 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904026070 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9000bdc50 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.630+0000 7fc8fcc00640 1 -- 172.21.15.37:0/3399759984 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc904005310 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc8fcc00640 1 -- 172.21.15.37:0/3399759984 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90401e040 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 10 client.27289 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56289248d400 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc8fcc00640 1 -- 172.21.15.37:0/3399759984 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9000bdc50 con 0x56289240ff20 2024-03-21T09:19:13.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc8fcc00640 1 -- 172.21.15.37:0/3399759984 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc904019820 con 0x56289240ff20 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 10 client.27289 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 20 client.27289 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56289248da90 0x56289248feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 4 client.27289 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56289237bae0 con 0x56289248da90 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.631+0000 7fc912000640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56289248da90 0x56289248feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.632+0000 7fc912000640 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56289248da90 0x56289248feb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4ca58e9ae18013 server_cookie=cbde44c52b74118d in_seq=0 out_seq=0 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.632+0000 7fc8fcc00640 10 client.27289 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.632+0000 7fc8fcc00640 1 -- 172.21.15.37:0/3399759984 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56289237bae0 con 0x56289248da90 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.632+0000 7fc8fcc00640 10 client.27289 handle_command_reply: tid=0 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.632+0000 7fc91cbe2740 1 client.27289 shutdown 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 msgr2=0x562892412560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892412560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc904009250 tx=0x7fc904011eb0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 shutdown_connections 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56289248da90 0x56289248feb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562892419290 0x562892412fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562892410c00 0x562892412aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 --2- 172.21.15.37:0/3399759984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56289240ff20 0x562892412560 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 >> 172.21.15.37:0/3399759984 conn(0x5628923d7200 msgr2=0x562892414520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc8fcc00640 0 client.27289 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 shutdown_connections 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 1 -- 172.21.15.37:0/3399759984 wait complete. 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.633+0000 7fc91cbe2740 20 client.27289 trim_cache size 0 max 16384 2024-03-21T09:19:13.632 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "a297735b-1988-4896-adb8-b350fcb72e45": { 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "a297735b-1988-4896-adb8-b350fcb72e45", 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:19:13.633 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc8f80a3090 msgr2=0x7fc8f80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc8f80a3090 0x7fc8f80a5550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005fd0 tx=0x7fc90c009500 comp rx=0 tx=0).stop 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 msgr2=0x7fc91419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 0x7fc91419a6e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc900009230 tx=0x7fc90002cb30 comp rx=0 tx=0).stop 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 shutdown_connections 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc8f80a3090 0x7fc8f80a5550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc914109850 0x7fc91419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc914104200 0x7fc91419a6e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 --2- 172.21.15.37:0/1008088803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914103840 0x7fc91419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 >> 172.21.15.37:0/1008088803 conn(0x7fc9140ff2f0 msgr2=0x7fc9140ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 shutdown_connections 2024-03-21T09:19:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:13.634+0000 7fc919e00640 1 -- 172.21.15.37:0/1008088803 wait complete. 2024-03-21T09:19:13.647 INFO:tasks.cephfs.filesystem:scrub status for tag:a297735b-1988-4896-adb8-b350fcb72e45 - {'path': '/', 'tag': 'a297735b-1988-4896-adb8-b350fcb72e45', 'options': 'recursive,force'} 2024-03-21T09:19:14.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:14 smithi071 ceph-mon[36597]: pgmap v215: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:19:14.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:14 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:19:14.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:14 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:14.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:14 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:19:14.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:14 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:14.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:14 smithi037 ceph-mon[31830]: pgmap v215: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:19:14.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:14 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:19:14.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:14 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:14.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:14 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:19:14.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:14 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:14.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:14 smithi053 ceph-mon[36169]: pgmap v215: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:19:14.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:14 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:19:14.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:14 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:14.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:14 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:19:14.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:14 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:16.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:16 smithi071 ceph-mon[36597]: pgmap v216: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:19:16.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:16 smithi071 ceph-mon[36597]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T09:19:16.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:16 smithi037 ceph-mon[31830]: pgmap v216: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:19:16.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:16 smithi037 ceph-mon[31830]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T09:19:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:16 smithi053 ceph-mon[36169]: pgmap v216: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:19:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:16 smithi053 ceph-mon[36169]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T09:19:17.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:17 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:19:17.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:17 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:19:17.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:17 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:19:18.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:18 smithi037 ceph-mon[31830]: pgmap v218: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:18.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:18 smithi071 ceph-mon[36597]: pgmap v218: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:18.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:18 smithi053 ceph-mon[36169]: pgmap v218: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:19.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:19 smithi071 ceph-mon[36597]: pgmap v219: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:19.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:19 smithi037 ceph-mon[31830]: pgmap v219: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:19.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:19 smithi053 ceph-mon[36169]: pgmap v219: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:22.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:22 smithi071 ceph-mon[36597]: pgmap v220: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:22.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:22 smithi037 ceph-mon[31830]: pgmap v220: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:22.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:22 smithi053 ceph-mon[36169]: pgmap v220: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-03-21T09:19:24.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:24 smithi071 ceph-mon[36597]: pgmap v221: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-03-21T09:19:24.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:24 smithi037 ceph-mon[31830]: pgmap v221: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-03-21T09:19:24.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:24 smithi053 ceph-mon[36169]: pgmap v221: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-03-21T09:19:26.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:26 smithi071 ceph-mon[36597]: pgmap v222: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-03-21T09:19:26.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:26 smithi037 ceph-mon[31830]: pgmap v222: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-03-21T09:19:26.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:26 smithi053 ceph-mon[36169]: pgmap v222: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-03-21T09:19:28.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:28 smithi071 ceph-mon[36597]: pgmap v223: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T09:19:28.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:28 smithi037 ceph-mon[31830]: pgmap v223: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T09:19:28.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:28 smithi053 ceph-mon[36169]: pgmap v223: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T09:19:30.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:30 smithi071 ceph-mon[36597]: pgmap v224: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:30.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:30 smithi037 ceph-mon[31830]: pgmap v224: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:30.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:30 smithi053 ceph-mon[36169]: pgmap v224: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:31.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:31 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:19:31.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:31 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:19:31.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:31 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:19:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:32 smithi071 ceph-mon[36597]: pgmap v225: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:32 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:19:32.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:32 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:19:32.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:32 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:19:32.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:32 smithi037 ceph-mon[31830]: pgmap v225: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:32.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:32 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:19:32.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:32 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:19:32.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:32 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:19:32.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:32 smithi053 ceph-mon[36169]: pgmap v225: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:32.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:32 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:19:32.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:32 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:19:32.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:32 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:19:34.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:34 smithi071 ceph-mon[36597]: pgmap v226: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:34.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:34 smithi037 ceph-mon[31830]: pgmap v226: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:34.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:34 smithi053 ceph-mon[36169]: pgmap v226: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:36.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:36 smithi071 ceph-mon[36597]: pgmap v227: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:36.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:36 smithi037 ceph-mon[31830]: pgmap v227: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:36.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:36 smithi053 ceph-mon[36169]: pgmap v227: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:38.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:38 smithi071 ceph-mon[36597]: pgmap v228: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:38.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:38 smithi037 ceph-mon[31830]: pgmap v228: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:38.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:38 smithi053 ceph-mon[36169]: pgmap v228: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:39.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:39 smithi037 ceph-mon[31830]: pgmap v229: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:39.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:39 smithi053 ceph-mon[36169]: pgmap v229: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:39.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:39 smithi071 ceph-mon[36597]: pgmap v229: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:42.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:42 smithi071 ceph-mon[36597]: pgmap v230: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:42.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:42 smithi037 ceph-mon[31830]: pgmap v230: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:42.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:42 smithi053 ceph-mon[36169]: pgmap v230: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:43.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:19:43.648 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:19:43.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 -- 172.21.15.37:0/381367910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc4103690 msgr2=0x7f0cc4103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 --2- 172.21.15.37:0/381367910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc4103690 0x7f0cc4103b10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009930 tx=0x7f0cb002bc20 comp rx=0 tx=0).stop 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 -- 172.21.15.37:0/381367910 shutdown_connections 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 --2- 172.21.15.37:0/381367910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc4104050 0x7f0cc4110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 --2- 172.21.15.37:0/381367910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc4103690 0x7f0cc4103b10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 --2- 172.21.15.37:0/381367910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc41096a0 0x7f0cc4109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.891+0000 7f0cc9200640 1 -- 172.21.15.37:0/381367910 >> 172.21.15.37:0/381367910 conn(0x7f0cc40ff140 msgr2=0x7f0cc4101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.892+0000 7f0cc9200640 1 -- 172.21.15.37:0/381367910 shutdown_connections 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.892+0000 7f0cc9200640 1 -- 172.21.15.37:0/381367910 wait complete. 2024-03-21T09:19:43.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.893+0000 7f0cc9200640 1 Processor -- start 2024-03-21T09:19:43.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.893+0000 7f0cc9200640 1 -- start start 2024-03-21T09:19:43.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.893+0000 7f0cc9200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 0x7f0cc4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc4104050 0x7f0cc419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 0x7f0cc419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4113b00 con 0x7f0cc4104050 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc9200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4113980 con 0x7f0cc4103690 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc9200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4113c80 con 0x7f0cc41096a0 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 0x7f0cc419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 0x7f0cc4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 0x7f0cc4199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35936/0 (socket says 172.21.15.37:35936) 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 0x7f0cc419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55242/0 (socket says 172.21.15.37:55242) 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 -- 172.21.15.37:0/1399621102 learned_addr learned my addr 172.21.15.37:0/1399621102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:43.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc4104050 0x7f0cc419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 -- 172.21.15.37:0/1399621102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 msgr2=0x7f0cc419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 0x7f0cc419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 -- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc4104050 msgr2=0x7f0cc419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc4104050 0x7f0cc419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3400640 1 -- 172.21.15.37:0/1399621102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc419efb0 con 0x7f0cc4103690 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.894+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 0x7f0cc419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.895+0000 7f0cc3400640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 0x7f0cc4199fc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9f0 tx=0x7f0cb400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.895+0000 7f0cbbe00640 1 -- 172.21.15.37:0/1399621102 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400cd80 con 0x7f0cc4103690 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.895+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc419f2a0 con 0x7f0cc4103690 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.895+0000 7f0cbbe00640 1 -- 172.21.15.37:0/1399621102 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cb4005500 con 0x7f0cc4103690 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.895+0000 7f0cbbe00640 1 -- 172.21.15.37:0/1399621102 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4010690 con 0x7f0cc4103690 2024-03-21T09:19:43.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.895+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41a6b30 con 0x7f0cc4103690 2024-03-21T09:19:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.898+0000 7f0ccd3f2740 1 Processor -- start 2024-03-21T09:19:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.898+0000 7f0cbbe00640 1 -- 172.21.15.37:0/1399621102 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f0cb40046c0 con 0x7f0cc4103690 2024-03-21T09:19:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.898+0000 7f0ccd3f2740 1 -- start start 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 0x55c995377b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995378570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995381460 0x55c995383850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9952dc250 con 0x55c9953780f0 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995284a30 con 0x55c995381460 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9952d82f0 con 0x55c995378dd0 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 0x55c995377b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 0x55c995377b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55246/0 (socket says 172.21.15.37:55246) 2024-03-21T09:19:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995378570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 -- 172.21.15.37:0/496442158 learned_addr learned my addr 172.21.15.37:0/496442158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995381460 0x55c995383850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 -- 172.21.15.37:0/496442158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995381460 msgr2=0x55c995383850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995381460 0x55c995383850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 -- 172.21.15.37:0/496442158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 msgr2=0x55c995378570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995378570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.899+0000 7f0cc3e00640 1 -- 172.21.15.37:0/496442158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c99526f690 con 0x55c995378dd0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 0x55c995377b20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc010750 tx=0x7f0cbc010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c0f921b5452237 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cbbe00640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0ca80a3090 0x7f0ca80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cb8c00640 1 -- 172.21.15.37:0/496442158 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00ead0 con 0x55c995378dd0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cb8c00640 1 -- 172.21.15.37:0/496442158 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc010e60 con 0x55c995378dd0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cb8c00640 1 -- 172.21.15.37:0/496442158 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc011640 con 0x55c995378dd0 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cbbe00640 1 -- 172.21.15.37:0/1399621102 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0cb40cffc0 con 0x7f0cc4103690 2024-03-21T09:19:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.900+0000 7f0cc3400640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0ca80a3090 0x7f0ca80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/496442158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 msgr2=0x55c995377b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0cc3400640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0ca80a3090 0x7f0ca80a5550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55c995379320 tx=0x7f0cb4004a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 0x55c995377b20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc010750 tx=0x7f0cbc010c10 comp rx=0 tx=0).stop 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/496442158 shutdown_connections 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995381460 0x55c995383850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995378570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/496442158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995378dd0 0x55c995377b20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/496442158 >> 172.21.15.37:0/496442158 conn(0x55c99533f200 msgr2=0x55c995372b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.901+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/496442158 shutdown_connections 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.902+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/496442158 wait complete. 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.902+0000 7f0ccd3f2740 1 -- start start 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.902+0000 7f0ccd3f2740 1 Processor -- start 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995380b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c99537e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9953842d0 con 0x55c9953780f0 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995297e80 con 0x55c995378dd0 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995384450 con 0x55c995381460 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0ccd3f2740 10 client.0 _ll_register_callbacks cb 0x55c9952aa130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0cc2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0cc2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35966/0 (socket says 172.21.15.37:35966) 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0cc2a00640 1 -- 172.21.15.37:0/3081916176 learned_addr learned my addr 172.21.15.37:0/3081916176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995380b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.903+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c99537e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc2a00640 1 -- 172.21.15.37:0/3081916176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 msgr2=0x55c99537e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c99537e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc2a00640 1 -- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 msgr2=0x55c995380b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995380b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc2a00640 1 -- 172.21.15.37:0/3081916176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9950ba7d0 con 0x55c995378dd0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c99537e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537ded0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0cb002c230 tx=0x7f0cb002c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e4e681142275ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cad600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cad600640 1 -- 172.21.15.37:0/3081916176 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb002cb40 con 0x55c995378dd0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cbc010e60 con 0x55c995378dd0 2024-03-21T09:19:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.904+0000 7f0cad600640 1 -- 172.21.15.37:0/3081916176 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0cb00046c0 con 0x55c995378dd0 2024-03-21T09:19:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0cad600640 1 -- 172.21.15.37:0/3081916176 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cb002cce0 con 0x55c995378dd0 2024-03-21T09:19:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 10 client.27275 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c994ff03f0 con 0x55c995378dd0 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0cad600640 1 -- 172.21.15.37:0/3081916176 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc010e60 con 0x55c995378dd0 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0cad600640 1 -- 172.21.15.37:0/3081916176 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0cb00035f0 con 0x55c995378dd0 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 10 client.27275 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 10 client.27275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 20 client.27275 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c995391310 0x55c995393700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 4 client.27275 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.905+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9952e9790 con 0x55c995391310 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.906+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c995391310 0x55c995393700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.906+0000 7f0cad600640 10 client.27275 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.906+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c995391310 0x55c995393700 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79116d5569fe1fa5 server_cookie=2c1145e413c29068 in_seq=0 out_seq=0 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.907+0000 7f0cad600640 1 -- 172.21.15.37:0/3081916176 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c9952e9790 con 0x55c995391310 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.907+0000 7f0cad600640 10 client.27275 handle_command_reply: tid=0 2024-03-21T09:19:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.907+0000 7f0ccd3f2740 1 client.27275 shutdown 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 msgr2=0x55c99537ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537ded0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0cb002c230 tx=0x7f0cb002c7d0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 shutdown_connections 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c99537e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c995391310 0x55c995393700 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537ded0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3081916176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9953780f0 0x55c995380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 >> 172.21.15.37:0/3081916176 conn(0x55c99533f200 msgr2=0x55c995374890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0cad600640 0 client.27275 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 shutdown_connections 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3081916176 wait complete. 2024-03-21T09:19:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.908+0000 7f0ccd3f2740 20 client.27275 trim_cache size 0 max 16384 2024-03-21T09:19:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 Processor -- start 2024-03-21T09:19:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 -- start start 2024-03-21T09:19:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 0x55c995380600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995378dd0 0x55c995380b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c995382850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41b3130 con 0x55c995378dd0 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41b0c30 con 0x55c9953780f0 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.926+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41aecf0 con 0x55c995381460 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 0x55c995380600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 0x55c995380600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35974/0 (socket says 172.21.15.37:35974) 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 -- 172.21.15.37:0/3363153138 learned_addr learned my addr 172.21.15.37:0/3363153138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995378dd0 0x55c995380b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c995382850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 -- 172.21.15.37:0/3363153138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 msgr2=0x55c995382850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c995382850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 -- 172.21.15.37:0/3363153138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995378dd0 msgr2=0x55c995380b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995378dd0 0x55c995380b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 -- 172.21.15.37:0/3363153138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c994fb00f0 con 0x55c9953780f0 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.927+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 0x55c995380600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc015040 tx=0x7f0cbc012dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6743376c3ecbfb62 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.928+0000 7f0cafe00640 1 -- 172.21.15.37:0/3363153138 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc025040 con 0x55c9953780f0 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.928+0000 7f0cafe00640 1 -- 172.21.15.37:0/3363153138 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc010e60 con 0x55c9953780f0 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.928+0000 7f0cafe00640 1 -- 172.21.15.37:0/3363153138 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00eb30 con 0x55c9953780f0 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3363153138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 msgr2=0x55c995380600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 0x55c995380600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc015040 tx=0x7f0cbc012dd0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3363153138 shutdown_connections 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c995381460 0x55c995382850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995378dd0 0x55c995380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/3363153138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9953780f0 0x55c995380600 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3363153138 >> 172.21.15.37:0/3363153138 conn(0x55c99533f200 msgr2=0x55c995372670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:43.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3363153138 shutdown_connections 2024-03-21T09:19:43.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/3363153138 wait complete. 2024-03-21T09:19:43.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.929+0000 7f0ccd3f2740 1 -- start start 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 Processor -- start 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9953780f0 0x55c99537a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995381460 0x55c99537b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41af2f0 con 0x55c995381460 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41b05d0 con 0x55c995378dd0 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41af910 con 0x55c9953780f0 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.930+0000 7f0ccd3f2740 10 client.0 _ll_register_callbacks cb 0x7f0cb0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9953780f0 0x55c99537a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9953780f0 0x55c99537a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55276/0 (socket says 172.21.15.37:55276) 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc3e00640 1 -- 172.21.15.37:0/2497332532 learned_addr learned my addr 172.21.15.37:0/2497332532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:35982/0 (socket says 172.21.15.37:35982) 2024-03-21T09:19:43.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995381460 0x55c99537b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 -- 172.21.15.37:0/2497332532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9953780f0 msgr2=0x55c99537a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9953780f0 0x55c99537a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 -- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995381460 msgr2=0x55c99537b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995381460 0x55c99537b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc2a00640 1 -- 172.21.15.37:0/2497332532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9953ebd60 con 0x55c995378dd0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.931+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995381460 0x55c99537b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0cc2a00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537acc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0005ab0 tx=0x7f0cb0004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f93008499d98b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0cad600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0cad600640 1 -- 172.21.15.37:0/2497332532 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0005d50 con 0x55c995378dd0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0cad600640 1 -- 172.21.15.37:0/2497332532 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0cb00048c0 con 0x55c995378dd0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cbc010e60 con 0x55c995378dd0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0cad600640 1 -- 172.21.15.37:0/2497332532 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cb0004ae0 con 0x55c995378dd0 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0ccd3f2740 10 client.27281 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:43.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c9953ed030 con 0x55c995378dd0 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.932+0000 7f0cad600640 1 -- 172.21.15.37:0/2497332532 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc010e60 con 0x55c995378dd0 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0cad600640 1 -- 172.21.15.37:0/2497332532 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0cb004b510 con 0x55c995378dd0 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0ccd3f2740 10 client.27281 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0ccd3f2740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0ccd3f2740 20 client.27281 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c9953ed7b0 0x55c9953efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0ccd3f2740 4 client.27281 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c9952e3ae0 con 0x55c9953ed7b0 2024-03-21T09:19:43.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.933+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c9953ed7b0 0x55c9953efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:43.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0cc3e00640 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c9953ed7b0 0x55c9953efbd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9651ae0f42020c server_cookie=ed994bb5a01188d1 in_seq=0 out_seq=0 2024-03-21T09:19:43.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0cad600640 10 client.27281 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:43.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0cad600640 1 -- 172.21.15.37:0/2497332532 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c9952e3ae0 con 0x55c9953ed7b0 2024-03-21T09:19:43.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0cad600640 10 client.27281 handle_command_reply: tid=0 2024-03-21T09:19:43.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 client.27281 shutdown 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 msgr2=0x55c99537acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537acc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0005ab0 tx=0x7f0cb0004610 comp rx=0 tx=0).stop 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 shutdown_connections 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c995381460 0x55c99537b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55c9953ed7b0 0x55c9953efbd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c995378dd0 0x55c99537acc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 --2- 172.21.15.37:0/2497332532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9953780f0 0x55c99537a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 >> 172.21.15.37:0/2497332532 conn(0x55c99533f200 msgr2=0x55c99537c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0cad600640 0 client.27281 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 shutdown_connections 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 1 -- 172.21.15.37:0/2497332532 wait complete. 2024-03-21T09:19:43.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.934+0000 7f0ccd3f2740 20 client.27281 trim_cache size 0 max 16384 2024-03-21T09:19:43.934 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:19:43.934 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:19:43.934 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:19:43.934 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0ca80a3090 msgr2=0x7f0ca80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0ca80a3090 0x7f0ca80a5550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55c995379320 tx=0x7f0cb4004a40 comp rx=0 tx=0).stop 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 msgr2=0x7f0cc4199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 0x7f0cc4199fc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9f0 tx=0x7f0cb400edf0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 shutdown_connections 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc41096a0 0x7f0cc419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc4104050 0x7f0cc419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f0ca80a3090 0x7f0ca80a5550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 --2- 172.21.15.37:0/1399621102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cc4103690 0x7f0cc4199fc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 >> 172.21.15.37:0/1399621102 conn(0x7f0cc40ff140 msgr2=0x7f0cc40ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 shutdown_connections 2024-03-21T09:19:43.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:43.935+0000 7f0cc9200640 1 -- 172.21.15.37:0/1399621102 wait complete. 2024-03-21T09:19:43.948 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:19:43.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:19:44.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 -- 172.21.15.37:0/3954202501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 msgr2=0x7fc964111ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 --2- 172.21.15.37:0/3954202501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc964111ff0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009900 tx=0x7fc94c02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 -- 172.21.15.37:0/3954202501 shutdown_connections 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 --2- 172.21.15.37:0/3954202501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 0x7fc964114920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 --2- 172.21.15.37:0/3954202501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc964111ff0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 --2- 172.21.15.37:0/3954202501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc964073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 -- 172.21.15.37:0/3954202501 >> 172.21.15.37:0/3954202501 conn(0x7fc9640fd070 msgr2=0x7fc9640ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 -- 172.21.15.37:0/3954202501 shutdown_connections 2024-03-21T09:19:44.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.154+0000 7fc969400640 1 -- 172.21.15.37:0/3954202501 wait complete. 2024-03-21T09:19:44.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.155+0000 7fc969400640 1 Processor -- start 2024-03-21T09:19:44.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.155+0000 7fc969400640 1 -- start start 2024-03-21T09:19:44.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc969400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc96419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc969400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc96419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc969400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 0x7fc9641a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc969400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964117f80 con 0x7fc964112530 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc969400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964117e00 con 0x7fc964074360 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc969400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964118100 con 0x7fc964073a40 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc963e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc96419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc95be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc96419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc963e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc96419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55300/0 (socket says 172.21.15.37:55300) 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc963e00640 1 -- 172.21.15.37:0/3276405565 learned_addr learned my addr 172.21.15.37:0/3276405565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc968a00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 0x7fc9641a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc95be00640 1 -- 172.21.15.37:0/3276405565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 msgr2=0x7fc96419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc95be00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc96419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc95be00640 1 -- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 msgr2=0x7fc9641a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.156+0000 7fc95be00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 0x7fc9641a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc95be00640 1 -- 172.21.15.37:0/3276405565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9641a33e0 con 0x7fc964074360 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc963e00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc96419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc968a00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 0x7fc9641a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc95be00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc96419e980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02c200 tx=0x7fc94c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c042020 con 0x7fc964074360 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc94c005000 con 0x7fc964074360 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9641a3670 con 0x7fc964074360 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c003b00 con 0x7fc964074360 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.157+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9641aaf60 con 0x7fc964074360 2024-03-21T09:19:44.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.159+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc964075180 con 0x7fc964074360 2024-03-21T09:19:44.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.160+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc94c045020 con 0x7fc964074360 2024-03-21T09:19:44.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.163+0000 7fc961600640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc9480a3080 0x7fc9480a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.163+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc94c0f1740 con 0x7fc964074360 2024-03-21T09:19:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.163+0000 7fc963e00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc9480a3080 0x7fc9480a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.164+0000 7fc963e00640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc9480a3080 0x7fc9480a5540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc954011550 tx=0x7fc954005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.166+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc94c04c020 con 0x7fc964074360 2024-03-21T09:19:44.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.415+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9641ab130 con 0x7fc964074360 2024-03-21T09:19:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.417+0000 7fc961600640 1 -- 172.21.15.37:0/3276405565 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fc94c0bdee0 con 0x7fc964074360 2024-03-21T09:19:44.413 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:19:44.413 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:19:44.413 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:19:44.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.420+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc9480a3080 msgr2=0x7fc9480a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.420+0000 7fc969400640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc9480a3080 0x7fc9480a5540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc954011550 tx=0x7fc954005f90 comp rx=0 tx=0).stop 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.420+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 msgr2=0x7fc96419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.420+0000 7fc969400640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc96419e980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02c200 tx=0x7fc94c004360 comp rx=0 tx=0).stop 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 shutdown_connections 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc964112530 0x7fc9641a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc9480a3080 0x7fc9480a5540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc964074360 0x7fc96419e980 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 --2- 172.21.15.37:0/3276405565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964073a40 0x7fc96419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 >> 172.21.15.37:0/3276405565 conn(0x7fc9640fd070 msgr2=0x7fc9640fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 shutdown_connections 2024-03-21T09:19:44.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.421+0000 7fc969400640 1 -- 172.21.15.37:0/3276405565 wait complete. 2024-03-21T09:19:44.445 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:19:44.445 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:19:44.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:44 smithi071 ceph-mon[36597]: pgmap v231: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:44.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:44 smithi037 ceph-mon[31830]: pgmap v231: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:44.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:44 smithi053 ceph-mon[36169]: pgmap v231: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:19:44.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 -- 172.21.15.37:0/1206834526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101ff0 msgr2=0x7f92f810e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 --2- 172.21.15.37:0/1206834526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101ff0 0x7f92f810e500 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009c10 tx=0x7f92f002bef0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 -- 172.21.15.37:0/1206834526 shutdown_connections 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 --2- 172.21.15.37:0/1206834526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101ff0 0x7f92f810e500 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 --2- 172.21.15.37:0/1206834526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101630 0x7f92f8101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 --2- 172.21.15.37:0/1206834526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92f8107640 0x7f92f8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.709+0000 7f92fca00640 1 -- 172.21.15.37:0/1206834526 >> 172.21.15.37:0/1206834526 conn(0x7f92f80fd140 msgr2=0x7f92f80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.710+0000 7f92fca00640 1 -- 172.21.15.37:0/1206834526 shutdown_connections 2024-03-21T09:19:44.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.710+0000 7f92fca00640 1 -- 172.21.15.37:0/1206834526 wait complete. 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 Processor -- start 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 -- start start 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 0x7f92f819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101ff0 0x7f92f819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92f8107640 0x7f92f819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8111a50 con 0x7f92f8107640 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81118d0 con 0x7f92f8101ff0 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92fca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8111bd0 con 0x7f92f8101630 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92f5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101ff0 0x7f92f819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 0x7f92f819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 0x7f92f819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55320/0 (socket says 172.21.15.37:55320) 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92f5400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101ff0 0x7f92f819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36028/0 (socket says 172.21.15.37:36028) 2024-03-21T09:19:44.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92f5e00640 1 -- 172.21.15.37:0/211988606 learned_addr learned my addr 172.21.15.37:0/211988606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.711+0000 7f92f6800640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92f8107640 0x7f92f819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.712+0000 7f92f5e00640 1 -- 172.21.15.37:0/211988606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101ff0 msgr2=0x7f92f819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.712+0000 7f92f5e00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101ff0 0x7f92f819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.712+0000 7f92f5e00640 1 -- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92f8107640 msgr2=0x7f92f819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.712+0000 7f92f5e00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92f8107640 0x7f92f819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.712+0000 7f92f5e00640 1 -- 172.21.15.37:0/211988606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f819f080 con 0x7f92f8101630 2024-03-21T09:19:44.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.712+0000 7f92f5e00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 0x7f92f819a090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f92e800e9f0 tx=0x7f92e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.713+0000 7f92eea00640 1 -- 172.21.15.37:0/211988606 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e800cd80 con 0x7f92f8101630 2024-03-21T09:19:44.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.713+0000 7f92eea00640 1 -- 172.21.15.37:0/211988606 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92e8010910 con 0x7f92f8101630 2024-03-21T09:19:44.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.713+0000 7f92eea00640 1 -- 172.21.15.37:0/211988606 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8018990 con 0x7f92f8101630 2024-03-21T09:19:44.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.713+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f819f370 con 0x7f92f8101630 2024-03-21T09:19:44.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.713+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81a6c00 con 0x7f92f8101630 2024-03-21T09:19:44.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.716+0000 7f92ff781740 1 Processor -- start 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.716+0000 7f92ff781740 1 -- start start 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.716+0000 7f92ff781740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x56232660aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.716+0000 7f92ff781740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326613630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92ff781740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326616050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92ff781740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232656e2c0 con 0x56232660b080 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92ff781740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562326516c90 con 0x562326613c00 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92ff781740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232656a360 con 0x56232660a6d0 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x56232660aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x56232660aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55322/0 (socket says 172.21.15.37:55322) 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 -- 172.21.15.37:0/1152598607 learned_addr learned my addr 172.21.15.37:0/1152598607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f5400640 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326616050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 -- 172.21.15.37:0/1152598607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 msgr2=0x562326616050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326616050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 -- 172.21.15.37:0/1152598607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 msgr2=0x562326613630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:19:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326613630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 -- 172.21.15.37:0/1152598607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56232653b080 con 0x56232660a6d0 2024-03-21T09:19:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.717+0000 7f92f6800640 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x56232660aab0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009c10 tx=0x7f92f0005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595d09adf0da388f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92e3000640 1 -- 172.21.15.37:0/1152598607 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0043030 con 0x56232660a6d0 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92e3000640 1 -- 172.21.15.37:0/1152598607 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f0033c50 con 0x56232660a6d0 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92eea00640 1 -- 172.21.15.37:0/211988606 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f92e8010ac0 con 0x7f92f8101630 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92e3000640 1 -- 172.21.15.37:0/1152598607 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f002c890 con 0x56232660a6d0 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 -- 172.21.15.37:0/1152598607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 msgr2=0x56232660aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x56232660aab0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009c10 tx=0x7f92f0005bd0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 -- 172.21.15.37:0/1152598607 shutdown_connections 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326616050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326613630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 --2- 172.21.15.37:0/1152598607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x56232660aab0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 -- 172.21.15.37:0/1152598607 >> 172.21.15.37:0/1152598607 conn(0x562326604da0 msgr2=0x562326605180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.718+0000 7f92ff781740 1 -- 172.21.15.37:0/1152598607 shutdown_connections 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.719+0000 7f92ff781740 1 -- 172.21.15.37:0/1152598607 wait complete. 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.719+0000 7f92ff781740 1 -- start start 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.719+0000 7f92eea00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92dc0a2fd0 0x7f92dc0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.719+0000 7f92ff781740 1 Processor -- start 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.719+0000 7f92eea00640 1 -- 172.21.15.37:0/211988606 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f92e80cfea0 con 0x7f92f8101630 2024-03-21T09:19:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92f6800640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92dc0a2fd0 0x7f92dc0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x562326613560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326610950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326610f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562326616b10 con 0x56232660b080 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232652a0d0 con 0x562326613c00 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562326616c90 con 0x56232660a6d0 2024-03-21T09:19:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92ff781740 10 client.0 _ll_register_callbacks cb 0x56232652ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x562326613560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x562326613560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55328/0 (socket says 172.21.15.37:55328) 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92f5e00640 1 -- 172.21.15.37:0/351623160 learned_addr learned my addr 172.21.15.37:0/351623160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92f5400640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326610950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.720+0000 7f92f6800640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326610f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5e00640 1 -- 172.21.15.37:0/351623160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 msgr2=0x562326610f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5e00640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326610f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5e00640 1 -- 172.21.15.37:0/351623160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 msgr2=0x562326610950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5e00640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326610950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5e00640 1 -- 172.21.15.37:0/351623160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623262823f0 con 0x56232660a6d0 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f6800640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326610f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5400640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326610950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f6800640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92dc0a2fd0 0x7f92dc0a5490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009b30 tx=0x7f92f002bef0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92f5e00640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x562326613560 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f92e80cd1f0 tx=0x7f92e809bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=defd341d86dfdfe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92d7400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92d7400640 1 -- 172.21.15.37:0/351623160 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80bada0 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92d7400640 1 -- 172.21.15.37:0/351623160 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f92e8018b30 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.721+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92f0033c50 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92d7400640 1 -- 172.21.15.37:0/351623160 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e80a4840 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 10 client.27325 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56232634c7d0 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92d7400640 1 -- 172.21.15.37:0/351623160 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f0033c50 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92d7400640 1 -- 172.21.15.37:0/351623160 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f92e8097c30 con 0x56232660a6d0 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 10 client.27325 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 10 client.27325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 20 client.27325 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326623b50 0x562326625f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 4 client.27325 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.722+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56232657b800 con 0x562326623b50 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.723+0000 7f92f5400640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326623b50 0x562326625f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.723+0000 7f92f5400640 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326623b50 0x562326625f40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab3ed650a7d7e19d server_cookie=6aa972590a804e61 in_seq=0 out_seq=0 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.723+0000 7f92d7400640 10 client.27325 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92d7400640 1 -- 172.21.15.37:0/351623160 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56232657b800 con 0x562326623b50 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92d7400640 10 client.27325 handle_command_reply: tid=0 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 client.27325 shutdown 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 msgr2=0x562326613560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x562326613560 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f92e80cd1f0 tx=0x7f92e809bb10 comp rx=0 tx=0).stop 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 shutdown_connections 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562326613c00 0x562326610f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326623b50 0x562326625f40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326610950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 --2- 172.21.15.37:0/351623160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660a6d0 0x562326613560 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92d7400640 0 client.27325 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 >> 172.21.15.37:0/351623160 conn(0x562326604da0 msgr2=0x562326606af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 shutdown_connections 2024-03-21T09:19:44.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 1 -- 172.21.15.37:0/351623160 wait complete. 2024-03-21T09:19:44.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.724+0000 7f92ff781740 20 client.27325 trim_cache size 0 max 16384 2024-03-21T09:19:44.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.739+0000 7f92ff781740 1 Processor -- start 2024-03-21T09:19:44.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.739+0000 7f92ff781740 1 -- start start 2024-03-21T09:19:44.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92ff781740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232660dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92ff781740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660b080 0x56232660e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92ff781740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562326613c00 0x562326612050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92ff781740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81b3130 con 0x562326613c00 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92ff781740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81b0c30 con 0x56232660a6d0 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92ff781740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81aecf0 con 0x56232660b080 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232660dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660b080 0x56232660e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232660dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36046/0 (socket says 172.21.15.37:36046) 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5e00640 1 -- 172.21.15.37:0/697951760 learned_addr learned my addr 172.21.15.37:0/697951760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f6800640 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562326613c00 0x562326612050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5400640 1 -- 172.21.15.37:0/697951760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 msgr2=0x56232660dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5400640 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232660dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5400640 1 -- 172.21.15.37:0/697951760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562326613c00 msgr2=0x562326612050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5400640 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562326613c00 0x562326612050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5400640 1 -- 172.21.15.37:0/697951760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56232660c540 con 0x56232660b080 2024-03-21T09:19:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.740+0000 7f92f5e00640 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232660dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.741+0000 7f92f5400640 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660b080 0x56232660e110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92e40096c0 tx=0x7f92e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eb8c4210dc647c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.741+0000 7f92e2600640 1 -- 172.21.15.37:0/697951760 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4004220 con 0x56232660b080 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.741+0000 7f92e2600640 1 -- 172.21.15.37:0/697951760 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92e4021070 con 0x56232660b080 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.741+0000 7f92e2600640 1 -- 172.21.15.37:0/697951760 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4010930 con 0x56232660b080 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 -- 172.21.15.37:0/697951760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660b080 msgr2=0x56232660e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660b080 0x56232660e110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92e40096c0 tx=0x7f92e40046c0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 -- 172.21.15.37:0/697951760 shutdown_connections 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562326613c00 0x562326612050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56232660b080 0x56232660e110 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 --2- 172.21.15.37:0/697951760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232660dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 -- 172.21.15.37:0/697951760 >> 172.21.15.37:0/697951760 conn(0x562326604da0 msgr2=0x562326610720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 -- 172.21.15.37:0/697951760 shutdown_connections 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.742+0000 7f92ff781740 1 -- 172.21.15.37:0/697951760 wait complete. 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.743+0000 7f92ff781740 1 -- start start 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.743+0000 7f92ff781740 1 Processor -- start 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232661baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326619e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562326613c00 0x56232661a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81af2f0 con 0x56232660b080 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81b05d0 con 0x56232660a6d0 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81af910 con 0x562326613c00 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92ff781740 10 client.0 _ll_register_callbacks cb 0x7f92e800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:44.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232661baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232661baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36056/0 (socket says 172.21.15.37:36056) 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 -- 172.21.15.37:0/1960472181 learned_addr learned my addr 172.21.15.37:0/1960472181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5400640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326619e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f6800640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562326613c00 0x56232661a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 -- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562326613c00 msgr2=0x56232661a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562326613c00 0x56232661a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 -- 172.21.15.37:0/1960472181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 msgr2=0x562326619e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326619e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.744+0000 7f92f5e00640 1 -- 172.21.15.37:0/1960472181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56232661aac0 con 0x56232660a6d0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92f5e00640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232661baf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f92e800c6a0 tx=0x7f92e80a0100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a825cc94729fda18 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92d7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92d7400640 1 -- 172.21.15.37:0/1960472181 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e809bcb0 con 0x56232660a6d0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92d7400640 1 -- 172.21.15.37:0/1960472181 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f92e80bada0 con 0x56232660a6d0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92e4021070 con 0x56232660a6d0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92d7400640 1 -- 172.21.15.37:0/1960472181 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e809b540 con 0x56232660a6d0 2024-03-21T09:19:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92ff781740 10 client.27302 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5623266801e0 con 0x56232660a6d0 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92d7400640 1 -- 172.21.15.37:0/1960472181 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92e4021070 con 0x56232660a6d0 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92d7400640 1 -- 172.21.15.37:0/1960472181 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f92e80a4a90 con 0x56232660a6d0 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.745+0000 7f92ff781740 10 client.27302 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92ff781740 10 client.27302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92ff781740 20 client.27302 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92ff781740 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326680880 0x562326682c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92ff781740 4 client.27302 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562326575b50 con 0x562326680880 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92f5400640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326680880 0x562326682c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92f5400640 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326680880 0x562326682c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=185989b6db5ace87 server_cookie=e2780d97e5ac2eba in_seq=0 out_seq=0 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.746+0000 7f92d7400640 10 client.27302 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92d7400640 1 -- 172.21.15.37:0/1960472181 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562326575b50 con 0x562326680880 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92d7400640 10 client.27302 handle_command_reply: tid=0 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 client.27302 shutdown 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 msgr2=0x56232661baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232661baf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f92e800c6a0 tx=0x7f92e80a0100 comp rx=0 tx=0).stop 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 shutdown_connections 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562326613c00 0x56232661a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x562326680880 0x562326682c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56232660b080 0x562326619e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 --2- 172.21.15.37:0/1960472181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56232660a6d0 0x56232661baf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 >> 172.21.15.37:0/1960472181 conn(0x562326604da0 msgr2=0x562326603620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92d7400640 0 client.27302 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:19:44.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 shutdown_connections 2024-03-21T09:19:44.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 1 -- 172.21.15.37:0/1960472181 wait complete. 2024-03-21T09:19:44.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.747+0000 7f92ff781740 20 client.27302 trim_cache size 0 max 16384 2024-03-21T09:19:44.748 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92dc0a2fd0 msgr2=0x7f92dc0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92dc0a2fd0 0x7f92dc0a5490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009b30 tx=0x7f92f002bef0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 msgr2=0x7f92f819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 0x7f92f819a090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f92e800e9f0 tx=0x7f92e800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 shutdown_connections 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92f8107640 0x7f92f819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f92dc0a2fd0 0x7f92dc0a5490 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92f8101ff0 0x7f92f819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 --2- 172.21.15.37:0/211988606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f8101630 0x7f92f819a090 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 >> 172.21.15.37:0/211988606 conn(0x7f92f80fd140 msgr2=0x7f92f810cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 shutdown_connections 2024-03-21T09:19:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.748+0000 7f92fca00640 1 -- 172.21.15.37:0/211988606 wait complete. 2024-03-21T09:19:44.763 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:19:44.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:19:44.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 -- 172.21.15.37:0/520674758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 msgr2=0x7f532c110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 --2- 172.21.15.37:0/520674758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c110730 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5324009c10 tx=0x7f532402bef0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 -- 172.21.15.37:0/520674758 shutdown_connections 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 --2- 172.21.15.37:0/520674758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c110730 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 --2- 172.21.15.37:0/520674758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 --2- 172.21.15.37:0/520674758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 0x7f532c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.997+0000 7f5332600640 1 -- 172.21.15.37:0/520674758 >> 172.21.15.37:0/520674758 conn(0x7f532c0ff310 msgr2=0x7f532c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.998+0000 7f5332600640 1 -- 172.21.15.37:0/520674758 shutdown_connections 2024-03-21T09:19:44.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.998+0000 7f5332600640 1 -- 172.21.15.37:0/520674758 wait complete. 2024-03-21T09:19:44.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.999+0000 7f5332600640 1 Processor -- start 2024-03-21T09:19:44.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:44.999+0000 7f5332600640 1 -- start start 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f5332600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f5332600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f5332600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 0x7f532c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f5332600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c113c90 con 0x7f532c109870 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f5332600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c113b10 con 0x7f532c103860 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f5332600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c113e10 con 0x7f532c104220 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55350/0 (socket says 172.21.15.37:55350) 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36066/0 (socket says 172.21.15.37:36066) 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 -- 172.21.15.37:0/1024270369 learned_addr learned my addr 172.21.15.37:0/1024270369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532be00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 0x7f532c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 -- 172.21.15.37:0/1024270369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 msgr2=0x7f532c19a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c19a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 -- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 msgr2=0x7f532c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 0x7f532c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532aa00640 1 -- 172.21.15.37:0/1024270369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c19f140 con 0x7f532c104220 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532be00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 0x7f532c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:44.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.000+0000 7f532b400640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:44.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.001+0000 7f532aa00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c19a690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f531800e9c0 tx=0x7f531800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:44.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.001+0000 7f5323e00640 1 -- 172.21.15.37:0/1024270369 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531800ccc0 con 0x7f532c104220 2024-03-21T09:19:44.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.001+0000 7f5323e00640 1 -- 172.21.15.37:0/1024270369 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5318005500 con 0x7f532c104220 2024-03-21T09:19:44.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.001+0000 7f5323e00640 1 -- 172.21.15.37:0/1024270369 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318010430 con 0x7f532c104220 2024-03-21T09:19:44.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.001+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c19f430 con 0x7f532c104220 2024-03-21T09:19:44.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.001+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c1a6cc0 con 0x7f532c104220 2024-03-21T09:19:45.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 Processor -- start 2024-03-21T09:19:45.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 -- start start 2024-03-21T09:19:45.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ab6d0 0x5625b07abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 0x5625b07b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07b4c00 0x5625b07b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b070f2c0 con 0x5625b07b4c00 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b06b7c90 con 0x5625b07ac080 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.005+0000 7f5335220740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b070b360 con 0x5625b07ab6d0 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 0x5625b07b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ab6d0 0x5625b07abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 0x5625b07b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36070/0 (socket says 172.21.15.37:36070) 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 -- 172.21.15.37:0/786592735 learned_addr learned my addr 172.21.15.37:0/786592735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532be00640 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ab6d0 0x5625b07abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55358/0 (socket says 172.21.15.37:55358) 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 -- 172.21.15.37:0/786592735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ab6d0 msgr2=0x5625b07abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ab6d0 0x5625b07abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 -- 172.21.15.37:0/786592735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07b4c00 msgr2=0x5625b07b7050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07b4c00 0x5625b07b7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 -- 172.21.15.37:0/786592735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b06dc080 con 0x5625b07ac080 2024-03-21T09:19:45.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f532b400640 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 0x5625b07b4630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f531c010400 tx=0x7f531c0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889bfdabcfe86dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.006+0000 7f5320c00640 1 -- 172.21.15.37:0/786592735 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c011760 con 0x5625b07ac080 2024-03-21T09:19:45.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5320c00640 1 -- 172.21.15.37:0/786592735 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f531c011de0 con 0x5625b07ac080 2024-03-21T09:19:45.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5320c00640 1 -- 172.21.15.37:0/786592735 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c01a380 con 0x5625b07ac080 2024-03-21T09:19:45.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 -- 172.21.15.37:0/786592735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 msgr2=0x5625b07b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 0x5625b07b4630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f531c010400 tx=0x7f531c0108c0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 -- 172.21.15.37:0/786592735 shutdown_connections 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07b4c00 0x5625b07b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ac080 0x5625b07b4630 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5323e00640 1 -- 172.21.15.37:0/1024270369 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f53180046c0 con 0x7f532c104220 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 --2- 172.21.15.37:0/786592735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ab6d0 0x5625b07abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 -- 172.21.15.37:0/786592735 >> 172.21.15.37:0/786592735 conn(0x5625b07a5da0 msgr2=0x5625b07a6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 -- 172.21.15.37:0/786592735 shutdown_connections 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.007+0000 7f5335220740 1 -- 172.21.15.37:0/786592735 wait complete. 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.008+0000 7f5335220740 1 -- start start 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.008+0000 7f5335220740 1 Processor -- start 2024-03-21T09:19:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b07b7b70 con 0x5625b07ab6d0 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b06cb0d0 con 0x5625b07b24e0 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b07b7cf0 con 0x5625b07ac080 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5335220740 10 client.0 _ll_register_callbacks cb 0x5625b06cfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45690/0 (socket says 172.21.15.37:45690) 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55362/0 (socket says 172.21.15.37:55362) 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532be00640 1 -- 172.21.15.37:0/3434958892 learned_addr learned my addr 172.21.15.37:0/3434958892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f5323e00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f53100a3020 0x7f53100a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 -- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 msgr2=0x5625b07c0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07c0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 -- 172.21.15.37:0/3434958892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 msgr2=0x5625b07b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.009+0000 7f532b400640 1 -- 172.21.15.37:0/3434958892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625b04233f0 con 0x5625b07ac080 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f5323e00640 1 -- 172.21.15.37:0/1024270369 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f53180cff60 con 0x7f532c104220 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f532be00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f53100a3020 0x7f53100a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f532b400640 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07b1fa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x5625b07a6e10 tx=0x7f531c01c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af91acd44bb5269 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f5315600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f5315600640 1 -- 172.21.15.37:0/3434958892 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c02b060 con 0x5625b07ac080 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f5315600640 1 -- 172.21.15.37:0/3434958892 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f531c011770 con 0x5625b07ac080 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f531c011de0 con 0x5625b07ac080 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.010+0000 7f532be00640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f53100a3020 0x7f53100a54e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53240099a0 tx=0x7f532402bef0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5315600640 1 -- 172.21.15.37:0/3434958892 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f531c032050 con 0x5625b07ac080 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 10 client.27343 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5625b04ed7d0 con 0x5625b07ac080 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5315600640 1 -- 172.21.15.37:0/3434958892 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f531c011de0 con 0x5625b07ac080 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5315600640 1 -- 172.21.15.37:0/3434958892 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f531c0269e0 con 0x5625b07ac080 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 10 client.27343 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 10 client.27343 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 20 client.27343 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b07c4fe0 0x5625b07c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 4 client.27343 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625b071c800 con 0x5625b07c4fe0 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.011+0000 7f532aa00640 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b07c4fe0 0x5625b07c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.012+0000 7f532aa00640 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b07c4fe0 0x5625b07c73d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e6504931a8b3e61 server_cookie=11fc2033b75bfd7c in_seq=0 out_seq=0 2024-03-21T09:19:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.012+0000 7f5315600640 10 client.27343 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5315600640 1 -- 172.21.15.37:0/3434958892 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5625b071c800 con 0x5625b07c4fe0 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5315600640 10 client.27343 handle_command_reply: tid=0 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 client.27343 shutdown 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 msgr2=0x5625b07b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07b1fa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x5625b07a6e10 tx=0x7f531c01c6b0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 shutdown_connections 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b07c4fe0 0x5625b07c73d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07b1fa0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 --2- 172.21.15.37:0/3434958892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5315600640 0 client.27343 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 >> 172.21.15.37:0/3434958892 conn(0x5625b07a5da0 msgr2=0x5625b07a7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 shutdown_connections 2024-03-21T09:19:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 1 -- 172.21.15.37:0/3434958892 wait complete. 2024-03-21T09:19:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.013+0000 7f5335220740 20 client.27343 trim_cache size 0 max 16384 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.031+0000 7f5335220740 1 Processor -- start 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.031+0000 7f5335220740 1 -- start start 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f5335220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f5335220740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07c1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f5335220740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f5335220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1b3310 con 0x5625b07ab6d0 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f5335220740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1b0e10 con 0x5625b07b24e0 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f5335220740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1aeed0 con 0x5625b07ac080 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07c1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07c1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55378/0 (socket says 172.21.15.37:55378) 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 -- 172.21.15.37:0/486768386 learned_addr learned my addr 172.21.15.37:0/486768386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532be00640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532b400640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 -- 172.21.15.37:0/486768386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 msgr2=0x5625b07aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 -- 172.21.15.37:0/486768386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 msgr2=0x5625b07c0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07c0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532aa00640 1 -- 172.21.15.37:0/486768386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b07affb0 con 0x5625b07ac080 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532be00640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:45.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.032+0000 7f532b400640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07c0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:45.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.033+0000 7f532aa00640 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07c1490 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f53180988c0 tx=0x7f5318004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7230a693e8a700 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.033+0000 7f5317e00640 1 -- 172.21.15.37:0/486768386 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531809ba30 con 0x5625b07ac080 2024-03-21T09:19:45.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.033+0000 7f5317e00640 1 -- 172.21.15.37:0/486768386 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f531809c5f0 con 0x5625b07ac080 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.033+0000 7f5317e00640 1 -- 172.21.15.37:0/486768386 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531809b850 con 0x5625b07ac080 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.033+0000 7f5335220740 1 -- 172.21.15.37:0/486768386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 msgr2=0x5625b07c1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.033+0000 7f5335220740 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07c1490 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f53180988c0 tx=0x7f5318004b00 comp rx=0 tx=0).stop 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 -- 172.21.15.37:0/486768386 shutdown_connections 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07b24e0 0x5625b07aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07ac080 0x5625b07c1490 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 --2- 172.21.15.37:0/486768386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ab6d0 0x5625b07c0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 -- 172.21.15.37:0/486768386 >> 172.21.15.37:0/486768386 conn(0x5625b07a5da0 msgr2=0x5625b07a4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 -- 172.21.15.37:0/486768386 shutdown_connections 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 -- 172.21.15.37:0/486768386 wait complete. 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 -- start start 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.034+0000 7f5335220740 1 Processor -- start 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 0x5625b07c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ac080 0x5625b07c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07b24e0 0x5625b081f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1af4d0 con 0x5625b07ac080 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1b07b0 con 0x5625b07ab6d0 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1afaf0 con 0x5625b07b24e0 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f5335220740 10 client.0 _ll_register_callbacks cb 0x7f5318005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 0x5625b07c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07b24e0 0x5625b081f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 0x5625b07c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36096/0 (socket says 172.21.15.37:36096) 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 -- 172.21.15.37:0/1739588682 learned_addr learned my addr 172.21.15.37:0/1739588682 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532aa00640 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ac080 0x5625b07c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 -- 172.21.15.37:0/1739588682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07b24e0 msgr2=0x5625b081f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07b24e0 0x5625b081f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 -- 172.21.15.37:0/1739588682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ac080 msgr2=0x5625b07c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ac080 0x5625b07c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.035+0000 7f532b400640 1 -- 172.21.15.37:0/1739588682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625b081fa50 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f532b400640 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 0x5625b07c7f40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5625b07b12c0 tx=0x7f531c0058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a143a021387fdf25 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5315600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5315600640 1 -- 172.21.15.37:0/1739588682 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c02b040 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5315600640 1 -- 172.21.15.37:0/1739588682 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f531c01a420 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f531809c5f0 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5315600640 1 -- 172.21.15.37:0/1739588682 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f531c03f050 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5335220740 10 client.27320 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5625b0820d50 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5315600640 1 -- 172.21.15.37:0/1739588682 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f531809c5f0 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.036+0000 7f5315600640 1 -- 172.21.15.37:0/1739588682 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f531c011980 con 0x5625b07ab6d0 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5335220740 10 client.27320 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5335220740 10 client.27320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5335220740 20 client.27320 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:45.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5335220740 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b0821300 0x5625b0823720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5335220740 4 client.27320 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:19:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5625b0716b50 con 0x5625b0821300 2024-03-21T09:19:45.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f532aa00640 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b0821300 0x5625b0823720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f532aa00640 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b0821300 0x5625b0823720 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c665338722b4ccc server_cookie=a09673cbe8c14d3c in_seq=0 out_seq=0 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.037+0000 7f5315600640 10 client.27320 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5315600640 1 -- 172.21.15.37:0/1739588682 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5625b0716b50 con 0x5625b0821300 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5315600640 10 client.27320 handle_command_reply: tid=0 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 client.27320 shutdown 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 msgr2=0x5625b07c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 0x5625b07c7f40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5625b07b12c0 tx=0x7f531c0058f0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 shutdown_connections 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x5625b0821300 0x5625b0823720 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5625b07b24e0 0x5625b081f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625b07ac080 0x5625b07c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 --2- 172.21.15.37:0/1739588682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625b07ab6d0 0x5625b07c7f40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5315600640 0 client.27320 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 >> 172.21.15.37:0/1739588682 conn(0x5625b07a5da0 msgr2=0x5625b07a7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 shutdown_connections 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 1 -- 172.21.15.37:0/1739588682 wait complete. 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.038+0000 7f5335220740 20 client.27320 trim_cache size 0 max 16384 2024-03-21T09:19:45.038 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f53100a3020 msgr2=0x7f53100a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f53100a3020 0x7f53100a54e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53240099a0 tx=0x7f532402bef0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 msgr2=0x7f532c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c19a690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f531800e9c0 tx=0x7f531800ee80 comp rx=0 tx=0).stop 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 shutdown_connections 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c109870 0x7f532c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f532c104220 0x7f532c19a690 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f53100a3020 0x7f53100a54e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 --2- 172.21.15.37:0/1024270369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f532c103860 0x7f532c19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 >> 172.21.15.37:0/1024270369 conn(0x7f532c0ff310 msgr2=0x7f532c0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 shutdown_connections 2024-03-21T09:19:45.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.039+0000 7f5332600640 1 -- 172.21.15.37:0/1024270369 wait complete. 2024-03-21T09:19:45.054 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:19:45.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:19:45.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 -- 172.21.15.37:0/3197106401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4105d20 msgr2=0x7fa6d4110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 --2- 172.21.15.37:0/3197106401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4105d20 0x7fa6d4110850 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009cd0 tx=0x7fa6cc02c1c0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 -- 172.21.15.37:0/3197106401 shutdown_connections 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 --2- 172.21.15.37:0/3197106401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4105d20 0x7fa6d4110850 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 --2- 172.21.15.37:0/3197106401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105380 0x7fa6d41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 --2- 172.21.15.37:0/3197106401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4103c40 0x7fa6d4104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 -- 172.21.15.37:0/3197106401 >> 172.21.15.37:0/3197106401 conn(0x7fa6d40ff350 msgr2=0x7fa6d4101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.276+0000 7fa6d9800640 1 -- 172.21.15.37:0/3197106401 shutdown_connections 2024-03-21T09:19:45.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.277+0000 7fa6d9800640 1 -- 172.21.15.37:0/3197106401 wait complete. 2024-03-21T09:19:45.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.278+0000 7fa6d9800640 1 Processor -- start 2024-03-21T09:19:45.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.278+0000 7fa6d9800640 1 -- start start 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 0x7fa6d4075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d9800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4105380 0x7fa6d40760f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105d20 0x7fa6d4076630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d9800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4113d90 con 0x7fa6d4105380 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d9800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4113c10 con 0x7fa6d4105d20 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d9800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4113f10 con 0x7fa6d4103c40 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 0x7fa6d4075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105d20 0x7fa6d4076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 0x7fa6d4075bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55398/0 (socket says 172.21.15.37:55398) 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4193159686 learned_addr learned my addr 172.21.15.37:0/4193159686 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2000640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4105380 0x7fa6d40760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4193159686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105d20 msgr2=0x7fa6d4076630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105d20 0x7fa6d4076630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4105380 msgr2=0x7fa6d40760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4105380 0x7fa6d40760f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4193159686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d41aa870 con 0x7fa6d4103c40 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.279+0000 7fa6d3400640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105d20 0x7fa6d4076630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.280+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 0x7fa6d4075bb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400ea20 tx=0x7fa6c400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.280+0000 7fa6cb400640 1 -- 172.21.15.37:0/4193159686 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400cd80 con 0x7fa6d4103c40 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.280+0000 7fa6cb400640 1 -- 172.21.15.37:0/4193159686 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4010910 con 0x7fa6d4103c40 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.280+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d41aab60 con 0x7fa6d4103c40 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.280+0000 7fa6cb400640 1 -- 172.21.15.37:0/4193159686 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4018920 con 0x7fa6d4103c40 2024-03-21T09:19:45.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.280+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d41ab100 con 0x7fa6d4103c40 2024-03-21T09:19:45.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.283+0000 7fa6cb400640 1 -- 172.21.15.37:0/4193159686 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa6c4010ac0 con 0x7fa6d4103c40 2024-03-21T09:19:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.284+0000 7fa6cb400640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6b80a3110 0x7fa6b80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.285+0000 7fa6cb400640 1 -- 172.21.15.37:0/4193159686 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa6c40cff80 con 0x7fa6d4103c40 2024-03-21T09:19:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.285+0000 7fa6d2000640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6b80a3110 0x7fa6b80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.285+0000 7fa6dc56a740 1 Processor -- start 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.285+0000 7fa6dc56a740 1 -- start start 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2000640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6b80a3110 0x7fa6b80a55d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4077320 tx=0x7fa6c0005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x559790408ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790411630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x559790414050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979036c2c0 con 0x559790409080 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559790314c90 con 0x559790411c00 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559790368360 con 0x5597904086d0 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x559790408ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x559790408ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55404/0 (socket says 172.21.15.37:55404) 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2321340261 learned_addr learned my addr 172.21.15.37:0/2321340261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d3400640 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x559790414050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2000640 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790411630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2321340261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 msgr2=0x559790414050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x559790414050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2321340261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 msgr2=0x559790411630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790411630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.286+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2321340261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559790339080 con 0x5597904086d0 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.287+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x559790408ab0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4098990 tx=0x7fa6c40a0060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06923cb2bb8f201 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.287+0000 7fa6bfe00640 1 -- 172.21.15.37:0/2321340261 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c409a2c0 con 0x5597904086d0 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.287+0000 7fa6bfe00640 1 -- 172.21.15.37:0/2321340261 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c40cd4f0 con 0x5597904086d0 2024-03-21T09:19:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.287+0000 7fa6bfe00640 1 -- 172.21.15.37:0/2321340261 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4097f70 con 0x5597904086d0 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2321340261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 msgr2=0x559790408ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x559790408ab0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4098990 tx=0x7fa6c40a0060 comp rx=0 tx=0).stop 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2321340261 shutdown_connections 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x559790414050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790411630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2321340261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x559790408ab0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2321340261 >> 172.21.15.37:0/2321340261 conn(0x559790402da0 msgr2=0x559790403180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2321340261 shutdown_connections 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.288+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2321340261 wait complete. 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.289+0000 7fa6dc56a740 1 -- start start 2024-03-21T09:19:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.289+0000 7fa6dc56a740 1 Processor -- start 2024-03-21T09:19:45.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 0x55979040e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790411c00 0x55979040f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559790414b70 con 0x559790411c00 2024-03-21T09:19:45.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597903280d0 con 0x559790409080 2024-03-21T09:19:45.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559790414cf0 con 0x5597904086d0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6dc56a740 10 client.0 _ll_register_callbacks cb 0x55979032ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 0x55979040e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 0x55979040e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36124/0 (socket says 172.21.15.37:36124) 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2000640 1 -- 172.21.15.37:0/2799407268 learned_addr learned my addr 172.21.15.37:0/2799407268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d3400640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790411c00 0x55979040f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2799407268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 msgr2=0x55979040e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 0x55979040e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790411c00 msgr2=0x55979040f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790411c00 0x55979040f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2a00640 1 -- 172.21.15.37:0/2799407268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597900803f0 con 0x5597904086d0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.290+0000 7fa6d2000640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 0x55979040e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904115e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4098990 tx=0x7fa6c40cd280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a98c35d3c3e56b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:45.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6bcc00640 1 -- 172.21.15.37:0/2799407268 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4010430 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6bcc00640 1 -- 172.21.15.37:0/2799407268 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa6c40105d0 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6c40cd4f0 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6bcc00640 1 -- 172.21.15.37:0/2799407268 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c4097d10 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6dc56a740 10 client.27361 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55979014a7d0 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6bcc00640 1 -- 172.21.15.37:0/2799407268 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c40cd4f0 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.291+0000 7fa6bcc00640 1 -- 172.21.15.37:0/2799407268 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa6c40b8070 con 0x5597904086d0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6dc56a740 10 client.27361 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6dc56a740 10 client.27361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6dc56a740 20 client.27361 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5597904219f0 0x559790423de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6dc56a740 4 client.27361 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559790379800 con 0x5597904219f0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6d2000640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5597904219f0 0x559790423de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6d2000640 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5597904219f0 0x559790423de0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5081243c5ae0c82a server_cookie=1c07c05291a86f66 in_seq=0 out_seq=0 2024-03-21T09:19:45.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.292+0000 7fa6bcc00640 10 client.27361 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.293+0000 7fa6bcc00640 1 -- 172.21.15.37:0/2799407268 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559790379800 con 0x5597904219f0 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.293+0000 7fa6bcc00640 10 client.27361 handle_command_reply: tid=0 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.293+0000 7fa6dc56a740 1 client.27361 shutdown 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.293+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 msgr2=0x5597904115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.293+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904115e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4098990 tx=0x7fa6c40cd280 comp rx=0 tx=0).stop 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 shutdown_connections 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790411c00 0x55979040f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5597904219f0 0x559790423de0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790409080 0x55979040e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/2799407268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904115e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6bcc00640 0 client.27361 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:45.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 >> 172.21.15.37:0/2799407268 conn(0x559790402da0 msgr2=0x559790404af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 shutdown_connections 2024-03-21T09:19:45.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 1 -- 172.21.15.37:0/2799407268 wait complete. 2024-03-21T09:19:45.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.294+0000 7fa6dc56a740 20 client.27361 trim_cache size 0 max 16384 2024-03-21T09:19:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.309+0000 7fa6dc56a740 1 Processor -- start 2024-03-21T09:19:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.309+0000 7fa6dc56a740 1 -- start start 2024-03-21T09:19:45.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.309+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.309+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x5597904110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x5597904115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41b7690 con 0x559790409080 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41b5190 con 0x559790411c00 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41b3250 con 0x5597904086d0 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x5597904115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55432/0 (socket says 172.21.15.37:55432) 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 -- 172.21.15.37:0/3061112887 learned_addr learned my addr 172.21.15.37:0/3061112887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2000640 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x5597904110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 -- 172.21.15.37:0/3061112887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 msgr2=0x5597904115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x5597904115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 -- 172.21.15.37:0/3061112887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 msgr2=0x5597904110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x5597904110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.310+0000 7fa6d2a00640 1 -- 172.21.15.37:0/3061112887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55979040f420 con 0x5597904086d0 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.311+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904048c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cd7f0 tx=0x7fa6c409a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42d1edeabda946a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.311+0000 7fa6bf400640 1 -- 172.21.15.37:0/3061112887 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c409b620 con 0x5597904086d0 2024-03-21T09:19:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.311+0000 7fa6bf400640 1 -- 172.21.15.37:0/3061112887 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4097eb0 con 0x5597904086d0 2024-03-21T09:19:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.311+0000 7fa6bf400640 1 -- 172.21.15.37:0/3061112887 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4096d90 con 0x5597904086d0 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 -- 172.21.15.37:0/3061112887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 msgr2=0x5597904048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904048c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cd7f0 tx=0x7fa6c409a490 comp rx=0 tx=0).stop 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 -- 172.21.15.37:0/3061112887 shutdown_connections 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559790411c00 0x5597904115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x5597904110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/3061112887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597904086d0 0x5597904048c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 -- 172.21.15.37:0/3061112887 >> 172.21.15.37:0/3061112887 conn(0x559790402da0 msgr2=0x5597904041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 -- 172.21.15.37:0/3061112887 shutdown_connections 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.312+0000 7fa6dc56a740 1 -- 172.21.15.37:0/3061112887 wait complete. 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.313+0000 7fa6dc56a740 1 -- start start 2024-03-21T09:19:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.313+0000 7fa6dc56a740 1 Processor -- start 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 0x559790424aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790424fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559790411c00 0x559790475f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41b3850 con 0x559790409080 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41b4b30 con 0x5597904086d0 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41b3e70 con 0x559790411c00 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6dc56a740 10 client.0 _ll_register_callbacks cb 0x7fa6c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 0x559790424aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559790411c00 0x559790475f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790424fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 0x559790424aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36136/0 (socket says 172.21.15.37:36136) 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4177596138 learned_addr learned my addr 172.21.15.37:0/4177596138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4177596138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559790411c00 msgr2=0x559790475f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559790411c00 0x559790475f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 msgr2=0x559790424fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790424fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.314+0000 7fa6d2a00640 1 -- 172.21.15.37:0/4177596138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559790476520 con 0x5597904086d0 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.315+0000 7fa6d2a00640 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 0x559790424aa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa6c409bcc0 tx=0x7fa6c40cd350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a91e6c19742b975 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.315+0000 7fa6bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.317+0000 7fa6bcc00640 1 -- 172.21.15.37:0/4177596138 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4085240 con 0x5597904086d0 2024-03-21T09:19:45.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.317+0000 7fa6bcc00640 1 -- 172.21.15.37:0/4177596138 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa6c4096b30 con 0x5597904086d0 2024-03-21T09:19:45.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.317+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6c4097eb0 con 0x5597904086d0 2024-03-21T09:19:45.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6bcc00640 1 -- 172.21.15.37:0/4177596138 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c409a2c0 con 0x5597904086d0 2024-03-21T09:19:45.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 10 client.27338 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:45.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559790476520 con 0x5597904086d0 2024-03-21T09:19:45.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6bcc00640 1 -- 172.21.15.37:0/4177596138 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4097eb0 con 0x5597904086d0 2024-03-21T09:19:45.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6bcc00640 1 -- 172.21.15.37:0/4177596138 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa6c40b8070 con 0x5597904086d0 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 10 client.27338 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 10 client.27338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 20 client.27338 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55979047da40 0x55979047fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 4 client.27338 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.318+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559790373b50 con 0x55979047da40 2024-03-21T09:19:45.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.319+0000 7fa6d2000640 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55979047da40 0x55979047fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.319+0000 7fa6d2000640 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55979047da40 0x55979047fe30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded9196282c2330c server_cookie=98a8a9f06fb99927 in_seq=0 out_seq=0 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.319+0000 7fa6bcc00640 10 client.27338 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6bcc00640 1 -- 172.21.15.37:0/4177596138 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559790373b50 con 0x55979047da40 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6bcc00640 10 client.27338 handle_command_reply: tid=0 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 client.27338 shutdown 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 msgr2=0x559790424aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 0x559790424aa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa6c409bcc0 tx=0x7fa6c40cd350 comp rx=0 tx=0).stop 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 shutdown_connections 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559790411c00 0x559790475f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559790409080 0x559790424fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55979047da40 0x55979047fe30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 --2- 172.21.15.37:0/4177596138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597904086d0 0x559790424aa0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 >> 172.21.15.37:0/4177596138 conn(0x559790402da0 msgr2=0x55979041d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6bcc00640 0 client.27338 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 shutdown_connections 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 1 -- 172.21.15.37:0/4177596138 wait complete. 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.320+0000 7fa6dc56a740 20 client.27338 trim_cache size 0 max 16384 2024-03-21T09:19:45.317 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:19:45.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6b80a3110 msgr2=0x7fa6b80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6b80a3110 0x7fa6b80a55d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4077320 tx=0x7fa6c0005f90 comp rx=0 tx=0).stop 2024-03-21T09:19:45.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 msgr2=0x7fa6d4075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:45.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 0x7fa6d4075bb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400ea20 tx=0x7fa6c400eee0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 shutdown_connections 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa6b80a3110 0x7fa6b80a55d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d4105d20 0x7fa6d4076630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4105380 0x7fa6d40760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 --2- 172.21.15.37:0/4193159686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6d4103c40 0x7fa6d4075bb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 >> 172.21.15.37:0/4193159686 conn(0x7fa6d40ff350 msgr2=0x7fa6d4107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 shutdown_connections 2024-03-21T09:19:45.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:45.321+0000 7fa6d9800640 1 -- 172.21.15.37:0/4193159686 wait complete. 2024-03-21T09:19:45.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:45 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3276405565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:19:45.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:45 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3276405565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:19:45.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:45 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3276405565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:19:46.335 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:19:46.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:19:46.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:19:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 -- 172.21.15.37:0/2748644474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 msgr2=0x7ff75810e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 --2- 172.21.15.37:0/2748644474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75810e0c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff750009ca0 tx=0x7ff75002c190 comp rx=0 tx=0).stop 2024-03-21T09:19:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 -- 172.21.15.37:0/2748644474 shutdown_connections 2024-03-21T09:19:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 --2- 172.21.15.37:0/2748644474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75810e0c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 --2- 172.21.15.37:0/2748644474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7581031e0 0x7ff75810b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 --2- 172.21.15.37:0/2748644474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7581028c0 0x7ff758102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 -- 172.21.15.37:0/2748644474 >> 172.21.15.37:0/2748644474 conn(0x7ff758077750 msgr2=0x7ff758077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 -- 172.21.15.37:0/2748644474 shutdown_connections 2024-03-21T09:19:46.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.593+0000 7ff760600640 1 -- 172.21.15.37:0/2748644474 wait complete. 2024-03-21T09:19:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.594+0000 7ff760600640 1 Processor -- start 2024-03-21T09:19:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.594+0000 7ff760600640 1 -- start start 2024-03-21T09:19:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff760600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7581028c0 0x7ff75819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff760600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7581031e0 0x7ff75819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff760600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff760600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758111a20 con 0x7ff7581031e0 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff760600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581118a0 con 0x7ff7581028c0 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff760600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758111ba0 con 0x7ff75810bc90 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff75e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7581028c0 0x7ff75819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff75d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7581031e0 0x7ff75819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff75e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55460/0 (socket says 172.21.15.37:55460) 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.595+0000 7ff75e600640 1 -- 172.21.15.37:0/3957244790 learned_addr learned my addr 172.21.15.37:0/3957244790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff75e600640 1 -- 172.21.15.37:0/3957244790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7581028c0 msgr2=0x7ff75819a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff75e600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7581028c0 0x7ff75819a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff75e600640 1 -- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7581031e0 msgr2=0x7ff75819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff75e600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7581031e0 0x7ff75819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff75e600640 1 -- 172.21.15.37:0/3957244790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75819f0e0 con 0x7ff75810bc90 2024-03-21T09:19:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff75e600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75819e9b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff75002c7a0 tx=0x7ff750035e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff756000640 1 -- 172.21.15.37:0/3957244790 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750039040 con 0x7ff75810bc90 2024-03-21T09:19:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff756000640 1 -- 172.21.15.37:0/3957244790 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff750044070 con 0x7ff75810bc90 2024-03-21T09:19:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75819f370 con 0x7ff75810bc90 2024-03-21T09:19:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.596+0000 7ff756000640 1 -- 172.21.15.37:0/3957244790 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750032940 con 0x7ff75810bc90 2024-03-21T09:19:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.597+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7581a6c60 con 0x7ff75810bc90 2024-03-21T09:19:46.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.600+0000 7ff7633e7740 1 Processor -- start 2024-03-21T09:19:46.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff756000640 1 -- 172.21.15.37:0/3957244790 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff750032ae0 con 0x7ff75810bc90 2024-03-21T09:19:46.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 -- start start 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7bd5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 0x564ad7bde040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bde580 0x564ad7be0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad7b39eb0 con 0x564ad7bd5080 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad7ae2a60 con 0x564ad7bd59a0 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.601+0000 7ff7633e7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad7b35f50 con 0x564ad7bde580 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 0x564ad7bde040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7bd5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 0x564ad7bde040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36158/0 (socket says 172.21.15.37:36158) 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 -- 172.21.15.37:0/3641769272 learned_addr learned my addr 172.21.15.37:0/3641769272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75d200640 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bde580 0x564ad7be0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 -- 172.21.15.37:0/3641769272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bde580 msgr2=0x564ad7be0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bde580 0x564ad7be0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 -- 172.21.15.37:0/3641769272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 msgr2=0x564ad7bd5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7bd5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75dc00640 1 -- 172.21.15.37:0/3641769272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ad79167e0 con 0x564ad7bd59a0 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.602+0000 7ff75d200640 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7bd5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:46.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff75dc00640 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 0x564ad7bde040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009cf0 tx=0x7ff74c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a3eaf3d69d30d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff74aa00640 1 -- 172.21.15.37:0/3641769272 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c01e070 con 0x564ad7bd59a0 2024-03-21T09:19:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff74aa00640 1 -- 172.21.15.37:0/3641769272 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74c019b10 con 0x564ad7bd59a0 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff74aa00640 1 -- 172.21.15.37:0/3641769272 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00ea60 con 0x564ad7bd59a0 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff756000640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7400a3110 0x7ff7400a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff756000640 1 -- 172.21.15.37:0/3957244790 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff7500f1f20 con 0x7ff75810bc90 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.603+0000 7ff75d200640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7400a3110 0x7ff7400a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 -- 172.21.15.37:0/3641769272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 msgr2=0x564ad7bde040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 0x564ad7bde040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009cf0 tx=0x7ff74c011810 comp rx=0 tx=0).stop 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 -- 172.21.15.37:0/3641769272 shutdown_connections 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bde580 0x564ad7be0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd59a0 0x564ad7bde040 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 --2- 172.21.15.37:0/3641769272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7bd5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 -- 172.21.15.37:0/3641769272 >> 172.21.15.37:0/3641769272 conn(0x564ad7bcf7b0 msgr2=0x564ad7bcfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 -- 172.21.15.37:0/3641769272 shutdown_connections 2024-03-21T09:19:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 -- 172.21.15.37:0/3641769272 wait complete. 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff75d200640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7400a3110 0x7ff7400a55d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff744005f60 tx=0x7ff7440096a0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.604+0000 7ff7633e7740 1 -- start start 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 Processor -- start 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 0x564ad7bddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd59a0 0x564ad7bdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bdb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad7be1370 con 0x564ad7bd59a0 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad7af5eb0 con 0x564ad7bde580 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad7be14f0 con 0x564ad7bd5080 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff7633e7740 10 client.0 _ll_register_callbacks cb 0x564ad7aea150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.605+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 0x564ad7bddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 0x564ad7bddf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55482/0 (socket says 172.21.15.37:55482) 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 -- 172.21.15.37:0/133718945 learned_addr learned my addr 172.21.15.37:0/133718945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75e600640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bdb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75d200640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd59a0 0x564ad7bdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 -- 172.21.15.37:0/133718945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 msgr2=0x564ad7bdb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bdb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 -- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd59a0 msgr2=0x564ad7bdb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd59a0 0x564ad7bdb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 -- 172.21.15.37:0/133718945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ad7976f70 con 0x564ad7bd5080 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75e600640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bdb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff75dc00640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 0x564ad7bddf10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0058c0 tx=0x7ff74c00e4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=568cbed70a588e86 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.606+0000 7ff73f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff73f400640 1 -- 172.21.15.37:0/133718945 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c027070 con 0x564ad7bd5080 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff73f400640 1 -- 172.21.15.37:0/133718945 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff74c019840 con 0x564ad7bd5080 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff74c019b10 con 0x564ad7bd5080 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff73f400640 1 -- 172.21.15.37:0/133718945 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74c033050 con 0x564ad7bd5080 2024-03-21T09:19:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff7633e7740 10 client.27379 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564ad780e170 con 0x564ad7bd5080 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff73f400640 1 -- 172.21.15.37:0/133718945 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74c019b10 con 0x564ad7bd5080 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.607+0000 7ff73f400640 1 -- 172.21.15.37:0/133718945 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff74c0046c0 con 0x564ad7bd5080 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff7633e7740 10 client.27379 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff7633e7740 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff7633e7740 20 client.27379 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff7633e7740 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7bee4b0 0x564ad7bf08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff7633e7740 4 client.27379 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ad7b473f0 con 0x564ad7bee4b0 2024-03-21T09:19:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff75d200640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7bee4b0 0x564ad7bf08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.608 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:46 smithi071 ceph-mon[36597]: pgmap v232: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.608+0000 7ff75d200640 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7bee4b0 0x564ad7bf08a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156753450f7ab88d server_cookie=69ed8e912ea11dd0 in_seq=0 out_seq=0 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.609+0000 7ff73f400640 10 client.27379 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.609+0000 7ff73f400640 1 -- 172.21.15.37:0/133718945 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564ad7b473f0 con 0x564ad7bee4b0 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.609+0000 7ff73f400640 10 client.27379 handle_command_reply: tid=0 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.609+0000 7ff7633e7740 1 client.27379 shutdown 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.609+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 msgr2=0x564ad7bddf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.609+0000 7ff7633e7740 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 0x564ad7bddf10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0058c0 tx=0x7ff74c00e4f0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 shutdown_connections 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7bee4b0 0x564ad7bf08a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bdb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff73f400640 0 client.27379 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd59a0 0x564ad7bdb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 --2- 172.21.15.37:0/133718945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd5080 0x564ad7bddf10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 >> 172.21.15.37:0/133718945 conn(0x564ad7bcf7b0 msgr2=0x564ad7bd1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 shutdown_connections 2024-03-21T09:19:46.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 1 -- 172.21.15.37:0/133718945 wait complete. 2024-03-21T09:19:46.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.610+0000 7ff7633e7740 20 client.27379 trim_cache size 0 max 16384 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 Processor -- start 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 -- start start 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 0x564ad7bd1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bdd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bde580 0x564ad7bddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581b31b0 con 0x564ad7bde580 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581b0cb0 con 0x564ad7bd5080 2024-03-21T09:19:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff7633e7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581aed70 con 0x564ad7bd59a0 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 0x564ad7bd1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 0x564ad7bd1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36176/0 (socket says 172.21.15.37:36176) 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff75dc00640 1 -- 172.21.15.37:0/4218198431 learned_addr learned my addr 172.21.15.37:0/4218198431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.628+0000 7ff75d200640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bdd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75e600640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bde580 0x564ad7bddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75d200640 1 -- 172.21.15.37:0/4218198431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 msgr2=0x564ad7bd1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75d200640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 0x564ad7bd1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75d200640 1 -- 172.21.15.37:0/4218198431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bde580 msgr2=0x564ad7bddf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75d200640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bde580 0x564ad7bddf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75d200640 1 -- 172.21.15.37:0/4218198431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ad7bcac20 con 0x564ad7bd59a0 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75dc00640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 0x564ad7bd1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75e600640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bde580 0x564ad7bddf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff75d200640 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bdd9d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x564ad7bdab00 tx=0x7ff744019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f2fcea23ac373a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff74a000640 1 -- 172.21.15.37:0/4218198431 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744034070 con 0x564ad7bd59a0 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.629+0000 7ff74a000640 1 -- 172.21.15.37:0/4218198431 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74402ed50 con 0x564ad7bd59a0 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff74a000640 1 -- 172.21.15.37:0/4218198431 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744023880 con 0x564ad7bd59a0 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 -- 172.21.15.37:0/4218198431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 msgr2=0x564ad7bdd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bdd9d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x564ad7bdab00 tx=0x7ff744019000 comp rx=0 tx=0).stop 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 -- 172.21.15.37:0/4218198431 shutdown_connections 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bde580 0x564ad7bddf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bdd9d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 --2- 172.21.15.37:0/4218198431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bd5080 0x564ad7bd1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 -- 172.21.15.37:0/4218198431 >> 172.21.15.37:0/4218198431 conn(0x564ad7bcf7b0 msgr2=0x564ad7bcdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 -- 172.21.15.37:0/4218198431 shutdown_connections 2024-03-21T09:19:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.630+0000 7ff7633e7740 1 -- 172.21.15.37:0/4218198431 wait complete. 2024-03-21T09:19:46.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 -- start start 2024-03-21T09:19:46.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 Processor -- start 2024-03-21T09:19:46.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7be6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bd8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bd8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581af370 con 0x564ad7bd5080 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581b0650 con 0x564ad7bde580 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581af990 con 0x564ad7bd59a0 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.631+0000 7ff7633e7740 10 client.0 _ll_register_callbacks cb 0x7ff74c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7be6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bd8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7be6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45804/0 (socket says 172.21.15.37:45804) 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75dc00640 1 -- 172.21.15.37:0/2037335234 learned_addr learned my addr 172.21.15.37:0/2037335234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75e600640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bd8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 -- 172.21.15.37:0/2037335234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 msgr2=0x564ad7bd8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bd8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 -- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 msgr2=0x564ad7be6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7be6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 -- 172.21.15.37:0/2037335234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ad7bd90b0 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75dc00640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7be6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff75d200640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bd8500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff744004260 tx=0x7ff7440043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9afd31a85e2780 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.632+0000 7ff73f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff73f400640 1 -- 172.21.15.37:0/2037335234 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74403c020 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff73f400640 1 -- 172.21.15.37:0/2037335234 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff744005240 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff74402ed50 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff73f400640 1 -- 172.21.15.37:0/2037335234 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff744034040 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff7633e7740 10 client.27385 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564ad7c4aa60 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff73f400640 1 -- 172.21.15.37:0/2037335234 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74402ed50 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff73f400640 1 -- 172.21.15.37:0/2037335234 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff744037070 con 0x564ad7bd59a0 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff7633e7740 10 client.27385 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.633+0000 7ff7633e7740 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff7633e7740 20 client.27385 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff7633e7740 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7c4b090 0x564ad7c4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff7633e7740 4 client.27385 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:19:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564ad7b41740 con 0x564ad7c4b090 2024-03-21T09:19:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff75dc00640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7c4b090 0x564ad7c4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff75dc00640 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7c4b090 0x564ad7c4d4b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a5c92641ddf33c server_cookie=31a3a0a214800b0b in_seq=0 out_seq=0 2024-03-21T09:19:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.634+0000 7ff73f400640 10 client.27385 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff73f400640 1 -- 172.21.15.37:0/2037335234 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564ad7b41740 con 0x564ad7c4b090 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff73f400640 10 client.27385 handle_command_reply: tid=0 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 client.27385 shutdown 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 msgr2=0x564ad7bd8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bd8500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff744004260 tx=0x7ff7440043b0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 shutdown_connections 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x564ad7c4b090 0x564ad7c4d4b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ad7bde580 0x564ad7bd8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ad7bd59a0 0x564ad7bd8500 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 --2- 172.21.15.37:0/2037335234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ad7bd5080 0x564ad7be6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff73f400640 0 client.27385 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.636+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 >> 172.21.15.37:0/2037335234 conn(0x564ad7bcf7b0 msgr2=0x564ad7bcdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.637+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 shutdown_connections 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.637+0000 7ff7633e7740 1 -- 172.21.15.37:0/2037335234 wait complete. 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.637+0000 7ff7633e7740 20 client.27385 trim_cache size 0 max 16384 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "6b4c9d91-db7d-4c1f-b8d9-3abee2ae7d94", 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:19:46.635 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:19:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7400a3110 msgr2=0x7ff7400a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7400a3110 0x7ff7400a55d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff744005f60 tx=0x7ff7440096a0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 msgr2=0x7ff75819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75819e9b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff75002c7a0 tx=0x7ff750035e00 comp rx=0 tx=0).stop 2024-03-21T09:19:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 shutdown_connections 2024-03-21T09:19:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff75810bc90 0x7ff75819e9b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7581031e0 0x7ff75819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff7400a3110 0x7ff7400a55d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 --2- 172.21.15.37:0/3957244790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7581028c0 0x7ff75819a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 >> 172.21.15.37:0/3957244790 conn(0x7ff758077750 msgr2=0x7ff758107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 shutdown_connections 2024-03-21T09:19:46.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.638+0000 7ff760600640 1 -- 172.21.15.37:0/3957244790 wait complete. 2024-03-21T09:19:46.649 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:46 smithi037 ceph-mon[31830]: pgmap v232: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:46.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:19:46.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:19:46.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:46 smithi053 ceph-mon[36169]: pgmap v232: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.870+0000 7ffb6e600640 1 -- 172.21.15.37:0/3382534839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb68102630 msgr2=0x7ffb68102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.870+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3382534839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb68102630 0x7ffb68102a10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009900 tx=0x7ffb5c02bcd0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 -- 172.21.15.37:0/3382534839 shutdown_connections 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3382534839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb6810ba50 0x7ffb6810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3382534839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6810b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3382534839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb68102630 0x7ffb68102a10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 -- 172.21.15.37:0/3382534839 >> 172.21.15.37:0/3382534839 conn(0x7ffb68077130 msgr2=0x7ffb680fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 -- 172.21.15.37:0/3382534839 shutdown_connections 2024-03-21T09:19:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.871+0000 7ffb6e600640 1 -- 172.21.15.37:0/3382534839 wait complete. 2024-03-21T09:19:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.872+0000 7ffb6e600640 1 Processor -- start 2024-03-21T09:19:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.872+0000 7ffb6e600640 1 -- start start 2024-03-21T09:19:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb68102630 0x7ffb68199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb6810ba50 0x7ffb6819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68111850 con 0x7ffb68102630 2024-03-21T09:19:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6e600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681116d0 con 0x7ffb6810ba50 2024-03-21T09:19:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681119d0 con 0x7ffb68102f50 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb68102630 0x7ffb68199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55514/0 (socket says 172.21.15.37:55514) 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb67e00640 1 -- 172.21.15.37:0/3321859348 learned_addr learned my addr 172.21.15.37:0/3321859348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.873+0000 7ffb6dc00640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb6810ba50 0x7ffb6819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb67e00640 1 -- 172.21.15.37:0/3321859348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb6810ba50 msgr2=0x7ffb6819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb67e00640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb6810ba50 0x7ffb6819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb67e00640 1 -- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb68102630 msgr2=0x7ffb68199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb67e00640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb68102630 0x7ffb68199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb67e00640 1 -- 172.21.15.37:0/3321859348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6819efa0 con 0x7ffb68102f50 2024-03-21T09:19:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb67e00640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6819a4f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400d950 tx=0x7ffb5400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb65600640 1 -- 172.21.15.37:0/3321859348 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54017070 con 0x7ffb68102f50 2024-03-21T09:19:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb65600640 1 -- 172.21.15.37:0/3321859348 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb54009c30 con 0x7ffb68102f50 2024-03-21T09:19:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6819f290 con 0x7ffb68102f50 2024-03-21T09:19:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb65600640 1 -- 172.21.15.37:0/3321859348 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400f6b0 con 0x7ffb68102f50 2024-03-21T09:19:46.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.874+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb681a6b20 con 0x7ffb68102f50 2024-03-21T09:19:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.878+0000 7ffb65600640 1 -- 172.21.15.37:0/3321859348 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ffb540109d0 con 0x7ffb68102f50 2024-03-21T09:19:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.879+0000 7ffb7299e740 1 Processor -- start 2024-03-21T09:19:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 -- start start 2024-03-21T09:19:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb65600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffb500a3110 0x7ffb500a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb65600640 1 -- 172.21.15.37:0/3321859348 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ffb540cf790 con 0x7ffb68102f50 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b399b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb6dc00640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffb500a3110 0x7ffb500a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b39a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b3a3460 0x56324b3a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b2fe250 con 0x56324b39add0 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b2a6a30 con 0x56324b39a0f0 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb7299e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b2fa2f0 con 0x56324b3a3460 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.880+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b39a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b39a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36210/0 (socket says 172.21.15.37:36210) 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb6dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b3a3460 0x56324b3a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b399b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 -- 172.21.15.37:0/2124795335 learned_addr learned my addr 172.21.15.37:0/2124795335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 -- 172.21.15.37:0/2124795335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b3a3460 msgr2=0x56324b3a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b3a3460 0x56324b3a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 -- 172.21.15.37:0/2124795335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 msgr2=0x56324b399b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b399b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 -- 172.21.15.37:0/2124795335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324b291690 con 0x56324b39a0f0 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb6dc00640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffb500a3110 0x7ffb500a55d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009fd0 tx=0x7ffb60014b70 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb67e00640 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b39a570 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffb540981a0 tx=0x7ffb54005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a61fc69c491d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb5a000640 1 -- 172.21.15.37:0/2124795335 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54020040 con 0x56324b39a0f0 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.881+0000 7ffb5a000640 1 -- 172.21.15.37:0/2124795335 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb5409f870 con 0x56324b39a0f0 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.882+0000 7ffb5a000640 1 -- 172.21.15.37:0/2124795335 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb540cce90 con 0x56324b39a0f0 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.882+0000 7ffb7299e740 1 -- 172.21.15.37:0/2124795335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 msgr2=0x56324b39a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.882+0000 7ffb7299e740 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b39a570 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffb540981a0 tx=0x7ffb54005f90 comp rx=0 tx=0).stop 2024-03-21T09:19:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 -- 172.21.15.37:0/2124795335 shutdown_connections 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b3a3460 0x56324b3a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b39a570 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 --2- 172.21.15.37:0/2124795335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b399b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 -- 172.21.15.37:0/2124795335 >> 172.21.15.37:0/2124795335 conn(0x56324b361200 msgr2=0x56324b394b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 -- 172.21.15.37:0/2124795335 shutdown_connections 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 -- 172.21.15.37:0/2124795335 wait complete. 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.883+0000 7ffb7299e740 1 -- start start 2024-03-21T09:19:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 Processor -- start 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b3a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39add0 0x56324b39feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b3a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b3a62d0 con 0x56324b3a3460 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b2b9e80 con 0x56324b39a0f0 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b3a6450 con 0x56324b39add0 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.884+0000 7ffb7299e740 10 client.0 _ll_register_callbacks cb 0x56324b2cc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b3a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b3a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36216/0 (socket says 172.21.15.37:36216) 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 -- 172.21.15.37:0/1790518225 learned_addr learned my addr 172.21.15.37:0/1790518225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb67e00640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39add0 0x56324b39feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6dc00640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b3a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 -- 172.21.15.37:0/1790518225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39add0 msgr2=0x56324b39feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39add0 0x56324b39feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 -- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 msgr2=0x56324b3a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b3a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 -- 172.21.15.37:0/1790518225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324b0dc7d0 con 0x56324b39a0f0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb67e00640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39add0 0x56324b39feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb6d200640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b3a2b20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006010 tx=0x7ffb5c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7505fd85e62c4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.885+0000 7ffb4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb4ea00640 1 -- 172.21.15.37:0/1790518225 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c044070 con 0x56324b39a0f0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb5409f870 con 0x56324b39a0f0 2024-03-21T09:19:46.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb4ea00640 1 -- 172.21.15.37:0/1790518225 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ffb5c0046c0 con 0x56324b39a0f0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb4ea00640 1 -- 172.21.15.37:0/1790518225 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb5c0048c0 con 0x56324b39a0f0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb7299e740 10 client.27368 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56324b0123f0 con 0x56324b39a0f0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb4ea00640 1 -- 172.21.15.37:0/1790518225 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb5409f870 con 0x56324b39a0f0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb4ea00640 1 -- 172.21.15.37:0/1790518225 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ffb5c003850 con 0x56324b39a0f0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.886+0000 7ffb7299e740 10 client.27368 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb7299e740 10 client.27368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb7299e740 20 client.27368 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b3b33b0 0x56324b3b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb7299e740 4 client.27368 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56324b30b790 con 0x56324b3b33b0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb67e00640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b3b33b0 0x56324b3b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb67e00640 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b3b33b0 0x56324b3b57a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab9cc59cd498a45 server_cookie=1da7a19c5a282ccf in_seq=0 out_seq=0 2024-03-21T09:19:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.887+0000 7ffb4ea00640 10 client.27368 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb4ea00640 1 -- 172.21.15.37:0/1790518225 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56324b30b790 con 0x56324b3b33b0 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb4ea00640 10 client.27368 handle_command_reply: tid=0 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 client.27368 shutdown 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 msgr2=0x56324b3a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b3a2b20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006010 tx=0x7ffb5c004610 comp rx=0 tx=0).stop 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 shutdown_connections 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b3a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39add0 0x56324b39feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b3b33b0 0x56324b3b57a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1790518225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39a0f0 0x56324b3a2b20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.888+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 >> 172.21.15.37:0/1790518225 conn(0x56324b361200 msgr2=0x56324b396890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.889+0000 7ffb4ea00640 0 client.27368 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.889+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 shutdown_connections 2024-03-21T09:19:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.889+0000 7ffb7299e740 1 -- 172.21.15.37:0/1790518225 wait complete. 2024-03-21T09:19:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.889+0000 7ffb7299e740 20 client.27368 trim_cache size 0 max 16384 2024-03-21T09:19:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 Processor -- start 2024-03-21T09:19:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 -- start start 2024-03-21T09:19:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b3a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b3a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b3a3460 0x56324b3a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681b3060 con 0x56324b39add0 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681b0b60 con 0x56324b3a3460 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.914+0000 7ffb7299e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681aec20 con 0x56324b39a0f0 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b3a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b3a3460 0x56324b3a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b3a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55534/0 (socket says 172.21.15.37:55534) 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 -- 172.21.15.37:0/1243466208 learned_addr learned my addr 172.21.15.37:0/1243466208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb67e00640 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b3a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 -- 172.21.15.37:0/1243466208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b3a3460 msgr2=0x56324b3a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b3a3460 0x56324b3a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 -- 172.21.15.37:0/1243466208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 msgr2=0x56324b3a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b3a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6d200640 1 -- 172.21.15.37:0/1243466208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324afd20f0 con 0x56324b39a0f0 2024-03-21T09:19:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.915+0000 7ffb6dc00640 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b3a3460 0x56324b3a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb6d200640 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b3a25e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c02c210 tx=0x7ffb5c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68025ddb46e0281 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb59600640 1 -- 172.21.15.37:0/1243466208 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c044070 con 0x56324b39a0f0 2024-03-21T09:19:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb59600640 1 -- 172.21.15.37:0/1243466208 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c0043e0 con 0x56324b39a0f0 2024-03-21T09:19:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb59600640 1 -- 172.21.15.37:0/1243466208 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0052f0 con 0x56324b39a0f0 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 -- 172.21.15.37:0/1243466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 msgr2=0x56324b3a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b3a25e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c02c210 tx=0x7ffb5c001e70 comp rx=0 tx=0).stop 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 -- 172.21.15.37:0/1243466208 shutdown_connections 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b3a3460 0x56324b3a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b39add0 0x56324b3a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 --2- 172.21.15.37:0/1243466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b3a25e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.916+0000 7ffb7299e740 1 -- 172.21.15.37:0/1243466208 >> 172.21.15.37:0/1243466208 conn(0x56324b361200 msgr2=0x56324b394670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.917+0000 7ffb7299e740 1 -- 172.21.15.37:0/1243466208 shutdown_connections 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.917+0000 7ffb7299e740 1 -- 172.21.15.37:0/1243466208 wait complete. 2024-03-21T09:19:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.917+0000 7ffb7299e740 1 -- start start 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.917+0000 7ffb7299e740 1 Processor -- start 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b39d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39add0 0x56324b39c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b39c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681af220 con 0x56324b3a3460 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681b0500 con 0x56324b39add0 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681af840 con 0x56324b39a0f0 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb7299e740 10 client.0 _ll_register_callbacks cb 0x7ffb5c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b39d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b39d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55548/0 (socket says 172.21.15.37:55548) 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 -- 172.21.15.37:0/335011089 learned_addr learned my addr 172.21.15.37:0/335011089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb67e00640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39add0 0x56324b39c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6dc00640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b39c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 -- 172.21.15.37:0/335011089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39add0 msgr2=0x56324b39c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39add0 0x56324b39c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 -- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 msgr2=0x56324b39c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.918+0000 7ffb6d200640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b39c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb6d200640 1 -- 172.21.15.37:0/335011089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324b39cba0 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb67e00640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39add0 0x56324b39c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb6d200640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b39d7f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c004e90 tx=0x7ffb5c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61d9206a649478e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb4ea00640 1 -- 172.21.15.37:0/335011089 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c044070 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb5c0043e0 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb4ea00640 1 -- 172.21.15.37:0/335011089 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ffb5c0037f0 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb4ea00640 1 -- 172.21.15.37:0/335011089 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb5c066050 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb7299e740 10 client.27403 fetch_fsmap learned FSMap version 30 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.919+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56324b40e880 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb4ea00640 1 -- 172.21.15.37:0/335011089 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c0043e0 con 0x56324b39a0f0 2024-03-21T09:19:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb4ea00640 1 -- 172.21.15.37:0/335011089 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ffb5c034720 con 0x56324b39a0f0 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb7299e740 10 client.27403 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb7299e740 10 client.27403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb7299e740 20 client.27403 populate_metadata read hostname 'smithi037' 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb7299e740 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b40ee60 0x56324b411280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb7299e740 4 client.27403 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56324b305ae0 con 0x56324b40ee60 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.920+0000 7ffb67e00640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b40ee60 0x56324b411280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb67e00640 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b40ee60 0x56324b411280 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4135e4a7719b29 server_cookie=4150c0fd341a89ba in_seq=0 out_seq=0 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb4ea00640 10 client.27403 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb4ea00640 1 -- 172.21.15.37:0/335011089 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56324b305ae0 con 0x56324b40ee60 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb4ea00640 10 client.27403 handle_command_reply: tid=0 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 client.27403 shutdown 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 msgr2=0x56324b39d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b39d7f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c004e90 tx=0x7ffb5c004610 comp rx=0 tx=0).stop 2024-03-21T09:19:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 shutdown_connections 2024-03-21T09:19:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56324b3a3460 0x56324b39c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56324b40ee60 0x56324b411280 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56324b39add0 0x56324b39c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 --2- 172.21.15.37:0/335011089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56324b39a0f0 0x56324b39d7f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb4ea00640 0 client.27403 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:19:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.921+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 >> 172.21.15.37:0/335011089 conn(0x56324b361200 msgr2=0x56324b39e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 shutdown_connections 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb7299e740 1 -- 172.21.15.37:0/335011089 wait complete. 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb7299e740 20 client.27403 trim_cache size 0 max 16384 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "6b4c9d91-db7d-4c1f-b8d9-3abee2ae7d94": { 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:19:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "6b4c9d91-db7d-4c1f-b8d9-3abee2ae7d94", 2024-03-21T09:19:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:19:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:19:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:19:46.923 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:19:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffb500a3110 msgr2=0x7ffb500a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffb500a3110 0x7ffb500a55d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009fd0 tx=0x7ffb60014b70 comp rx=0 tx=0).stop 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 msgr2=0x7ffb6819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6819a4f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400d950 tx=0x7ffb5400de10 comp rx=0 tx=0).stop 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 shutdown_connections 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.922+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffb500a3110 0x7ffb500a55d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.923+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb6810ba50 0x7ffb6819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.923+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb68102f50 0x7ffb6819a4f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.923+0000 7ffb6e600640 1 --2- 172.21.15.37:0/3321859348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb68102630 0x7ffb68199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.923+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 >> 172.21.15.37:0/3321859348 conn(0x7ffb68077130 msgr2=0x7ffb680fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.923+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 shutdown_connections 2024-03-21T09:19:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:19:46.923+0000 7ffb6e600640 1 -- 172.21.15.37:0/3321859348 wait complete. 2024-03-21T09:19:46.935 INFO:tasks.cephfs.filesystem:scrub status for tag:6b4c9d91-db7d-4c1f-b8d9-3abee2ae7d94 - {'path': '/', 'tag': '6b4c9d91-db7d-4c1f-b8d9-3abee2ae7d94', 'options': 'recursive,force'} 2024-03-21T09:19:48.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:48 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:19:48.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:48 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:48.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:48 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:19:48.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:48 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:48.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:48 smithi071 ceph-mon[36597]: pgmap v233: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:48.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:48 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:19:48.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:48 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:48.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:48 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:19:48.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:48 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:48.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:48 smithi037 ceph-mon[31830]: pgmap v233: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:48.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:48 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:19:48.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:48 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:48.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:48 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:19:48.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:48 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:19:48.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:48 smithi053 ceph-mon[36169]: pgmap v233: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:50.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:50 smithi071 ceph-mon[36597]: pgmap v234: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:50.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:50 smithi037 ceph-mon[31830]: pgmap v234: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:50.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:50 smithi053 ceph-mon[36169]: pgmap v234: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:52.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:52 smithi071 ceph-mon[36597]: pgmap v235: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:52.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:52 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:19:52.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:52 smithi037 ceph-mon[31830]: pgmap v235: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:52.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:52 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:19:52.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:52 smithi053 ceph-mon[36169]: pgmap v235: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:52.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:52 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:19:53.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:53 smithi071 ceph-mon[36597]: pgmap v236: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:53.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:53 smithi037 ceph-mon[31830]: pgmap v236: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:53.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:53 smithi053 ceph-mon[36169]: pgmap v236: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:56.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:56 smithi071 ceph-mon[36597]: pgmap v237: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:56.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:56 smithi037 ceph-mon[31830]: pgmap v237: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:56.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:56 smithi053 ceph-mon[36169]: pgmap v237: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:19:58.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:19:58 smithi071 ceph-mon[36597]: pgmap v238: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:58.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:19:58 smithi037 ceph-mon[31830]: pgmap v238: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:19:58.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:19:58 smithi053 ceph-mon[36169]: pgmap v238: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:00.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:00 smithi071 ceph-mon[36597]: pgmap v239: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:00.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:00 smithi071 ceph-mon[36597]: overall HEALTH_OK 2024-03-21T09:20:00.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:00 smithi037 ceph-mon[31830]: pgmap v239: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:00.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:00 smithi037 ceph-mon[31830]: overall HEALTH_OK 2024-03-21T09:20:00.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:00 smithi053 ceph-mon[36169]: pgmap v239: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:00.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:00 smithi053 ceph-mon[36169]: overall HEALTH_OK 2024-03-21T09:20:02.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:02 smithi071 ceph-mon[36597]: pgmap v240: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:02.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:02 smithi037 ceph-mon[31830]: pgmap v240: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:02.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:02 smithi053 ceph-mon[36169]: pgmap v240: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:04.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:04 smithi071 ceph-mon[36597]: pgmap v241: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:20:04.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:04 smithi037 ceph-mon[31830]: pgmap v241: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:20:04.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:04 smithi053 ceph-mon[36169]: pgmap v241: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:20:06.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:06 smithi071 ceph-mon[36597]: pgmap v242: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:20:06.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:06 smithi037 ceph-mon[31830]: pgmap v242: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:20:06.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:06 smithi053 ceph-mon[36169]: pgmap v242: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:20:07.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:07 smithi037 ceph-mon[31830]: pgmap v243: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:20:07.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:07 smithi037 ceph-mon[31830]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T09:20:07.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:07 smithi053 ceph-mon[36169]: pgmap v243: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:20:07.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:07 smithi053 ceph-mon[36169]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T09:20:07.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:07 smithi071 ceph-mon[36597]: pgmap v243: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:20:07.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:07 smithi071 ceph-mon[36597]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T09:20:10.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:10 smithi071 ceph-mon[36597]: pgmap v245: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:20:10.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:10 smithi037 ceph-mon[31830]: pgmap v245: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:20:10.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:10 smithi053 ceph-mon[36169]: pgmap v245: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:20:12.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:12 smithi071 ceph-mon[36597]: pgmap v246: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:20:12.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:12 smithi037 ceph-mon[31830]: pgmap v246: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:20:12.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:12 smithi053 ceph-mon[36169]: pgmap v246: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:20:14.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:14 smithi071 ceph-mon[36597]: pgmap v247: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:20:14.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:14 smithi037 ceph-mon[31830]: pgmap v247: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:20:14.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:14 smithi053 ceph-mon[36169]: pgmap v247: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:20:16.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:16 smithi071 ceph-mon[36597]: pgmap v248: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:20:16.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:16 smithi071 ceph-mon[36597]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T09:20:16.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:16 smithi037 ceph-mon[31830]: pgmap v248: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:20:16.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:16 smithi037 ceph-mon[31830]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T09:20:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:16 smithi053 ceph-mon[36169]: pgmap v248: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:20:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:16 smithi053 ceph-mon[36169]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T09:20:16.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:20:16.936 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.172+0000 7f12bc000640 1 -- 172.21.15.37:0/2082100944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4073a70 msgr2=0x7f12b4073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.172+0000 7f12bc000640 1 --2- 172.21.15.37:0/2082100944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4073a70 0x7f12b4073e50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009900 tx=0x7f12a802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 -- 172.21.15.37:0/2082100944 shutdown_connections 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 --2- 172.21.15.37:0/2082100944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 0x7f12b41102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 --2- 172.21.15.37:0/2082100944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4074390 0x7f12b410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 --2- 172.21.15.37:0/2082100944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4073a70 0x7f12b4073e50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 -- 172.21.15.37:0/2082100944 >> 172.21.15.37:0/2082100944 conn(0x7f12b40fd130 msgr2=0x7f12b40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 -- 172.21.15.37:0/2082100944 shutdown_connections 2024-03-21T09:20:17.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.173+0000 7f12bc000640 1 -- 172.21.15.37:0/2082100944 wait complete. 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 Processor -- start 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 -- start start 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4073a70 0x7f12b419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 0x7f12b419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 0x7f12b419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4113c40 con 0x7f12b4073a70 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4113ac0 con 0x7f12b4074390 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.174+0000 7f12bc000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4113dc0 con 0x7f12b410dee0 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 0x7f12b419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 0x7f12b419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42486/0 (socket says 172.21.15.37:42486) 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 -- 172.21.15.37:0/2410533637 learned_addr learned my addr 172.21.15.37:0/2410533637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 0x7f12b419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b9600640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4073a70 0x7f12b419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 -- 172.21.15.37:0/2410533637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 msgr2=0x7f12b419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 0x7f12b419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 -- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4073a70 msgr2=0x7f12b419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4073a70 0x7f12b419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 -- 172.21.15.37:0/2410533637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b419f190 con 0x7f12b4074390 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b9600640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4073a70 0x7f12b419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12ba000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 0x7f12b419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.175+0000 7f12b8c00640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 0x7f12b419a6e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f12a000e9f0 tx=0x7f12a000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.176+0000 7f12b2000640 1 -- 172.21.15.37:0/2410533637 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a000ccc0 con 0x7f12b4074390 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.176+0000 7f12b2000640 1 -- 172.21.15.37:0/2410533637 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a0010910 con 0x7f12b4074390 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.176+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b419f480 con 0x7f12b4074390 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.176+0000 7f12b2000640 1 -- 172.21.15.37:0/2410533637 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00188a0 con 0x7f12b4074390 2024-03-21T09:20:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.176+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b41a6d10 con 0x7f12b4074390 2024-03-21T09:20:17.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.180+0000 7f12bec18740 1 Processor -- start 2024-03-21T09:20:17.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.180+0000 7f12bec18740 1 -- start start 2024-03-21T09:20:17.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.180+0000 7f12b2000640 1 -- 172.21.15.37:0/2410533637 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f12a0010ac0 con 0x7f12b4074390 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12bec18740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959acc00 0x5582959ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12bec18740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 0x5582959ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12bec18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x5582959b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12bec18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558295910250 con 0x5582959b5290 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12bec18740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582958b8a30 con 0x5582959abf20 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12bec18740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829590c2f0 con 0x5582959acc00 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12b9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 0x5582959ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x5582959b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12b9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 0x5582959ac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42488/0 (socket says 172.21.15.37:42488) 2024-03-21T09:20:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12b9600640 1 -- 172.21.15.37:0/860023191 learned_addr learned my addr 172.21.15.37:0/860023191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.181+0000 7f12ba000640 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959acc00 0x5582959ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12b9600640 1 -- 172.21.15.37:0/860023191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959acc00 msgr2=0x5582959ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12b9600640 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959acc00 0x5582959ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12b9600640 1 -- 172.21.15.37:0/860023191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 msgr2=0x5582959b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12b9600640 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x5582959b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12b9600640 1 -- 172.21.15.37:0/860023191 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582958a3690 con 0x5582959abf20 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12ba000640 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959acc00 0x5582959ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12b9600640 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 0x5582959ac3a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009900 tx=0x7f12a8005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76498fd755e09c5f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12a6600640 1 -- 172.21.15.37:0/860023191 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8042070 con 0x5582959abf20 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12a6600640 1 -- 172.21.15.37:0/860023191 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a802cb20 con 0x5582959abf20 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.182+0000 7f12a6600640 1 -- 172.21.15.37:0/860023191 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80046c0 con 0x5582959abf20 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12b2000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f129c0a3090 0x7f129c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12b2000640 1 -- 172.21.15.37:0/2410533637 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f12a00d0130 con 0x7f12b4074390 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12ba000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f129c0a3090 0x7f129c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 -- 172.21.15.37:0/860023191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 msgr2=0x5582959ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 0x5582959ac3a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009900 tx=0x7f12a8005dd0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 -- 172.21.15.37:0/860023191 shutdown_connections 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x5582959b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959abf20 0x5582959ac3a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 --2- 172.21.15.37:0/860023191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959acc00 0x5582959ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 -- 172.21.15.37:0/860023191 >> 172.21.15.37:0/860023191 conn(0x558295973200 msgr2=0x5582959a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 -- 172.21.15.37:0/860023191 shutdown_connections 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.183+0000 7f12bec18740 1 -- 172.21.15.37:0/860023191 wait complete. 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.184+0000 7f12bec18740 1 -- start start 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.184+0000 7f12ba000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f129c0a3090 0x7f129c0a5550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0115a0 tx=0x7f12ac009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.184+0000 7f12bec18740 1 Processor -- start 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582959b80a0 con 0x5582959abf20 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582958cbe80 con 0x5582959acc00 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582959b8220 con 0x5582959b5290 2024-03-21T09:20:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12bec18740 10 client.0 _ll_register_callbacks cb 0x5582958de130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42494/0 (socket says 172.21.15.37:42494) 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 -- 172.21.15.37:0/1615511348 learned_addr learned my addr 172.21.15.37:0/1615511348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 -- 172.21.15.37:0/1615511348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 msgr2=0x5582959b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 -- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 msgr2=0x5582959b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12b8c00640 1 -- 172.21.15.37:0/1615511348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582956ee7d0 con 0x5582959acc00 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.185+0000 7f12ba000640 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f12b8c00640 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b1d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f12a00a45a0 tx=0x7f12a000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b177793d711839 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f129aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f129aa00640 1 -- 172.21.15.37:0/1615511348 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00cd8a0 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f129aa00640 1 -- 172.21.15.37:0/1615511348 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f12a00bc0d0 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12a802cb20 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f129aa00640 1 -- 172.21.15.37:0/1615511348 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a00bc2d0 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f12bec18740 10 client.27386 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5582956243f0 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.186+0000 7f129aa00640 1 -- 172.21.15.37:0/1615511348 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a802cb20 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f129aa00640 1 -- 172.21.15.37:0/1615511348 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f12a00d20f0 con 0x5582959acc00 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12bec18740 10 client.27386 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12bec18740 10 client.27386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12bec18740 20 client.27386 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12bec18740 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5582959c4fd0 0x5582959c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12bec18740 4 client.27386 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55829591d790 con 0x5582959c4fd0 2024-03-21T09:20:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.187+0000 7f12b9600640 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5582959c4fd0 0x5582959c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.188+0000 7f12b9600640 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5582959c4fd0 0x5582959c73c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d5a79c8e879c0ec server_cookie=66c2368a960e552f in_seq=0 out_seq=0 2024-03-21T09:20:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.188+0000 7f129aa00640 10 client.27386 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.188+0000 7f129aa00640 1 -- 172.21.15.37:0/1615511348 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55829591d790 con 0x5582959c4fd0 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.188+0000 7f129aa00640 10 client.27386 handle_command_reply: tid=0 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.188+0000 7f12bec18740 1 client.27386 shutdown 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 msgr2=0x5582959b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b1d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f12a00a45a0 tx=0x7f12a000ce60 comp rx=0 tx=0).stop 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 shutdown_connections 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5582959c4fd0 0x5582959c73c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b1d10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 --2- 172.21.15.37:0/1615511348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 >> 172.21.15.37:0/1615511348 conn(0x558295973200 msgr2=0x5582959a86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f129aa00640 0 client.27386 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 shutdown_connections 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 1 -- 172.21.15.37:0/1615511348 wait complete. 2024-03-21T09:20:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.189+0000 7f12bec18740 20 client.27386 trim_cache size 0 max 16384 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 Processor -- start 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 -- start start 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41b3230 con 0x5582959abf20 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41b0d30 con 0x5582959acc00 2024-03-21T09:20:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12bec18740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41aedf0 con 0x5582959b5290 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12b9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42504/0 (socket says 172.21.15.37:42504) 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.207+0000 7f12b8c00640 1 -- 172.21.15.37:0/127172325 learned_addr learned my addr 172.21.15.37:0/127172325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12b8c00640 1 -- 172.21.15.37:0/127172325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 msgr2=0x5582959b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12b8c00640 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12b8c00640 1 -- 172.21.15.37:0/127172325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 msgr2=0x5582959b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12b8c00640 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12b8c00640 1 -- 172.21.15.37:0/127172325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582955e40f0 con 0x5582959acc00 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12b8c00640 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b4980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f12a0000c00 tx=0x7f12a009a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ed233e659582af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.208+0000 7f12a5c00640 1 -- 172.21.15.37:0/127172325 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a009c7c0 con 0x5582959acc00 2024-03-21T09:20:17.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12a5c00640 1 -- 172.21.15.37:0/127172325 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a00a0210 con 0x5582959acc00 2024-03-21T09:20:17.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12a5c00640 1 -- 172.21.15.37:0/127172325 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00cd6a0 con 0x5582959acc00 2024-03-21T09:20:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 -- 172.21.15.37:0/127172325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 msgr2=0x5582959b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b4980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f12a0000c00 tx=0x7f12a009a5d0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 -- 172.21.15.37:0/127172325 shutdown_connections 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959b5290 0x5582959b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b4980 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 --2- 172.21.15.37:0/127172325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959abf20 0x5582959b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.209+0000 7f12bec18740 1 -- 172.21.15.37:0/127172325 >> 172.21.15.37:0/127172325 conn(0x558295973200 msgr2=0x5582959b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.210+0000 7f12bec18740 1 -- 172.21.15.37:0/127172325 shutdown_connections 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.210+0000 7f12bec18740 1 -- 172.21.15.37:0/127172325 wait complete. 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.210+0000 7f12bec18740 1 -- start start 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.210+0000 7f12bec18740 1 Processor -- start 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959abf20 0x5582959b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x558295a19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41af3f0 con 0x5582959b5290 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41b06d0 con 0x5582959acc00 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41afa10 con 0x5582959abf20 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12bec18740 10 client.0 _ll_register_callbacks cb 0x7f12a00d38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12ba000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x558295a19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959abf20 0x5582959b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42506/0 (socket says 172.21.15.37:42506) 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 -- 172.21.15.37:0/2295507813 learned_addr learned my addr 172.21.15.37:0/2295507813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 -- 172.21.15.37:0/2295507813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959abf20 msgr2=0x5582959b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959abf20 0x5582959b9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 -- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 msgr2=0x558295a19820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x558295a19820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.211+0000 7f12b8c00640 1 -- 172.21.15.37:0/2295507813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558295a19de0 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12b8c00640 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b96c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c6a0 tx=0x7f12a00d3cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5562dc9a45ab3a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f129aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f129aa00640 1 -- 172.21.15.37:0/2295507813 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00cd8b0 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f129aa00640 1 -- 172.21.15.37:0/2295507813 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f12a00b5120 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12a00a0210 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f129aa00640 1 -- 172.21.15.37:0/2295507813 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a00a4ad0 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 10 client.27392 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558295a19de0 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f129aa00640 1 -- 172.21.15.37:0/2295507813 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a00a0210 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f129aa00640 1 -- 172.21.15.37:0/2295507813 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f12a0097e10 con 0x5582959acc00 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 10 client.27392 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 20 client.27392 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558295a21430 0x558295a23850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 4 client.27392 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:20:17.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.212+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558295917ae0 con 0x558295a21430 2024-03-21T09:20:17.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.213+0000 7f12b9600640 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558295a21430 0x558295a23850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.213+0000 7f12b9600640 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558295a21430 0x558295a23850 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9096ea2f85786016 server_cookie=df5a95396a8f04b7 in_seq=0 out_seq=0 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.213+0000 7f129aa00640 10 client.27392 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.213+0000 7f129aa00640 1 -- 172.21.15.37:0/2295507813 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558295917ae0 con 0x558295a21430 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.213+0000 7f129aa00640 10 client.27392 handle_command_reply: tid=0 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.213+0000 7f12bec18740 1 client.27392 shutdown 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 msgr2=0x5582959b96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b96c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c6a0 tx=0x7f12a00d3cd0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 shutdown_connections 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582959b5290 0x558295a19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x558295a21430 0x558295a23850 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582959acc00 0x5582959b96c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 --2- 172.21.15.37:0/2295507813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582959abf20 0x5582959b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 >> 172.21.15.37:0/2295507813 conn(0x558295973200 msgr2=0x5582959a7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f129aa00640 0 client.27392 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 shutdown_connections 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 1 -- 172.21.15.37:0/2295507813 wait complete. 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.214+0000 7f12bec18740 20 client.27392 trim_cache size 0 max 16384 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:20:17.215 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f129c0a3090 msgr2=0x7f129c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f129c0a3090 0x7f129c0a5550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0115a0 tx=0x7f12ac009500 comp rx=0 tx=0).stop 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 msgr2=0x7f12b419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 0x7f12b419a6e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f12a000e9f0 tx=0x7f12a000edf0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 shutdown_connections 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12b410dee0 0x7f12b419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f129c0a3090 0x7f129c0a5550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12b4074390 0x7f12b419a6e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 --2- 172.21.15.37:0/2410533637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12b4073a70 0x7f12b419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 >> 172.21.15.37:0/2410533637 conn(0x7f12b40fd130 msgr2=0x7f12b40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 shutdown_connections 2024-03-21T09:20:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.215+0000 7f12bc000640 1 -- 172.21.15.37:0/2410533637 wait complete. 2024-03-21T09:20:17.231 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:20:17.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.465+0000 7fe705000640 1 -- 172.21.15.37:0/2012246812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700103760 msgr2=0x7fe700103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.465+0000 7fe705000640 1 --2- 172.21.15.37:0/2012246812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700103760 0x7fe700103be0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009910 tx=0x7fe6e802bc20 comp rx=0 tx=0).stop 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.465+0000 7fe705000640 1 -- 172.21.15.37:0/2012246812 shutdown_connections 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.465+0000 7fe705000640 1 --2- 172.21.15.37:0/2012246812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe700104120 0x7fe700110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.465+0000 7fe705000640 1 --2- 172.21.15.37:0/2012246812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700103760 0x7fe700103be0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.465+0000 7fe705000640 1 --2- 172.21.15.37:0/2012246812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe700109770 0x7fe700109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.466+0000 7fe705000640 1 -- 172.21.15.37:0/2012246812 >> 172.21.15.37:0/2012246812 conn(0x7fe7000ff1d0 msgr2=0x7fe7001015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.466+0000 7fe705000640 1 -- 172.21.15.37:0/2012246812 shutdown_connections 2024-03-21T09:20:17.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.466+0000 7fe705000640 1 -- 172.21.15.37:0/2012246812 wait complete. 2024-03-21T09:20:17.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.467+0000 7fe705000640 1 Processor -- start 2024-03-21T09:20:17.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.468+0000 7fe705000640 1 -- start start 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.468+0000 7fe705000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe700103760 0x7fe70019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.468+0000 7fe705000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 0x7fe70019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.468+0000 7fe705000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe700109770 0x7fe70019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe705000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700113b80 con 0x7fe700103760 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe705000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700113a00 con 0x7fe700104120 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe705000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700113d00 con 0x7fe700109770 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 0x7fe70019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6ff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe700103760 0x7fe70019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 0x7fe70019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42536/0 (socket says 172.21.15.37:42536) 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 -- 172.21.15.37:0/2210138815 learned_addr learned my addr 172.21.15.37:0/2210138815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe700109770 0x7fe70019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 -- 172.21.15.37:0/2210138815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe700109770 msgr2=0x7fe70019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe700109770 0x7fe70019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 -- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe700103760 msgr2=0x7fe70019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe700103760 0x7fe70019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.469+0000 7fe6f7e00640 1 -- 172.21.15.37:0/2210138815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70019eff0 con 0x7fe700104120 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.470+0000 7fe6f7e00640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 0x7fe70019a540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8000c00 tx=0x7fe6e80323b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.470+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8041020 con 0x7fe700104120 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.470+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6e802ce50 con 0x7fe700104120 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.470+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70019f280 con 0x7fe700104120 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.470+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8039040 con 0x7fe700104120 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.470+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7001a6b70 con 0x7fe700104120 2024-03-21T09:20:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.471+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70004f470 con 0x7fe700104120 2024-03-21T09:20:17.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.473+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe6e8044020 con 0x7fe700104120 2024-03-21T09:20:17.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.475+0000 7fe6fcc00640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe6e40a3100 0x7fe6e40a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.475+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe6e80f0ff0 con 0x7fe700104120 2024-03-21T09:20:17.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.475+0000 7fe6ff400640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe6e40a3100 0x7fe6e40a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.476+0000 7fe6ff400640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe6e40a3100 0x7fe6e40a55c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005d00 tx=0x7fe6f000b410 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.478+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6e804b020 con 0x7fe700104120 2024-03-21T09:20:17.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.722+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe70010bcd0 con 0x7fe700104120 2024-03-21T09:20:17.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.723+0000 7fe6fcc00640 1 -- 172.21.15.37:0/2210138815 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fe6e80bd7f0 con 0x7fe700104120 2024-03-21T09:20:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:20:17.720 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:20:17.720 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:20:17.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe6e40a3100 msgr2=0x7fe6e40a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe6e40a3100 0x7fe6e40a55c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005d00 tx=0x7fe6f000b410 comp rx=0 tx=0).stop 2024-03-21T09:20:17.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 msgr2=0x7fe70019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 0x7fe70019a540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8000c00 tx=0x7fe6e80323b0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 shutdown_connections 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe700109770 0x7fe70019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe6e40a3100 0x7fe6e40a55c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe700104120 0x7fe70019a540 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 --2- 172.21.15.37:0/2210138815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe700103760 0x7fe70019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 >> 172.21.15.37:0/2210138815 conn(0x7fe7000ff1d0 msgr2=0x7fe7000ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.725+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 shutdown_connections 2024-03-21T09:20:17.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.726+0000 7fe705000640 1 -- 172.21.15.37:0/2210138815 wait complete. 2024-03-21T09:20:17.748 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:20:17.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:20:17.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 -- 172.21.15.37:0/1773528500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2498101e60 msgr2=0x7f249810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 --2- 172.21.15.37:0/1773528500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2498101e60 0x7f249810e320 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2490009cd0 tx=0x7f249002c0f0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 -- 172.21.15.37:0/1773528500 shutdown_connections 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 --2- 172.21.15.37:0/1773528500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2498101e60 0x7f249810e320 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 --2- 172.21.15.37:0/1773528500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24981014a0 0x7f2498101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 --2- 172.21.15.37:0/1773528500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498107560 0x7f2498107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 -- 172.21.15.37:0/1773528500 >> 172.21.15.37:0/1773528500 conn(0x7f24980fcfb0 msgr2=0x7f24980ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 -- 172.21.15.37:0/1773528500 shutdown_connections 2024-03-21T09:20:17.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.976+0000 7f249ca00640 1 -- 172.21.15.37:0/1773528500 wait complete. 2024-03-21T09:20:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.978+0000 7f249ca00640 1 Processor -- start 2024-03-21T09:20:17.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.979+0000 7f249ca00640 1 -- start start 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.979+0000 7f249ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 0x7f249819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.979+0000 7f249ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 0x7f249819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.979+0000 7f249ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2498107560 0x7f2498197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f249ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498111840 con 0x7f2498107560 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f249ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981116c0 con 0x7f2498101e60 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f249ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981119c0 con 0x7f24981014a0 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 0x7f249819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2495e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 0x7f249819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 0x7f249819d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45176/0 (socket says 172.21.15.37:45176) 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 -- 172.21.15.37:0/754226084 learned_addr learned my addr 172.21.15.37:0/754226084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2495e00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 0x7f249819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42552/0 (socket says 172.21.15.37:42552) 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2497200640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2498107560 0x7f2498197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 -- 172.21.15.37:0/754226084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 msgr2=0x7f249819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 0x7f249819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 -- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2498107560 msgr2=0x7f2498197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2498107560 0x7f2498197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2496800640 1 -- 172.21.15.37:0/754226084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498198260 con 0x7f24981014a0 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2495e00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 0x7f249819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.980+0000 7f2497200640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2498107560 0x7f2498197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.981+0000 7f2496800640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 0x7f249819d930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f248800e9f0 tx=0x7f248800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.981+0000 7f248f400640 1 -- 172.21.15.37:0/754226084 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800cd80 con 0x7f24981014a0 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.981+0000 7f248f400640 1 -- 172.21.15.37:0/754226084 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2488005500 con 0x7f24981014a0 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.981+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498198550 con 0x7f24981014a0 2024-03-21T09:20:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.981+0000 7f248f400640 1 -- 172.21.15.37:0/754226084 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488010690 con 0x7f24981014a0 2024-03-21T09:20:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.981+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24981a47f0 con 0x7f24981014a0 2024-03-21T09:20:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.984+0000 7f248f400640 1 -- 172.21.15.37:0/754226084 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f24880046c0 con 0x7f24981014a0 2024-03-21T09:20:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.984+0000 7f24a01dd740 1 Processor -- start 2024-03-21T09:20:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 -- start start 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd05c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd0e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629afc692c0 con 0x5629afd06260 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629afc11c90 con 0x5629afd058b0 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f24a01dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629afc65360 con 0x5629afd0ede0 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f2497200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd05c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f2497200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd05c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42562/0 (socket says 172.21.15.37:42562) 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f2497200640 1 -- 172.21.15.37:0/4078161786 learned_addr learned my addr 172.21.15.37:0/4078161786 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f2496800640 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd0e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.985+0000 7f2495e00640 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2497200640 1 -- 172.21.15.37:0/4078161786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 msgr2=0x5629afd11230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2497200640 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd11230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2497200640 1 -- 172.21.15.37:0/4078161786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 msgr2=0x5629afd0e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2497200640 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd0e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2497200640 1 -- 172.21.15.37:0/4078161786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629afc36080 con 0x5629afd058b0 2024-03-21T09:20:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2495e00640 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd11230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2497200640 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd05c90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f249002c700 tx=0x7f2490005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd54498adc03392b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f248f400640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f247c0a3090 0x7f247c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2483400640 1 -- 172.21.15.37:0/4078161786 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490043020 con 0x5629afd058b0 2024-03-21T09:20:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2483400640 1 -- 172.21.15.37:0/4078161786 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f249002cca0 con 0x5629afd058b0 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.986+0000 7f2483400640 1 -- 172.21.15.37:0/4078161786 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490039040 con 0x5629afd058b0 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f248f400640 1 -- 172.21.15.37:0/754226084 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f24880d0070 con 0x7f24981014a0 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f2496800640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f247c0a3090 0x7f247c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 -- 172.21.15.37:0/4078161786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 msgr2=0x5629afd05c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd05c90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f249002c700 tx=0x7f2490005fa0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 -- 172.21.15.37:0/4078161786 shutdown_connections 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd11230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd0e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4078161786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd05c90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 -- 172.21.15.37:0/4078161786 >> 172.21.15.37:0/4078161786 conn(0x5629afcfff80 msgr2=0x5629afd00360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f2496800640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f247c0a3090 0x7f247c0a5550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5629afd05380 tx=0x7f2488004a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.987+0000 7f24a01dd740 1 -- 172.21.15.37:0/4078161786 shutdown_connections 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.988+0000 7f24a01dd740 1 -- 172.21.15.37:0/4078161786 wait complete. 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.988+0000 7f24a01dd740 1 -- start start 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.988+0000 7f24a01dd740 1 Processor -- start 2024-03-21T09:20:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd18ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd06260 0x5629afd19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd0ede0 0x5629afd132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629afd11cf0 con 0x5629afd0ede0 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629afc250d0 con 0x5629afd058b0 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629afd11e70 con 0x5629afd06260 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f24a01dd740 10 client.0 _ll_register_callbacks cb 0x5629afc29e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2497200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd18ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2497200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd18ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42574/0 (socket says 172.21.15.37:42574) 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2495e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd06260 0x5629afd19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2497200640 1 -- 172.21.15.37:0/4214100802 learned_addr learned my addr 172.21.15.37:0/4214100802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2497200640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd0ede0 0x5629afd132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2495e00640 1 -- 172.21.15.37:0/4214100802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 msgr2=0x5629afd18ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2495e00640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd18ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2495e00640 1 -- 172.21.15.37:0/4214100802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd0ede0 msgr2=0x5629afd132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2495e00640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd0ede0 0x5629afd132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2495e00640 1 -- 172.21.15.37:0/4214100802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629af97d3f0 con 0x5629afd06260 2024-03-21T09:20:17.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.989+0000 7f2497200640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd18ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f2495e00640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd06260 0x5629afd19400 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2498198c20 tx=0x7f2484012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71fca13f39c12ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f2477e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f2477e00640 1 -- 172.21.15.37:0/4214100802 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248401f060 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f249002cca0 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f2477e00640 1 -- 172.21.15.37:0/4214100802 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f248401a3f0 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f2477e00640 1 -- 172.21.15.37:0/4214100802 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2484026050 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f24a01dd740 10 client.27439 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.990+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5629afa477d0 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f2477e00640 1 -- 172.21.15.37:0/4214100802 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f249002cca0 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f2477e00640 1 -- 172.21.15.37:0/4214100802 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2484019030 con 0x5629afd06260 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f24a01dd740 10 client.27439 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f24a01dd740 10 client.27439 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f24a01dd740 20 client.27439 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd23160 0x5629afd25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f24a01dd740 4 client.27439 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629afc76800 con 0x5629afd23160 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.991+0000 7f2497200640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd23160 0x5629afd25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:17.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.992+0000 7f2497200640 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd23160 0x5629afd25550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1844e4e4c50bb3be server_cookie=9a66e29447b1e659 in_seq=0 out_seq=0 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.992+0000 7f2477e00640 10 client.27439 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f2477e00640 1 -- 172.21.15.37:0/4214100802 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5629afc76800 con 0x5629afd23160 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f2477e00640 10 client.27439 handle_command_reply: tid=0 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 client.27439 shutdown 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd06260 msgr2=0x5629afd19400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd06260 0x5629afd19400 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2498198c20 tx=0x7f2484012bc0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 shutdown_connections 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd23160 0x5629afd25550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd0ede0 0x5629afd132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd06260 0x5629afd19400 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f2477e00640 0 client.27439 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4214100802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd18ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 >> 172.21.15.37:0/4214100802 conn(0x5629afcfff80 msgr2=0x5629afd01cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 shutdown_connections 2024-03-21T09:20:17.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 1 -- 172.21.15.37:0/4214100802 wait complete. 2024-03-21T09:20:17.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:17.993+0000 7f24a01dd740 20 client.27439 trim_cache size 0 max 16384 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 Processor -- start 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 -- start start 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd058b0 0x5629afd192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 0x5629afcfefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981aeb80 con 0x5629afd058b0 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981ac680 con 0x5629afd06260 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.007+0000 7f24a01dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981aa740 con 0x5629afd0ede0 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 0x5629afcfefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 0x5629afcfefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42576/0 (socket says 172.21.15.37:42576) 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 -- 172.21.15.37:0/4226936457 learned_addr learned my addr 172.21.15.37:0/4226936457 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2497200640 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd058b0 0x5629afd192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 -- 172.21.15.37:0/4226936457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 msgr2=0x5629afd0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2497200640 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd0d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 -- 172.21.15.37:0/4226936457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd058b0 msgr2=0x5629afd192a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd058b0 0x5629afd192a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 -- 172.21.15.37:0/4226936457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629afd0e480 con 0x5629afd06260 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.008+0000 7f2495e00640 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 0x5629afcfefe0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2484001eb0 tx=0x7f2484005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3cf635859cc6ad6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f2482a00640 1 -- 172.21.15.37:0/4226936457 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248402e020 con 0x5629afd06260 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f2482a00640 1 -- 172.21.15.37:0/4226936457 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2484021e50 con 0x5629afd06260 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f2482a00640 1 -- 172.21.15.37:0/4226936457 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840055e0 con 0x5629afd06260 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 -- 172.21.15.37:0/4226936457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 msgr2=0x5629afcfefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 0x5629afcfefe0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2484001eb0 tx=0x7f2484005bf0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 -- 172.21.15.37:0/4226936457 shutdown_connections 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd06260 0x5629afcfefe0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 --2- 172.21.15.37:0/4226936457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd058b0 0x5629afd192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 -- 172.21.15.37:0/4226936457 >> 172.21.15.37:0/4226936457 conn(0x5629afcfff80 msgr2=0x5629afcfe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 -- 172.21.15.37:0/4226936457 shutdown_connections 2024-03-21T09:20:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.009+0000 7f24a01dd740 1 -- 172.21.15.37:0/4226936457 wait complete. 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.010+0000 7f24a01dd740 1 -- start start 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.010+0000 7f24a01dd740 1 Processor -- start 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd1d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd1d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd1da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981aad40 con 0x5629afd06260 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981ac020 con 0x5629afd058b0 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981ab360 con 0x5629afd0ede0 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f24a01dd740 10 client.0 _ll_register_callbacks cb 0x7f248400bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd1d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd1d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42590/0 (socket says 172.21.15.37:42590) 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 -- 172.21.15.37:0/2529217487 learned_addr learned my addr 172.21.15.37:0/2529217487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2495e00640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd1d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd1da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 -- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 msgr2=0x5629afd1da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd1da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 -- 172.21.15.37:0/2529217487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 msgr2=0x5629afd1d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd1d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.011+0000 7f2497200640 1 -- 172.21.15.37:0/2529217487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629afd7d9b0 con 0x5629afd058b0 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f2497200640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd1d000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2490003af0 tx=0x7f249002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef62f013d3776978 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f2477e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f2477e00640 1 -- 172.21.15.37:0/2529217487 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490043040 con 0x5629afd058b0 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2484021e50 con 0x5629afd058b0 2024-03-21T09:20:18.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f2477e00640 1 -- 172.21.15.37:0/2529217487 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f24900328a0 con 0x5629afd058b0 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f2477e00640 1 -- 172.21.15.37:0/2529217487 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2490039040 con 0x5629afd058b0 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f24a01dd740 10 client.27416 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5629afd7ec60 con 0x5629afd058b0 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.012+0000 7f2477e00640 1 -- 172.21.15.37:0/2529217487 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2484021e50 con 0x5629afd058b0 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f2477e00640 1 -- 172.21.15.37:0/2529217487 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f249003f470 con 0x5629afd058b0 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f24a01dd740 10 client.27416 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:18.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f24a01dd740 10 client.27416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f24a01dd740 20 client.27416 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f24a01dd740 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd7ee90 0x5629afd81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f24a01dd740 4 client.27416 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629afc70b50 con 0x5629afd7ee90 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f2495e00640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd7ee90 0x5629afd81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f2495e00640 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd7ee90 0x5629afd81280 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9134f58512f467ef server_cookie=f0c6ec5b6476ee10 in_seq=0 out_seq=0 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.013+0000 7f2477e00640 10 client.27416 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f2477e00640 1 -- 172.21.15.37:0/2529217487 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5629afc70b50 con 0x5629afd7ee90 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f2477e00640 10 client.27416 handle_command_reply: tid=0 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 client.27416 shutdown 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 msgr2=0x5629afd1d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd1d000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2490003af0 tx=0x7f249002e040 comp rx=0 tx=0).stop 2024-03-21T09:20:18.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 shutdown_connections 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5629afd0ede0 0x5629afd1da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5629afd7ee90 0x5629afd81280 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629afd06260 0x5629afd1d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 --2- 172.21.15.37:0/2529217487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5629afd058b0 0x5629afd1d000 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f2477e00640 0 client.27416 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 >> 172.21.15.37:0/2529217487 conn(0x5629afcfff80 msgr2=0x5629afcff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 shutdown_connections 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 1 -- 172.21.15.37:0/2529217487 wait complete. 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.014+0000 7f24a01dd740 20 client.27416 trim_cache size 0 max 16384 2024-03-21T09:20:18.015 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f247c0a3090 msgr2=0x7f247c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f247c0a3090 0x7f247c0a5550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5629afd05380 tx=0x7f2488004a40 comp rx=0 tx=0).stop 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 msgr2=0x7f249819d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 0x7f249819d930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f248800e9f0 tx=0x7f248800edf0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 shutdown_connections 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2498107560 0x7f2498197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f247c0a3090 0x7f247c0a5550 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2498101e60 0x7f249819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 --2- 172.21.15.37:0/754226084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24981014a0 0x7f249819d930 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 >> 172.21.15.37:0/754226084 conn(0x7f24980fcfb0 msgr2=0x7f24980fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 shutdown_connections 2024-03-21T09:20:18.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.015+0000 7f249ca00640 1 -- 172.21.15.37:0/754226084 wait complete. 2024-03-21T09:20:18.029 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:20:18.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:20:18.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.258+0000 7f455ac00640 1 -- 172.21.15.37:0/2240492600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554104020 msgr2=0x7f4554110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 --2- 172.21.15.37:0/2240492600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554104020 0x7f4554110530 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f454c009cd0 tx=0x7f454c02c0e0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 -- 172.21.15.37:0/2240492600 shutdown_connections 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 --2- 172.21.15.37:0/2240492600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554104020 0x7f4554110530 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 --2- 172.21.15.37:0/2240492600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554103660 0x7f4554103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 --2- 172.21.15.37:0/2240492600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554109670 0x7f4554109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 -- 172.21.15.37:0/2240492600 >> 172.21.15.37:0/2240492600 conn(0x7f45540ff150 msgr2=0x7f4554101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 -- 172.21.15.37:0/2240492600 shutdown_connections 2024-03-21T09:20:18.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.259+0000 7f455ac00640 1 -- 172.21.15.37:0/2240492600 wait complete. 2024-03-21T09:20:18.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.260+0000 7f455ac00640 1 Processor -- start 2024-03-21T09:20:18.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.260+0000 7f455ac00640 1 -- start start 2024-03-21T09:20:18.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.260+0000 7f455ac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554103660 0x7f4554199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.260+0000 7f455ac00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554104020 0x7f455419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455ac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554109670 0x7f455419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455ac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4554113a80 con 0x7f4554103660 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455ac00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4554113900 con 0x7f4554104020 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455ac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4554113c00 con 0x7f4554109670 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f4559800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554103660 0x7f4554199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f4558e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554104020 0x7f455419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554109670 0x7f455419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f4559800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554103660 0x7f4554199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45156/0 (socket says 172.21.15.37:45156) 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f4558e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554104020 0x7f455419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42604/0 (socket says 172.21.15.37:42604) 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f4559800640 1 -- 172.21.15.37:0/1212271420 learned_addr learned my addr 172.21.15.37:0/1212271420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455a200640 1 -- 172.21.15.37:0/1212271420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554104020 msgr2=0x7f455419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455a200640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554104020 0x7f455419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455a200640 1 -- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554103660 msgr2=0x7f4554199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455a200640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554103660 0x7f4554199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.261+0000 7f455a200640 1 -- 172.21.15.37:0/1212271420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455419ef90 con 0x7f4554109670 2024-03-21T09:20:18.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.262+0000 7f455a200640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554109670 0x7f455419e860 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f454c037040 tx=0x7f454c032460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.262+0000 7f4552000640 1 -- 172.21.15.37:0/1212271420 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c039040 con 0x7f4554109670 2024-03-21T09:20:18.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.262+0000 7f4552000640 1 -- 172.21.15.37:0/1212271420 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f454c044070 con 0x7f4554109670 2024-03-21T09:20:18.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.262+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455419f220 con 0x7f4554109670 2024-03-21T09:20:18.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.262+0000 7f4552000640 1 -- 172.21.15.37:0/1212271420 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c0042a0 con 0x7f4554109670 2024-03-21T09:20:18.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.262+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a6b10 con 0x7f4554109670 2024-03-21T09:20:18.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.266+0000 7f455ef8d740 1 Processor -- start 2024-03-21T09:20:18.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f4552000640 1 -- 172.21.15.37:0/1212271420 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f454c003970 con 0x7f4554109670 2024-03-21T09:20:18.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 -- start start 2024-03-21T09:20:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4add5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4adde810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ade1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4ad392c0 con 0x55da4addede0 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4ace1c90 con 0x55da4add6260 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f455ef8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4ad35360 con 0x55da4add58b0 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f4559800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4adde810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f4559800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4adde810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42614/0 (socket says 172.21.15.37:42614) 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.267+0000 7f4559800640 1 -- 172.21.15.37:0/1995042320 learned_addr learned my addr 172.21.15.37:0/1995042320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4558e00640 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4add5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4558e00640 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ade1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4559800640 1 -- 172.21.15.37:0/1995042320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 msgr2=0x55da4add5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4559800640 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4add5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4559800640 1 -- 172.21.15.37:0/1995042320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 msgr2=0x55da4ade1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4559800640 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ade1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4559800640 1 -- 172.21.15.37:0/1995042320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da4ad06080 con 0x55da4add6260 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4558e00640 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ade1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4559800640 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4adde810 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4548009d40 tx=0x7f4548011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ed7575a7c1d723 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4552000640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f453c0a3110 0x7f453c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4546a00640 1 -- 172.21.15.37:0/1995042320 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548004220 con 0x55da4add6260 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4546a00640 1 -- 172.21.15.37:0/1995042320 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45480043c0 con 0x55da4add6260 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.268+0000 7f4546a00640 1 -- 172.21.15.37:0/1995042320 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45480035f0 con 0x55da4add6260 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f4552000640 1 -- 172.21.15.37:0/1212271420 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f454c0f1dc0 con 0x7f4554109670 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f4558e00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f453c0a3110 0x7f453c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 -- 172.21.15.37:0/1995042320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 msgr2=0x55da4adde810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4adde810 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4548009d40 tx=0x7f4548011810 comp rx=0 tx=0).stop 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 -- 172.21.15.37:0/1995042320 shutdown_connections 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ade1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4adde810 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1995042320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4add5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 -- 172.21.15.37:0/1995042320 >> 172.21.15.37:0/1995042320 conn(0x55da4adcff80 msgr2=0x55da4add0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 -- 172.21.15.37:0/1995042320 shutdown_connections 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f4558e00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f453c0a3110 0x7f453c0a55d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f45541017b0 tx=0x7f4540004730 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.269+0000 7f455ef8d740 1 -- 172.21.15.37:0/1995042320 wait complete. 2024-03-21T09:20:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.270+0000 7f455ef8d740 1 -- start start 2024-03-21T09:20:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.270+0000 7f455ef8d740 1 Processor -- start 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.270+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.270+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4addba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.270+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4addbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.270+0000 7f455ef8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4ade1d50 con 0x55da4add6260 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455ef8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4acf50d0 con 0x55da4add58b0 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455ef8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4ade1ed0 con 0x55da4addede0 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455ef8d740 10 client.0 _ll_register_callbacks cb 0x55da4acf9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f4559800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f4559800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42626/0 (socket says 172.21.15.37:42626) 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f4559800640 1 -- 172.21.15.37:0/1566612128 learned_addr learned my addr 172.21.15.37:0/1566612128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f4558e00640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4addba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4addbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 -- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 msgr2=0x55da4adde6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 -- 172.21.15.37:0/1566612128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 msgr2=0x55da4addba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4addba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 -- 172.21.15.37:0/1566612128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da4aa4d3f0 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f4559800640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f455a200640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4addbff0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f454c0ef1c0 tx=0x7f454c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b5669123ebf44f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.271+0000 7f453b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f453b400640 1 -- 172.21.15.37:0/1566612128 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c0be5f0 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f453b400640 1 -- 172.21.15.37:0/1566612128 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f454c0be150 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45480043c0 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f453b400640 1 -- 172.21.15.37:0/1566612128 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454c0bdef0 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f455ef8d740 10 client.27454 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55da4ab177d0 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f453b400640 1 -- 172.21.15.37:0/1566612128 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45480043c0 con 0x55da4addede0 2024-03-21T09:20:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.272+0000 7f453b400640 1 -- 172.21.15.37:0/1566612128 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f454c03c030 con 0x55da4addede0 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f455ef8d740 10 client.27454 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f455ef8d740 10 client.27454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f455ef8d740 20 client.27454 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4adeeba0 0x55da4adf0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f455ef8d740 4 client.27454 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da4ad46800 con 0x55da4adeeba0 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.273+0000 7f4559800640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4adeeba0 0x55da4adf0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.274+0000 7f4559800640 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4adeeba0 0x55da4adf0ff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c38775995658a247 server_cookie=88035baf88e5a64a in_seq=0 out_seq=0 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.274+0000 7f453b400640 10 client.27454 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.274+0000 7f453b400640 1 -- 172.21.15.37:0/1566612128 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55da4ad46800 con 0x55da4adeeba0 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.274+0000 7f453b400640 10 client.27454 handle_command_reply: tid=0 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 client.27454 shutdown 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 msgr2=0x55da4addbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4addbff0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f454c0ef1c0 tx=0x7f454c0095c0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 shutdown_connections 2024-03-21T09:20:18.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4adeeba0 0x55da4adf0ff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4addbff0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4addba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 --2- 172.21.15.37:0/1566612128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f453b400640 0 client.27454 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 >> 172.21.15.37:0/1566612128 conn(0x55da4adcff80 msgr2=0x55da4add1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 shutdown_connections 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 1 -- 172.21.15.37:0/1566612128 wait complete. 2024-03-21T09:20:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.275+0000 7f455ef8d740 20 client.27454 trim_cache size 0 max 16384 2024-03-21T09:20:18.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 Processor -- start 2024-03-21T09:20:18.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 -- start start 2024-03-21T09:20:18.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4adde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4add8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541b3070 con 0x55da4add6260 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541b0b70 con 0x55da4add58b0 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455ef8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541aec30 con 0x55da4addede0 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f4559800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f455a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4add8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f4559800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42642/0 (socket says 172.21.15.37:42642) 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f4558e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4adde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.289+0000 7f4559800640 1 -- 172.21.15.37:0/902862456 learned_addr learned my addr 172.21.15.37:0/902862456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f455a200640 1 -- 172.21.15.37:0/902862456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 msgr2=0x55da4adde1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f455a200640 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f455a200640 1 -- 172.21.15.37:0/902862456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 msgr2=0x55da4adde6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f455a200640 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4adde6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f455a200640 1 -- 172.21.15.37:0/902862456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da4adcfab0 con 0x55da4addede0 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f455a200640 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4add8cd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f454c0ba8c0 tx=0x7f454c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb7df36d864996 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f4546000640 1 -- 172.21.15.37:0/902862456 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c0bda30 con 0x55da4addede0 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f4546000640 1 -- 172.21.15.37:0/902862456 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f454c0be5f0 con 0x55da4addede0 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.290+0000 7f4546000640 1 -- 172.21.15.37:0/902862456 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c0be150 con 0x55da4addede0 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 -- 172.21.15.37:0/902862456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 msgr2=0x55da4add8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4add8cd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f454c0ba8c0 tx=0x7f454c005df0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 -- 172.21.15.37:0/902862456 shutdown_connections 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4addede0 0x55da4add8cd0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4add6260 0x55da4adde6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 --2- 172.21.15.37:0/902862456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add58b0 0x55da4adde1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 -- 172.21.15.37:0/902862456 >> 172.21.15.37:0/902862456 conn(0x55da4adcff80 msgr2=0x55da4adcf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 -- 172.21.15.37:0/902862456 shutdown_connections 2024-03-21T09:20:18.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 -- 172.21.15.37:0/902862456 wait complete. 2024-03-21T09:20:18.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.291+0000 7f455ef8d740 1 -- start start 2024-03-21T09:20:18.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 Processor -- start 2024-03-21T09:20:18.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4ade4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4ade4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ae49490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541af230 con 0x55da4addede0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541b0510 con 0x55da4add6260 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541af850 con 0x55da4add58b0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455ef8d740 10 client.0 _ll_register_callbacks cb 0x7f454c0b9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f4559800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4ade4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f4559800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4ade4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45258/0 (socket says 172.21.15.37:45258) 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f4559800640 1 -- 172.21.15.37:0/3071002719 learned_addr learned my addr 172.21.15.37:0/3071002719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f4558e00640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4ade4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.292+0000 7f455a200640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ae49490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f4559800640 1 -- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 msgr2=0x55da4ade4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f4559800640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4ade4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f4559800640 1 -- 172.21.15.37:0/3071002719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 msgr2=0x55da4ae49490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f4559800640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ae49490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f4559800640 1 -- 172.21.15.37:0/3071002719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da4ade5640 con 0x55da4add58b0 2024-03-21T09:20:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f4559800640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4ade4860 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4548013010 tx=0x7f4548005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d848ba3bbd8515 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f453b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f453b400640 1 -- 172.21.15.37:0/3071002719 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454801d070 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f453b400640 1 -- 172.21.15.37:0/3071002719 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4548004040 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.293+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f454c0be5f0 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f453b400640 1 -- 172.21.15.37:0/3071002719 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454803c050 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 10 client.27460 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55da4ae4aaa0 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f453b400640 1 -- 172.21.15.37:0/3071002719 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f454c0be5f0 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f453b400640 1 -- 172.21.15.37:0/3071002719 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4548023040 con 0x55da4add58b0 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 10 client.27460 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 10 client.27460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 20 client.27460 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4ae4b060 0x55da4ae4d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 4 client.27460 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:20:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55da4ad40b50 con 0x55da4ae4b060 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.294+0000 7f4558e00640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4ae4b060 0x55da4ae4d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f4558e00640 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4ae4b060 0x55da4ae4d480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f34231c130b96f server_cookie=99fb22e10573a187 in_seq=0 out_seq=0 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f453b400640 10 client.27460 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f453b400640 1 -- 172.21.15.37:0/3071002719 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55da4ad40b50 con 0x55da4ae4b060 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f453b400640 10 client.27460 handle_command_reply: tid=0 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 client.27460 shutdown 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 msgr2=0x55da4ade4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4ade4860 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4548013010 tx=0x7f4548005780 comp rx=0 tx=0).stop 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 shutdown_connections 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55da4ae4b060 0x55da4ae4d480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4addede0 0x55da4ae49490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da4add6260 0x55da4ade4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 --2- 172.21.15.37:0/3071002719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55da4add58b0 0x55da4ade4860 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 >> 172.21.15.37:0/3071002719 conn(0x55da4adcff80 msgr2=0x55da4add1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.295+0000 7f453b400640 0 client.27460 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 shutdown_connections 2024-03-21T09:20:18.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ef8d740 1 -- 172.21.15.37:0/3071002719 wait complete. 2024-03-21T09:20:18.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ef8d740 20 client.27460 trim_cache size 0 max 16384 2024-03-21T09:20:18.297 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:20:18.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f453c0a3110 msgr2=0x7f453c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f453c0a3110 0x7f453c0a55d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f45541017b0 tx=0x7f4540004730 comp rx=0 tx=0).stop 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554109670 msgr2=0x7f455419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554109670 0x7f455419e860 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f454c037040 tx=0x7f454c032460 comp rx=0 tx=0).stop 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 shutdown_connections 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4554109670 0x7f455419e860 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f453c0a3110 0x7f453c0a55d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554104020 0x7f455419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 --2- 172.21.15.37:0/1212271420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4554103660 0x7f4554199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.296+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 >> 172.21.15.37:0/1212271420 conn(0x7f45540ff150 msgr2=0x7f45540ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.297+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 shutdown_connections 2024-03-21T09:20:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.297+0000 7f455ac00640 1 -- 172.21.15.37:0/1212271420 wait complete. 2024-03-21T09:20:18.311 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:20:18.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:20:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 -- 172.21.15.37:0/3428202825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8104220 msgr2=0x7fa4e8110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 --2- 172.21.15.37:0/3428202825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8104220 0x7fa4e8110730 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009930 tx=0x7fa4e002bc30 comp rx=0 tx=0).stop 2024-03-21T09:20:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 -- 172.21.15.37:0/3428202825 shutdown_connections 2024-03-21T09:20:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 --2- 172.21.15.37:0/3428202825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8104220 0x7fa4e8110730 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 --2- 172.21.15.37:0/3428202825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 --2- 172.21.15.37:0/3428202825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8109870 0x7fa4e8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 -- 172.21.15.37:0/3428202825 >> 172.21.15.37:0/3428202825 conn(0x7fa4e80ff310 msgr2=0x7fa4e8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 -- 172.21.15.37:0/3428202825 shutdown_connections 2024-03-21T09:20:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.530+0000 7fa4efa00640 1 -- 172.21.15.37:0/3428202825 wait complete. 2024-03-21T09:20:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.531+0000 7fa4efa00640 1 Processor -- start 2024-03-21T09:20:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.531+0000 7fa4efa00640 1 -- start start 2024-03-21T09:20:18.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.531+0000 7fa4efa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.531+0000 7fa4efa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8104220 0x7fa4e819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4efa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8109870 0x7fa4e819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4efa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8113c90 con 0x7fa4e8104220 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4efa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8113b10 con 0x7fa4e8103860 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4efa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8113e10 con 0x7fa4e8109870 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42676/0 (socket says 172.21.15.37:42676) 2024-03-21T09:20:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 -- 172.21.15.37:0/2339051077 learned_addr learned my addr 172.21.15.37:0/2339051077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4eda00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8109870 0x7fa4e819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8104220 0x7fa4e819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 -- 172.21.15.37:0/2339051077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8109870 msgr2=0x7fa4e819ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8109870 0x7fa4e819ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 -- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8104220 msgr2=0x7fa4e819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8104220 0x7fa4e819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4ed000640 1 -- 172.21.15.37:0/2339051077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e819f140 con 0x7fa4e8103860 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.532+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8104220 0x7fa4e819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.533+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e819a150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800eab0 tx=0x7fa4d800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.533+0000 7fa4e5600640 1 -- 172.21.15.37:0/2339051077 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800ce40 con 0x7fa4e8103860 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.533+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e819f430 con 0x7fa4e8103860 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.533+0000 7fa4e5600640 1 -- 172.21.15.37:0/2339051077 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8011950 con 0x7fa4e8103860 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.533+0000 7fa4e5600640 1 -- 172.21.15.37:0/2339051077 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8010ae0 con 0x7fa4e8103860 2024-03-21T09:20:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.533+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e81a6cc0 con 0x7fa4e8103860 2024-03-21T09:20:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.536+0000 7fa4f262a740 1 Processor -- start 2024-03-21T09:20:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.536+0000 7fa4f262a740 1 -- start start 2024-03-21T09:20:18.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 0x5611e76adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76b6c00 0x5611e76b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4f262a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e76112c0 con 0x5611e76ae080 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4f262a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e75b9c90 con 0x5611e76b6c00 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4f262a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e760d360 con 0x5611e76ad6d0 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 0x5611e76adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 0x5611e76adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45280/0 (socket says 172.21.15.37:45280) 2024-03-21T09:20:18.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 -- 172.21.15.37:0/1743725911 learned_addr learned my addr 172.21.15.37:0/1743725911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76b6c00 0x5611e76b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 -- 172.21.15.37:0/1743725911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76b6c00 msgr2=0x5611e76b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76b6c00 0x5611e76b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 -- 172.21.15.37:0/1743725911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 msgr2=0x5611e76b6630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:20:18.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.537+0000 7fa4eda00640 1 -- 172.21.15.37:0/1743725911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611e75de080 con 0x5611e76ad6d0 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.538+0000 7fa4e5600640 1 -- 172.21.15.37:0/2339051077 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa4d8011b00 con 0x7fa4e8103860 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.538+0000 7fa4eda00640 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 0x5611e76adab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009930 tx=0x7fa4e0009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eeb9833befd5edf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.538+0000 7fa4de000640 1 -- 172.21.15.37:0/1743725911 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0045070 con 0x5611e76ad6d0 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.538+0000 7fa4de000640 1 -- 172.21.15.37:0/1743725911 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4e002c690 con 0x5611e76ad6d0 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.538+0000 7fa4de000640 1 -- 172.21.15.37:0/1743725911 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0004b60 con 0x5611e76ad6d0 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 -- 172.21.15.37:0/1743725911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 msgr2=0x5611e76adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 0x5611e76adab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009930 tx=0x7fa4e0009610 comp rx=0 tx=0).stop 2024-03-21T09:20:18.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 -- 172.21.15.37:0/1743725911 shutdown_connections 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76b6c00 0x5611e76b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1743725911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ad6d0 0x5611e76adab0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 -- 172.21.15.37:0/1743725911 >> 172.21.15.37:0/1743725911 conn(0x5611e76a7da0 msgr2=0x5611e76a8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 -- 172.21.15.37:0/1743725911 shutdown_connections 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 -- 172.21.15.37:0/1743725911 wait complete. 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.539+0000 7fa4f262a740 1 -- start start 2024-03-21T09:20:18.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.540+0000 7fa4e5600640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa4cc0a2fa0 0x7fa4cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.540+0000 7fa4f262a740 1 Processor -- start 2024-03-21T09:20:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.540+0000 7fa4e5600640 1 -- 172.21.15.37:0/2339051077 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa4d80d0220 con 0x7fa4e8103860 2024-03-21T09:20:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.540+0000 7fa4eda00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa4cc0a2fa0 0x7fa4cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.540+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.540+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e76b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4f262a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e76b9b10 con 0x5611e76ae080 2024-03-21T09:20:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4f262a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e75cd0d0 con 0x5611e76ad6d0 2024-03-21T09:20:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4f262a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611e76b9c90 con 0x5611e76b6c00 2024-03-21T09:20:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4f262a740 10 client.0 _ll_register_callbacks cb 0x5611e75d1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4eda00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa4cc0a2fa0 0x7fa4cc0a5460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4e002ceb0 tx=0x7fa4e002bc30 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e76b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42700/0 (socket says 172.21.15.37:42700) 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 -- 172.21.15.37:0/2721615350 learned_addr learned my addr 172.21.15.37:0/2721615350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 -- 172.21.15.37:0/2721615350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 msgr2=0x5611e76b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e76b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 -- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 msgr2=0x5611e76b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.541+0000 7fa4ed000640 1 -- 172.21.15.37:0/2721615350 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611e73253f0 con 0x5611e76ad6d0 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76b64c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cd620 tx=0x7fa4d809bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36360701516ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2721615350 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80b5060 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2721615350 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa4d809c470 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611e761e800 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2721615350 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d809c210 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4f262a740 10 client.27446 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5611e73ef7d0 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2721615350 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611e761e800 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.542+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2721615350 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa4d800f040 con 0x5611e76ad6d0 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4f262a740 10 client.27446 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4f262a740 10 client.27446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4f262a740 20 client.27446 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e76c6990 0x5611e76c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4f262a740 4 client.27446 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611e7618b50 con 0x5611e76c6990 2024-03-21T09:20:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.543+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e76c6990 0x5611e76c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.544+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e76c6990 0x5611e76c8d80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367c0f2a3b9fdcf9 server_cookie=788071050ef7fe4b in_seq=0 out_seq=0 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.544+0000 7fa4d6a00640 10 client.27446 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.544+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2721615350 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611e7618b50 con 0x5611e76c6990 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.544+0000 7fa4d6a00640 10 client.27446 handle_command_reply: tid=0 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.544+0000 7fa4f262a740 1 client.27446 shutdown 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 msgr2=0x5611e76b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76b64c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cd620 tx=0x7fa4d809bd50 comp rx=0 tx=0).stop 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 shutdown_connections 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e76b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ae080 0x5611e76b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e76c6990 0x5611e76c8d80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2721615350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76b64c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 >> 172.21.15.37:0/2721615350 conn(0x5611e76a7da0 msgr2=0x5611e76a9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4d6a00640 0 client.27446 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 shutdown_connections 2024-03-21T09:20:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 1 -- 172.21.15.37:0/2721615350 wait complete. 2024-03-21T09:20:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.545+0000 7fa4f262a740 20 client.27446 trim_cache size 0 max 16384 2024-03-21T09:20:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 Processor -- start 2024-03-21T09:20:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 -- start start 2024-03-21T09:20:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ae080 0x5611e76b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76b6c00 0x5611e76b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81b31f0 con 0x5611e76b6c00 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81b0cf0 con 0x5611e76ad6d0 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.559+0000 7fa4f262a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81aedb0 con 0x5611e76ae080 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ae080 0x5611e76b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42716/0 (socket says 172.21.15.37:42716) 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 -- 172.21.15.37:0/1704625768 learned_addr learned my addr 172.21.15.37:0/1704625768 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4eda00640 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76b6c00 0x5611e76b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 -- 172.21.15.37:0/1704625768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ae080 msgr2=0x5611e76b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ae080 0x5611e76b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 -- 172.21.15.37:0/1704625768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76b6c00 msgr2=0x5611e76b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76b6c00 0x5611e76b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 -- 172.21.15.37:0/1704625768 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611e76a3510 con 0x5611e76ad6d0 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4eda00640 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76b6c00 0x5611e76b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4ed000640 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76a70f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cdb70 tx=0x7fa4d809be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45255308167dbcc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4dd600640 1 -- 172.21.15.37:0/1704625768 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80a4960 con 0x5611e76ad6d0 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4dd600640 1 -- 172.21.15.37:0/1704625768 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4d80a4c20 con 0x5611e76ad6d0 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.560+0000 7fa4dd600640 1 -- 172.21.15.37:0/1704625768 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80981c0 con 0x5611e76ad6d0 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 -- 172.21.15.37:0/1704625768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 msgr2=0x5611e76a70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76a70f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cdb70 tx=0x7fa4d809be80 comp rx=0 tx=0).stop 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 -- 172.21.15.37:0/1704625768 shutdown_connections 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76b6c00 0x5611e76b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76ae080 0x5611e76b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 --2- 172.21.15.37:0/1704625768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ad6d0 0x5611e76a70f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 -- 172.21.15.37:0/1704625768 >> 172.21.15.37:0/1704625768 conn(0x5611e76a7da0 msgr2=0x5611e76aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 -- 172.21.15.37:0/1704625768 shutdown_connections 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.561+0000 7fa4f262a740 1 -- 172.21.15.37:0/1704625768 wait complete. 2024-03-21T09:20:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 -- start start 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 Processor -- start 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ad6d0 0x5611e771b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 0x5611e771bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e7716160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81af3b0 con 0x5611e76ad6d0 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81b0690 con 0x5611e76ae080 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81af9d0 con 0x5611e76b6c00 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.562+0000 7fa4f262a740 10 client.0 _ll_register_callbacks cb 0x7fa4d80984b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 0x5611e771bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e7716160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 0x5611e771bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42728/0 (socket says 172.21.15.37:42728) 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e7716160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45300/0 (socket says 172.21.15.37:45300) 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 -- 172.21.15.37:0/2448958825 learned_addr learned my addr 172.21.15.37:0/2448958825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ad6d0 0x5611e771b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 -- 172.21.15.37:0/2448958825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 msgr2=0x5611e7716160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e7716160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 -- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ad6d0 msgr2=0x5611e771b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ad6d0 0x5611e771b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 -- 172.21.15.37:0/2448958825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611e76c90f0 con 0x5611e76ae080 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ad6d0 0x5611e771b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4eda00640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e7716160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4e7e00640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 0x5611e771bb80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009d00 tx=0x7fa4d001fad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77fd355ec182cdc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.563+0000 7fa4d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2448958825 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0012650 con 0x5611e76ae080 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4d80a4c20 con 0x5611e76ae080 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2448958825 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa4d00127f0 con 0x5611e76ae080 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2448958825 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d0031050 con 0x5611e76ae080 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4f262a740 10 client.27452 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:18.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5611e7722ce0 con 0x5611e76ae080 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.564+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2448958825 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4d80a4c20 con 0x5611e76ae080 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2448958825 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa4d0026020 con 0x5611e76ae080 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4f262a740 10 client.27452 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4f262a740 10 client.27452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4f262a740 20 client.27452 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e77233f0 0x5611e7725810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4f262a740 4 client.27452 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5611e7596500 con 0x5611e77233f0 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.565+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e77233f0 0x5611e7725810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4ed000640 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e77233f0 0x5611e7725810 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31438d2fcf0210e server_cookie=683a98791cf7b122 in_seq=0 out_seq=0 2024-03-21T09:20:18.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4d6a00640 10 client.27452 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4d6a00640 1 -- 172.21.15.37:0/2448958825 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5611e7596500 con 0x5611e77233f0 2024-03-21T09:20:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4d6a00640 10 client.27452 handle_command_reply: tid=0 2024-03-21T09:20:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 client.27452 shutdown 2024-03-21T09:20:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 msgr2=0x5611e771bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 0x5611e771bb80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009d00 tx=0x7fa4d001fad0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 shutdown_connections 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611e76b6c00 0x5611e7716160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5611e77233f0 0x5611e7725810 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611e76ae080 0x5611e771bb80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 --2- 172.21.15.37:0/2448958825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611e76ad6d0 0x5611e771b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4d6a00640 0 client.27452 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 >> 172.21.15.37:0/2448958825 conn(0x5611e76a7da0 msgr2=0x5611e76a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 shutdown_connections 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.566+0000 7fa4f262a740 1 -- 172.21.15.37:0/2448958825 wait complete. 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4f262a740 20 client.27452 trim_cache size 0 max 16384 2024-03-21T09:20:18.569 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:20:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa4cc0a2fa0 msgr2=0x7fa4cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa4cc0a2fa0 0x7fa4cc0a5460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4e002ceb0 tx=0x7fa4e002bc30 comp rx=0 tx=0).stop 2024-03-21T09:20:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 msgr2=0x7fa4e819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e819a150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800eab0 tx=0x7fa4d800ef70 comp rx=0 tx=0).stop 2024-03-21T09:20:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 shutdown_connections 2024-03-21T09:20:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8109870 0x7fa4e819ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e8104220 0x7fa4e819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fa4cc0a2fa0 0x7fa4cc0a5460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 --2- 172.21.15.37:0/2339051077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e8103860 0x7fa4e819a150 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 >> 172.21.15.37:0/2339051077 conn(0x7fa4e80ff310 msgr2=0x7fa4e80ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.567+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 shutdown_connections 2024-03-21T09:20:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:18.568+0000 7fa4efa00640 1 -- 172.21.15.37:0/2339051077 wait complete. 2024-03-21T09:20:18.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:18 smithi037 ceph-mon[31830]: pgmap v250: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-03-21T09:20:18.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:18 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2210138815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:20:18.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:18 smithi071 ceph-mon[36597]: pgmap v250: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-03-21T09:20:18.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:18 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2210138815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:20:18.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:18 smithi053 ceph-mon[36169]: pgmap v250: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-03-21T09:20:18.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:18 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2210138815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:20:19.586 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:20:19.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:20:19.586 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:20:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 -- 172.21.15.37:0/4183047040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 msgr2=0x7f8a4410cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 --2- 172.21.15.37:0/4183047040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 0x7f8a4410cd60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009900 tx=0x7f8a3402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 -- 172.21.15.37:0/4183047040 shutdown_connections 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 --2- 172.21.15.37:0/4183047040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 0x7f8a44114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 --2- 172.21.15.37:0/4183047040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a44107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 --2- 172.21.15.37:0/4183047040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 0x7f8a4410cd60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 -- 172.21.15.37:0/4183047040 >> 172.21.15.37:0/4183047040 conn(0x7f8a440772a0 msgr2=0x7f8a440796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 -- 172.21.15.37:0/4183047040 shutdown_connections 2024-03-21T09:20:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.853+0000 7f8a43200640 1 -- 172.21.15.37:0/4183047040 wait complete. 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.854+0000 7f8a43200640 1 Processor -- start 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 -- start start 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a4419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 0x7f8a4419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 0x7f8a441a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44117ec0 con 0x7f8a44108440 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44117d40 con 0x7f8a44107aa0 2024-03-21T09:20:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a43200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44118040 con 0x7f8a4410c980 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a4419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 0x7f8a4419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a4419e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42744/0 (socket says 172.21.15.37:42744) 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.855+0000 7f8a41e00640 1 -- 172.21.15.37:0/320756919 learned_addr learned my addr 172.21.15.37:0/320756919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a42800640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 0x7f8a441a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a41e00640 1 -- 172.21.15.37:0/320756919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 msgr2=0x7f8a441a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a41e00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 0x7f8a441a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a41e00640 1 -- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 msgr2=0x7f8a4419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a41e00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 0x7f8a4419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a41e00640 1 -- 172.21.15.37:0/320756919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a441a3370 con 0x7f8a44107aa0 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a3be00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 0x7f8a4419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a41e00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a4419e380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000c00 tx=0x7f8a34004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.856+0000 7f8a3aa00640 1 -- 172.21.15.37:0/320756919 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34042020 con 0x7f8a44107aa0 2024-03-21T09:20:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.857+0000 7f8a3aa00640 1 -- 172.21.15.37:0/320756919 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a34005000 con 0x7f8a44107aa0 2024-03-21T09:20:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.857+0000 7f8a3aa00640 1 -- 172.21.15.37:0/320756919 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34003b20 con 0x7f8a44107aa0 2024-03-21T09:20:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.857+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a441a3600 con 0x7f8a44107aa0 2024-03-21T09:20:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.857+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a441aaef0 con 0x7f8a44107aa0 2024-03-21T09:20:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a3aa00640 1 -- 172.21.15.37:0/320756919 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8a34045020 con 0x7f8a44107aa0 2024-03-21T09:20:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 Processor -- start 2024-03-21T09:20:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 -- start start 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 0x55bfd68c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd6828eb0 con 0x55bfd68c4bb0 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd67d1a60 con 0x55bfd68c4290 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.860+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd6824f50 con 0x55bfd68cd790 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 0x55bfd68c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 0x55bfd68c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42758/0 (socket says 172.21.15.37:42758) 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 -- 172.21.15.37:0/1788159476 learned_addr learned my addr 172.21.15.37:0/1788159476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a41e00640 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a3be00640 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 -- 172.21.15.37:0/1788159476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 msgr2=0x55bfd68cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 -- 172.21.15.37:0/1788159476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 msgr2=0x55bfd68cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 -- 172.21.15.37:0/1788159476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd66057e0 con 0x55bfd68c4290 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a41e00640 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:19.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.861+0000 7f8a42800640 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 0x55bfd68c4670 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009f50 tx=0x7f8a3c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a8d9669e0f9b3b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a33400640 1 -- 172.21.15.37:0/1788159476 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c004d20 con 0x55bfd68c4290 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a33400640 1 -- 172.21.15.37:0/1788159476 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a3c004050 con 0x55bfd68c4290 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a33400640 1 -- 172.21.15.37:0/1788159476 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c005280 con 0x55bfd68c4290 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a3aa00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8a280a3090 0x7f8a280a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a3aa00640 1 -- 172.21.15.37:0/320756919 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a340f1680 con 0x7f8a44107aa0 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a41e00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8a280a3090 0x7f8a280a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 -- 172.21.15.37:0/1788159476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 msgr2=0x55bfd68c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 0x55bfd68c4670 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009f50 tx=0x7f8a3c0118d0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 -- 172.21.15.37:0/1788159476 shutdown_connections 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/1788159476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4290 0x55bfd68c4670 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.862+0000 7f8a4ba35740 1 -- 172.21.15.37:0/1788159476 >> 172.21.15.37:0/1788159476 conn(0x55bfd68be9c0 msgr2=0x55bfd68beda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:19.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.863+0000 7f8a4ba35740 1 -- 172.21.15.37:0/1788159476 shutdown_connections 2024-03-21T09:20:19.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.863+0000 7f8a4ba35740 1 -- 172.21.15.37:0/1788159476 wait complete. 2024-03-21T09:20:19.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.863+0000 7f8a41e00640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8a280a3090 0x7f8a280a5550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a340eeb90 tx=0x7f8a34004610 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:19.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.863+0000 7f8a4ba35740 1 -- start start 2024-03-21T09:20:19.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.863+0000 7f8a4ba35740 1 Processor -- start 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68cd790 0x55bfd68caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd68d0580 con 0x55bfd68cd790 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd67e4eb0 con 0x55bfd68c4bb0 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd68d0700 con 0x55bfd68c4290 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a4ba35740 10 client.0 _ll_register_callbacks cb 0x55bfd67d9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42772/0 (socket says 172.21.15.37:42772) 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a3be00640 1 -- 172.21.15.37:0/951396609 learned_addr learned my addr 172.21.15.37:0/951396609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.864+0000 7f8a42800640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68cd790 0x55bfd68caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a3be00640 1 -- 172.21.15.37:0/951396609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 msgr2=0x55bfd68cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a3be00640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a3be00640 1 -- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68cd790 msgr2=0x55bfd68caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a3be00640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68cd790 0x55bfd68caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a3be00640 1 -- 172.21.15.37:0/951396609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfd6665f70 con 0x55bfd68c4bb0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a42800640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68cd790 0x55bfd68caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a3be00640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68ca4a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c012d00 tx=0x7f8a2c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f827cd8b99b39d81 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a27e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a27e00640 1 -- 172.21.15.37:0/951396609 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c009d90 con 0x55bfd68c4bb0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a27e00640 1 -- 172.21.15.37:0/951396609 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a2c012110 con 0x55bfd68c4bb0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.865+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a3c004050 con 0x55bfd68c4bb0 2024-03-21T09:20:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a27e00640 1 -- 172.21.15.37:0/951396609 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a2c004050 con 0x55bfd68c4bb0 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 10 client.27464 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bfd64fd170 con 0x55bfd68c4bb0 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a27e00640 1 -- 172.21.15.37:0/951396609 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a3c004050 con 0x55bfd68c4bb0 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a27e00640 1 -- 172.21.15.37:0/951396609 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8a2c010430 con 0x55bfd68c4bb0 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 10 client.27464 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 10 client.27464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 20 client.27464 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd68dd860 0x55bfd68dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 4 client.27464 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.866+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfd68363f0 con 0x55bfd68dd860 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.867+0000 7f8a42800640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd68dd860 0x55bfd68dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.867+0000 7f8a42800640 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd68dd860 0x55bfd68dfc50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b309c93bc2f6983 server_cookie=97283d8d492399ff in_seq=0 out_seq=0 2024-03-21T09:20:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.867+0000 7f8a27e00640 10 client.27464 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.868+0000 7f8a27e00640 1 -- 172.21.15.37:0/951396609 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bfd68363f0 con 0x55bfd68dd860 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.868+0000 7f8a27e00640 10 client.27464 handle_command_reply: tid=0 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.868+0000 7f8a4ba35740 1 client.27464 shutdown 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.868+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 msgr2=0x55bfd68ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68ca4a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c012d00 tx=0x7f8a2c00f790 comp rx=0 tx=0).stop 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 shutdown_connections 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68cd790 0x55bfd68caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd68dd860 0x55bfd68dfc50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68ca4a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/951396609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a27e00640 0 client.27464 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 >> 172.21.15.37:0/951396609 conn(0x55bfd68be9c0 msgr2=0x55bfd68c0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 shutdown_connections 2024-03-21T09:20:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 1 -- 172.21.15.37:0/951396609 wait complete. 2024-03-21T09:20:19.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.869+0000 7f8a4ba35740 20 client.27464 trim_cache size 0 max 16384 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 Processor -- start 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 -- start start 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4290 0x55bfd68ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68bd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441b74d0 con 0x55bfd68c4290 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441b4fd0 con 0x55bfd68c4bb0 2024-03-21T09:20:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441b3090 con 0x55bfd68cd790 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.887+0000 7f8a42800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4290 0x55bfd68ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42786/0 (socket says 172.21.15.37:42786) 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 -- 172.21.15.37:0/3651221583 learned_addr learned my addr 172.21.15.37:0/3651221583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a42800640 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68bd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 -- 172.21.15.37:0/3651221583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 msgr2=0x55bfd68bd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68bd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 -- 172.21.15.37:0/3651221583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4290 msgr2=0x55bfd68ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4290 0x55bfd68ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 -- 172.21.15.37:0/3651221583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd68cfef0 con 0x55bfd68c4bb0 2024-03-21T09:20:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a42800640 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4290 0x55bfd68ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:19.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.888+0000 7f8a3be00640 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0107d0 tx=0x7f8a2c00fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67e2124236b2bfe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:19.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a32a00640 1 -- 172.21.15.37:0/3651221583 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c02d070 con 0x55bfd68c4bb0 2024-03-21T09:20:19.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a32a00640 1 -- 172.21.15.37:0/3651221583 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a2c009d90 con 0x55bfd68c4bb0 2024-03-21T09:20:19.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a32a00640 1 -- 172.21.15.37:0/3651221583 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c010c00 con 0x55bfd68c4bb0 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 -- 172.21.15.37:0/3651221583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 msgr2=0x55bfd68cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0107d0 tx=0x7f8a2c00fc90 comp rx=0 tx=0).stop 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 -- 172.21.15.37:0/3651221583 shutdown_connections 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68cd790 0x55bfd68bd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68c4bb0 0x55bfd68cd110 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/3651221583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4290 0x55bfd68ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 -- 172.21.15.37:0/3651221583 >> 172.21.15.37:0/3651221583 conn(0x55bfd68be9c0 msgr2=0x55bfd68bd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 -- 172.21.15.37:0/3651221583 shutdown_connections 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.889+0000 7f8a4ba35740 1 -- 172.21.15.37:0/3651221583 wait complete. 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.890+0000 7f8a4ba35740 1 -- start start 2024-03-21T09:20:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.890+0000 7f8a4ba35740 1 Processor -- start 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68d2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68cd790 0x55bfd68d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441b3690 con 0x55bfd68c4bb0 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441b4970 con 0x55bfd68cd790 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441b3cb0 con 0x55bfd68c4290 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a4ba35740 10 client.0 _ll_register_callbacks cb 0x7f8a2c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68d2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68d2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45348/0 (socket says 172.21.15.37:45348) 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 -- 172.21.15.37:0/679712322 learned_addr learned my addr 172.21.15.37:0/679712322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68cd790 0x55bfd68d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a3be00640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 -- 172.21.15.37:0/679712322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68cd790 msgr2=0x55bfd68d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68cd790 0x55bfd68d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 -- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 msgr2=0x55bfd68d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a42800640 1 -- 172.21.15.37:0/679712322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfd6939120 con 0x55bfd68c4290 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.891+0000 7f8a3be00640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a42800640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68d2920 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009f20 tx=0x7f8a3c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d53142bdd90195 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a27e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a27e00640 1 -- 172.21.15.37:0/679712322 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c004260 con 0x55bfd68c4290 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a27e00640 1 -- 172.21.15.37:0/679712322 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a3c004400 con 0x55bfd68c4290 2024-03-21T09:20:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a2c009d90 con 0x55bfd68c4290 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a27e00640 1 -- 172.21.15.37:0/679712322 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3c027440 con 0x55bfd68c4290 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a4ba35740 10 client.27496 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.892+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bfd693a460 con 0x55bfd68c4290 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a27e00640 1 -- 172.21.15.37:0/679712322 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a2c009d90 con 0x55bfd68c4290 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a27e00640 1 -- 172.21.15.37:0/679712322 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8a3c00f620 con 0x55bfd68c4290 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a4ba35740 10 client.27496 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a4ba35740 10 client.27496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a4ba35740 20 client.27496 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:19.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd693a970 0x55bfd693cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a4ba35740 4 client.27496 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bfd6830740 con 0x55bfd693a970 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.893+0000 7f8a3be00640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd693a970 0x55bfd693cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.894+0000 7f8a3be00640 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd693a970 0x55bfd693cd90 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e869ea9d61b8e2a server_cookie=3f57b1bb2f4d16f9 in_seq=0 out_seq=0 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.894+0000 7f8a27e00640 10 client.27496 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.895+0000 7f8a27e00640 1 -- 172.21.15.37:0/679712322 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bfd6830740 con 0x55bfd693a970 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.895+0000 7f8a27e00640 10 client.27496 handle_command_reply: tid=0 2024-03-21T09:20:19.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.895+0000 7f8a4ba35740 1 client.27496 shutdown 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 msgr2=0x55bfd68d2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68d2920 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009f20 tx=0x7f8a3c005fd0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 shutdown_connections 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55bfd693a970 0x55bfd693cd90 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd68cd790 0x55bfd68d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bfd68c4bb0 0x55bfd68d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 --2- 172.21.15.37:0/679712322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfd68c4290 0x55bfd68d2920 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 >> 172.21.15.37:0/679712322 conn(0x55bfd68be9c0 msgr2=0x55bfd68bda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a27e00640 0 client.27496 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 shutdown_connections 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 1 -- 172.21.15.37:0/679712322 wait complete. 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.896+0000 7f8a4ba35740 20 client.27496 trim_cache size 0 max 16384 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "98a2adc5-0141-44fa-b5dd-428c6cc816cc", 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:20:19.895 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8a280a3090 msgr2=0x7f8a280a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8a280a3090 0x7f8a280a5550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a340eeb90 tx=0x7f8a34004610 comp rx=0 tx=0).stop 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 msgr2=0x7f8a4419e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a4419e380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000c00 tx=0x7f8a34004360 comp rx=0 tx=0).stop 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 shutdown_connections 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a4410c980 0x7f8a441a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a44108440 0x7f8a4419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8a280a3090 0x7f8a280a5550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 --2- 172.21.15.37:0/320756919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a44107aa0 0x7f8a4419e380 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.897+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 >> 172.21.15.37:0/320756919 conn(0x7f8a440772a0 msgr2=0x7f8a44077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.898+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 shutdown_connections 2024-03-21T09:20:19.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:19.898+0000 7f8a43200640 1 -- 172.21.15.37:0/320756919 wait complete. 2024-03-21T09:20:19.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:20:19.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 -- 172.21.15.37:0/3788102895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8109830 msgr2=0x7f8cf8109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 --2- 172.21.15.37:0/3788102895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8109830 0x7f8cf8109c10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009930 tx=0x7f8cec02bc30 comp rx=0 tx=0).stop 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 -- 172.21.15.37:0/3788102895 shutdown_connections 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 --2- 172.21.15.37:0/3788102895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf81041e0 0x7f8cf81106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 --2- 172.21.15.37:0/3788102895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8103820 0x7f8cf8103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 --2- 172.21.15.37:0/3788102895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8109830 0x7f8cf8109c10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 -- 172.21.15.37:0/3788102895 >> 172.21.15.37:0/3788102895 conn(0x7f8cf80ff310 msgr2=0x7f8cf8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.139+0000 7f8cff400640 1 -- 172.21.15.37:0/3788102895 shutdown_connections 2024-03-21T09:20:20.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.140+0000 7f8cff400640 1 -- 172.21.15.37:0/3788102895 wait complete. 2024-03-21T09:20:20.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.141+0000 7f8cff400640 1 Processor -- start 2024-03-21T09:20:20.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.141+0000 7f8cff400640 1 -- start start 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.141+0000 7f8cff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8103820 0x7f8cf819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.141+0000 7f8cff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf81041e0 0x7f8cf819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cff400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 0x7f8cf819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cff400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8113c50 con 0x7f8cf81041e0 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cff400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8113ad0 con 0x7f8cf8109830 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cff400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8113dd0 con 0x7f8cf8103820 2024-03-21T09:20:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8103820 0x7f8cf819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 0x7f8cf819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8103820 0x7f8cf819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45352/0 (socket says 172.21.15.37:45352) 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 0x7f8cf819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42826/0 (socket says 172.21.15.37:42826) 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf81041e0 0x7f8cf819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfca00640 1 -- 172.21.15.37:0/4092972017 learned_addr learned my addr 172.21.15.37:0/4092972017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 -- 172.21.15.37:0/4092972017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8103820 msgr2=0x7f8cf819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8103820 0x7f8cf819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 -- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf81041e0 msgr2=0x7f8cf819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf81041e0 0x7f8cf819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.142+0000 7f8cfd400640 1 -- 172.21.15.37:0/4092972017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf819f1a0 con 0x7f8cf8109830 2024-03-21T09:20:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.143+0000 7f8cfd400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 0x7f8cf819ea70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009fd0 tx=0x7f8cf000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.143+0000 7f8cf5600640 1 -- 172.21.15.37:0/4092972017 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000ce20 con 0x7f8cf8109830 2024-03-21T09:20:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.143+0000 7f8cf5600640 1 -- 172.21.15.37:0/4092972017 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cf0005020 con 0x7f8cf8109830 2024-03-21T09:20:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.143+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf819f490 con 0x7f8cf8109830 2024-03-21T09:20:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.143+0000 7f8cf5600640 1 -- 172.21.15.37:0/4092972017 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0010720 con 0x7f8cf8109830 2024-03-21T09:20:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.143+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf81a6d20 con 0x7f8cf8109830 2024-03-21T09:20:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 Processor -- start 2024-03-21T09:20:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8cf5600640 1 -- 172.21.15.37:0/4092972017 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8cf0005540 con 0x7f8cf8109830 2024-03-21T09:20:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 -- start start 2024-03-21T09:20:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 0x562838039950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838043290 0x562838045680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562837f9e250 con 0x562838039f20 2024-03-21T09:20:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562837f46a30 con 0x56283803ac00 2024-03-21T09:20:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8d02201740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562837f9a2f0 con 0x562838043290 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.148+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 0x562838039950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 0x562838039950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:42838/0 (socket says 172.21.15.37:42838) 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 -- 172.21.15.37:0/1559697456 learned_addr learned my addr 172.21.15.37:0/1559697456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838043290 0x562838045680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cfca00640 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 -- 172.21.15.37:0/1559697456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838043290 msgr2=0x562838045680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838043290 0x562838045680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 -- 172.21.15.37:0/1559697456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 msgr2=0x56283803a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 -- 172.21.15.37:0/1559697456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562837f31690 con 0x56283803ac00 2024-03-21T09:20:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.149+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 0x562838039950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40093e0 tx=0x7f8ce4009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54d60a0d0d04ec60 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8ce9c00640 1 -- 172.21.15.37:0/1559697456 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400f7c0 con 0x56283803ac00 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8ce9c00640 1 -- 172.21.15.37:0/1559697456 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ce400fe40 con 0x56283803ac00 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8ce9c00640 1 -- 172.21.15.37:0/1559697456 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400e3b0 con 0x56283803ac00 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8cf5600640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8ce00a3090 0x7f8ce00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8cf5600640 1 -- 172.21.15.37:0/4092972017 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8cf00d02f0 con 0x7f8cf8109830 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8cfca00640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8ce00a3090 0x7f8ce00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8d02201740 1 -- 172.21.15.37:0/1559697456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 msgr2=0x562838039950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.150+0000 7f8d02201740 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 0x562838039950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40093e0 tx=0x7f8ce4009370 comp rx=0 tx=0).stop 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 -- 172.21.15.37:0/1559697456 shutdown_connections 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838043290 0x562838045680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 --2- 172.21.15.37:0/1559697456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56283803ac00 0x562838039950 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 -- 172.21.15.37:0/1559697456 >> 172.21.15.37:0/1559697456 conn(0x562838001200 msgr2=0x562838034970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 -- 172.21.15.37:0/1559697456 shutdown_connections 2024-03-21T09:20:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 -- 172.21.15.37:0/1559697456 wait complete. 2024-03-21T09:20:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8cfca00640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8ce00a3090 0x7f8ce00a5550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0096c0 tx=0x7f8cec02df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.151+0000 7f8d02201740 1 -- start start 2024-03-21T09:20:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.152+0000 7f8d02201740 1 Processor -- start 2024-03-21T09:20:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.152+0000 7f8d02201740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x562838042990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.152+0000 7f8d02201740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8d02201740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x562838040260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8d02201740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562838046100 con 0x562838039f20 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8d02201740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562837f59e80 con 0x562838043290 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8d02201740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562838046280 con 0x56283803ac00 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8d02201740 10 client.0 _ll_register_callbacks cb 0x562837f6c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x562838042990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x562838040260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x562838042990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45302/0 (socket says 172.21.15.37:45302) 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cfca00640 1 -- 172.21.15.37:0/4272691564 learned_addr learned my addr 172.21.15.37:0/4272691564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cf7e00640 1 -- 172.21.15.37:0/4272691564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 msgr2=0x562838040260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x562838040260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cf7e00640 1 -- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 msgr2=0x562838042990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x562838042990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cf7e00640 1 -- 172.21.15.37:0/4272691564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562837d7c7d0 con 0x56283803ac00 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.153+0000 7f8cfd400640 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x562838040260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803fd20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40093e0 tx=0x7f8ce4009410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5bb1e2c488a45f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8cde000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8cde000640 1 -- 172.21.15.37:0/4272691564 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4004040 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8cde000640 1 -- 172.21.15.37:0/4272691564 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8ce4019070 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ce400fe40 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8cde000640 1 -- 172.21.15.37:0/4272691564 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ce4033050 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8d02201740 10 client.27508 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.154+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562837cb23f0 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8cde000640 1 -- 172.21.15.37:0/4272691564 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ce400fe40 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8cde000640 1 -- 172.21.15.37:0/4272691564 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8ce401d020 con 0x56283803ac00 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8d02201740 10 client.27508 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8d02201740 10 client.27508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8d02201740 20 client.27508 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8d02201740 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562838053070 0x562838055460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8d02201740 4 client.27508 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562837fab790 con 0x562838053070 2024-03-21T09:20:20.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.155+0000 7f8cfca00640 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562838053070 0x562838055460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.156+0000 7f8cfca00640 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562838053070 0x562838055460 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a334a1f91c24a9a server_cookie=8a5c78fb50a5c305 in_seq=0 out_seq=0 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.156+0000 7f8cde000640 10 client.27508 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8cde000640 1 -- 172.21.15.37:0/4272691564 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562837fab790 con 0x562838053070 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8cde000640 10 client.27508 handle_command_reply: tid=0 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 client.27508 shutdown 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 msgr2=0x56283803fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803fd20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40093e0 tx=0x7f8ce4009410 comp rx=0 tx=0).stop 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 shutdown_connections 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562838053070 0x562838055460 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x562838040260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803fd20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8cde000640 0 client.27508 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 --2- 172.21.15.37:0/4272691564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x562838042990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 >> 172.21.15.37:0/4272691564 conn(0x562838001200 msgr2=0x562838035e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.157+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 shutdown_connections 2024-03-21T09:20:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.158+0000 7f8d02201740 1 -- 172.21.15.37:0/4272691564 wait complete. 2024-03-21T09:20:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.158+0000 7f8d02201740 20 client.27508 trim_cache size 0 max 16384 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.175+0000 7f8d02201740 1 Processor -- start 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 -- start start 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 0x562838042450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283803ac00 0x562838042990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283804ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81b3260 con 0x56283803ac00 2024-03-21T09:20:20.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81b0d60 con 0x562838043290 2024-03-21T09:20:20.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8d02201740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81aee20 con 0x562838039f20 2024-03-21T09:20:20.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 0x562838042450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283804ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 0x562838042450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45380/0 (socket says 172.21.15.37:45380) 2024-03-21T09:20:20.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8cfca00640 1 -- 172.21.15.37:0/3841754983 learned_addr learned my addr 172.21.15.37:0/3841754983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:20.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.176+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283803ac00 0x562838042990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfd400640 1 -- 172.21.15.37:0/3841754983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 msgr2=0x562838042450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfd400640 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 0x562838042450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfd400640 1 -- 172.21.15.37:0/3841754983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283803ac00 msgr2=0x562838042990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfd400640 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283803ac00 0x562838042990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfd400640 1 -- 172.21.15.37:0/3841754983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56283803b920 con 0x562838043290 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfca00640 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 0x562838042450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8cfd400640 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283804ea00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00cd5e0 tx=0x7f8cf00985a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5ab0952495548b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8ce9200640 1 -- 172.21.15.37:0/3841754983 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf009a320 con 0x562838043290 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8ce9200640 1 -- 172.21.15.37:0/3841754983 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cf009a580 con 0x562838043290 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.177+0000 7f8ce9200640 1 -- 172.21.15.37:0/3841754983 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00a4800 con 0x562838043290 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 -- 172.21.15.37:0/3841754983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 msgr2=0x56283804ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283804ea00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00cd5e0 tx=0x7f8cf00985a0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 -- 172.21.15.37:0/3841754983 shutdown_connections 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283804ea00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283803ac00 0x562838042990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 --2- 172.21.15.37:0/3841754983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562838039f20 0x562838042450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 -- 172.21.15.37:0/3841754983 >> 172.21.15.37:0/3841754983 conn(0x562838001200 msgr2=0x5628380344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 -- 172.21.15.37:0/3841754983 shutdown_connections 2024-03-21T09:20:20.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.178+0000 7f8d02201740 1 -- 172.21.15.37:0/3841754983 wait complete. 2024-03-21T09:20:20.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.179+0000 7f8d02201740 1 -- start start 2024-03-21T09:20:20.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.179+0000 7f8d02201740 1 Processor -- start 2024-03-21T09:20:20.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283803cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81af420 con 0x562838039f20 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81b0700 con 0x562838043290 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81afa40 con 0x56283803ac00 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8d02201740 10 client.0 _ll_register_callbacks cb 0x7f8ce4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cfca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cfd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283803cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45384/0 (socket says 172.21.15.37:45384) 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 -- 172.21.15.37:0/3558005747 learned_addr learned my addr 172.21.15.37:0/3558005747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 -- 172.21.15.37:0/3558005747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 msgr2=0x56283803cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283803cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 -- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 msgr2=0x56283803c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cf7e00640 1 -- 172.21.15.37:0/3558005747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562837c720f0 con 0x56283803ac00 2024-03-21T09:20:20.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cfd400640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283803cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.180+0000 7f8cfca00640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8cf7e00640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803c7e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x5628380412f0 tx=0x7f8ce4009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9004ac9605e29d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8cde000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8cde000640 1 -- 172.21.15.37:0/3558005747 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce401fa70 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cf009a580 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8cde000640 1 -- 172.21.15.37:0/3558005747 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8ce400f790 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8cde000640 1 -- 172.21.15.37:0/3558005747 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ce40052c0 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8d02201740 10 client.27514 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5628380ae760 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.181+0000 7f8cde000640 1 -- 172.21.15.37:0/3558005747 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cf009a580 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8cde000640 1 -- 172.21.15.37:0/3558005747 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8ce401d040 con 0x56283803ac00 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8d02201740 10 client.27514 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8d02201740 10 client.27514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8d02201740 20 client.27514 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8d02201740 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5628380aedf0 0x5628380b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8d02201740 4 client.27514 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:20:20.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562837fa5ae0 con 0x5628380aedf0 2024-03-21T09:20:20.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.182+0000 7f8cfca00640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5628380aedf0 0x5628380b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:20.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8cfca00640 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5628380aedf0 0x5628380b1210 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56ae6a4e4f08619 server_cookie=4026f383f9dff6e9 in_seq=0 out_seq=0 2024-03-21T09:20:20.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8cde000640 10 client.27514 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8cde000640 1 -- 172.21.15.37:0/3558005747 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562837fa5ae0 con 0x5628380aedf0 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8cde000640 10 client.27514 handle_command_reply: tid=0 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 client.27514 shutdown 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 msgr2=0x56283803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803c7e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x5628380412f0 tx=0x7f8ce4009e80 comp rx=0 tx=0).stop 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 shutdown_connections 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5628380aedf0 0x5628380b1210 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562838043290 0x56283803cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283803ac00 0x56283803c7e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 --2- 172.21.15.37:0/3558005747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562838039f20 0x56283803c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8cde000640 0 client.27514 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:20.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 >> 172.21.15.37:0/3558005747 conn(0x562838001200 msgr2=0x56283803e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 shutdown_connections 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.183+0000 7f8d02201740 1 -- 172.21.15.37:0/3558005747 wait complete. 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8d02201740 20 client.27514 trim_cache size 0 max 16384 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout: "98a2adc5-0141-44fa-b5dd-428c6cc816cc": { 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "98a2adc5-0141-44fa-b5dd-428c6cc816cc", 2024-03-21T09:20:20.184 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8ce00a3090 msgr2=0x7f8ce00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8ce00a3090 0x7f8ce00a5550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0096c0 tx=0x7f8cec02df90 comp rx=0 tx=0).stop 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 msgr2=0x7f8cf819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:20.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 0x7f8cf819ea70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009fd0 tx=0x7f8cf000ef10 comp rx=0 tx=0).stop 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 shutdown_connections 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8ce00a3090 0x7f8ce00a5550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cf8109830 0x7f8cf819ea70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf81041e0 0x7f8cf819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 --2- 172.21.15.37:0/4092972017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf8103820 0x7f8cf819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 >> 172.21.15.37:0/4092972017 conn(0x7f8cf80ff310 msgr2=0x7f8cf80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 shutdown_connections 2024-03-21T09:20:20.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:20.184+0000 7f8cff400640 1 -- 172.21.15.37:0/4092972017 wait complete. 2024-03-21T09:20:20.204 INFO:tasks.cephfs.filesystem:scrub status for tag:98a2adc5-0141-44fa-b5dd-428c6cc816cc - {'path': '/', 'tag': '98a2adc5-0141-44fa-b5dd-428c6cc816cc', 'options': 'recursive,force'} 2024-03-21T09:20:20.604 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:20 smithi071 ceph-mon[36597]: pgmap v251: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:20:20.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:20 smithi037 ceph-mon[31830]: pgmap v251: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:20:20.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:20 smithi053 ceph-mon[36169]: pgmap v251: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:20:21.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:21 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:20:21.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:21 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:21.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:21 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:20:21.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:21 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:21.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:21 smithi071 ceph-mon[36597]: pgmap v252: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:20:21.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:21 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:20:21.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:21 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:21.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:21 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:20:21.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:21 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:21.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:21 smithi037 ceph-mon[31830]: pgmap v252: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:20:21.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:21 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:20:21.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:21 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:21.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:21 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:20:21.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:21 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:21.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:21 smithi053 ceph-mon[36169]: pgmap v252: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:20:22.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:22 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:20:22.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:22 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:20:22.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:22 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:20:23.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:23 smithi037 ceph-mon[31830]: pgmap v253: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-03-21T09:20:23.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:23 smithi053 ceph-mon[36169]: pgmap v253: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-03-21T09:20:23.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:23 smithi071 ceph-mon[36597]: pgmap v253: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-03-21T09:20:26.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:26 smithi071 ceph-mon[36597]: pgmap v254: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-03-21T09:20:26.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:26 smithi037 ceph-mon[31830]: pgmap v254: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-03-21T09:20:26.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:26 smithi053 ceph-mon[36169]: pgmap v254: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-03-21T09:20:28.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:28 smithi071 ceph-mon[36597]: pgmap v255: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T09:20:28.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:28 smithi037 ceph-mon[31830]: pgmap v255: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T09:20:28.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:28 smithi053 ceph-mon[36169]: pgmap v255: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T09:20:30.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:30 smithi071 ceph-mon[36597]: pgmap v256: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:30.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:30 smithi037 ceph-mon[31830]: pgmap v256: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:30.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:30 smithi053 ceph-mon[36169]: pgmap v256: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:32.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:32 smithi071 ceph-mon[36597]: pgmap v257: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:32.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:32 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:20:32.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:32 smithi037 ceph-mon[31830]: pgmap v257: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:32.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:32 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:20:32.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:32 smithi053 ceph-mon[36169]: pgmap v257: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:32.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:32 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:20:33.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:33 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:20:33.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:33 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:20:33.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:33 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:20:33.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:33 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:20:33.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:33 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:20:33.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:33 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:20:33.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:33 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:20:33.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:33 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:20:33.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:33 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:20:34.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:34 smithi071 ceph-mon[36597]: pgmap v258: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:34.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:34 smithi037 ceph-mon[31830]: pgmap v258: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:34.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:34 smithi053 ceph-mon[36169]: pgmap v258: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:35.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:35 smithi071 ceph-mon[36597]: pgmap v259: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:35.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:35 smithi037 ceph-mon[31830]: pgmap v259: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:35.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:35 smithi053 ceph-mon[36169]: pgmap v259: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:38.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:38 smithi071 ceph-mon[36597]: pgmap v260: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:38.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:38 smithi037 ceph-mon[31830]: pgmap v260: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:38.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:38 smithi053 ceph-mon[36169]: pgmap v260: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:40.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:40 smithi071 ceph-mon[36597]: pgmap v261: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:40.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:40 smithi037 ceph-mon[31830]: pgmap v261: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:40.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:40 smithi053 ceph-mon[36169]: pgmap v261: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:42.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:42 smithi071 ceph-mon[36597]: pgmap v262: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:42.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:42 smithi037 ceph-mon[31830]: pgmap v262: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:42.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:42 smithi053 ceph-mon[36169]: pgmap v262: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:43.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:43 smithi037 ceph-mon[31830]: pgmap v263: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:43.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:43 smithi053 ceph-mon[36169]: pgmap v263: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:43.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:43 smithi071 ceph-mon[36597]: pgmap v263: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:46.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:46 smithi071 ceph-mon[36597]: pgmap v264: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:46.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:46 smithi037 ceph-mon[31830]: pgmap v264: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:46.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:46 smithi053 ceph-mon[36169]: pgmap v264: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:48.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:48 smithi071 ceph-mon[36597]: pgmap v265: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:48.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:48 smithi037 ceph-mon[31830]: pgmap v265: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:48.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:48 smithi053 ceph-mon[36169]: pgmap v265: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:50.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:20:50.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:20:50.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 -- 172.21.15.37:0/2042232084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00100d40 msgr2=0x7fcc00105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 --2- 172.21.15.37:0/2042232084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00100d40 0x7fcc00105200 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009860 tx=0x7fcbec02bc10 comp rx=0 tx=0).stop 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 -- 172.21.15.37:0/2042232084 shutdown_connections 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 --2- 172.21.15.37:0/2042232084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00105820 0x7fcc00105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 --2- 172.21.15.37:0/2042232084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00100d40 0x7fcc00105200 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 --2- 172.21.15.37:0/2042232084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc00100420 0x7fcc00100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 -- 172.21.15.37:0/2042232084 >> 172.21.15.37:0/2042232084 conn(0x7fcc00077130 msgr2=0x7fcc000fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 -- 172.21.15.37:0/2042232084 shutdown_connections 2024-03-21T09:20:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.430+0000 7fcc05600640 1 -- 172.21.15.37:0/2042232084 wait complete. 2024-03-21T09:20:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.431+0000 7fcc05600640 1 Processor -- start 2024-03-21T09:20:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 -- start start 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 0x7fcc0019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc00100d40 0x7fcc0019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00105820 0x7fcc001a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00117e30 con 0x7fcc00100d40 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00117cb0 con 0x7fcc00105820 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc05600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00117fb0 con 0x7fcc00100420 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 0x7fcc0019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 0x7fcc0019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49464/0 (socket says 172.21.15.37:49464) 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcbffe00640 1 -- 172.21.15.37:0/1193289137 learned_addr learned my addr 172.21.15.37:0/1193289137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:50.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.432+0000 7fcc04c00640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00105820 0x7fcc001a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcbff400640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc00100d40 0x7fcc0019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc04c00640 1 -- 172.21.15.37:0/1193289137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 msgr2=0x7fcc0019e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc04c00640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 0x7fcc0019e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc04c00640 1 -- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc00100d40 msgr2=0x7fcc0019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc04c00640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc00100d40 0x7fcc0019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc04c00640 1 -- 172.21.15.37:0/1193289137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc001a3340 con 0x7fcc00105820 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcbffe00640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 0x7fcc0019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc04c00640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00105820 0x7fcc001a2c10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009fd0 tx=0x7fcbf800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcbfcc00640 1 -- 172.21.15.37:0/1193289137 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800ce20 con 0x7fcc00105820 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcbfcc00640 1 -- 172.21.15.37:0/1193289137 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbf8005020 con 0x7fcc00105820 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.433+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc001a3630 con 0x7fcc00105820 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.434+0000 7fcbfcc00640 1 -- 172.21.15.37:0/1193289137 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8010720 con 0x7fcc00105820 2024-03-21T09:20:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.434+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc001aaec0 con 0x7fcc00105820 2024-03-21T09:20:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.436+0000 7fcbfcc00640 1 -- 172.21.15.37:0/1193289137 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcbf8004b20 con 0x7fcc00105820 2024-03-21T09:20:50.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 Processor -- start 2024-03-21T09:20:50.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 -- start start 2024-03-21T09:20:50.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a39dd0 0x559a98a38b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 0x559a98a39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a42460 0x559a98a44850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9899d250 con 0x559a98a39dd0 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a98945a30 con 0x559a98a42460 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcc09a06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a989992f0 con 0x559a98a390f0 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.437+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 0x559a98a39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a39dd0 0x559a98a38b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 0x559a98a39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49466/0 (socket says 172.21.15.37:49466) 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 -- 172.21.15.37:0/187745504 learned_addr learned my addr 172.21.15.37:0/187745504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbff400640 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a42460 0x559a98a44850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 -- 172.21.15.37:0/187745504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a42460 msgr2=0x559a98a44850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a42460 0x559a98a44850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 -- 172.21.15.37:0/187745504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a39dd0 msgr2=0x559a98a38b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a39dd0 0x559a98a38b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 -- 172.21.15.37:0/187745504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a98930690 con 0x559a98a390f0 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbff400640 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a39dd0 0x559a98a38b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbffe00640 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 0x559a98a39570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00115c0 tx=0x7fcbf0011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd12468501614093 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbf5200640 1 -- 172.21.15.37:0/187745504 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000f6c0 con 0x559a98a390f0 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbfcc00640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcbe40a3090 0x7fcbe40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbf5200640 1 -- 172.21.15.37:0/187745504 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbf000fd40 con 0x559a98a390f0 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.438+0000 7fcbf5200640 1 -- 172.21.15.37:0/187745504 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00183b0 con 0x559a98a390f0 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcbfcc00640 1 -- 172.21.15.37:0/1193289137 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcbf80d02a0 con 0x7fcc00105820 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcbff400640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcbe40a3090 0x7fcbe40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 -- 172.21.15.37:0/187745504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 msgr2=0x559a98a39570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 0x559a98a39570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00115c0 tx=0x7fcbf0011a80 comp rx=0 tx=0).stop 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 -- 172.21.15.37:0/187745504 shutdown_connections 2024-03-21T09:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a42460 0x559a98a44850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a390f0 0x559a98a39570 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 --2- 172.21.15.37:0/187745504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a39dd0 0x559a98a38b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.439+0000 7fcc09a06740 1 -- 172.21.15.37:0/187745504 >> 172.21.15.37:0/187745504 conn(0x559a98a00200 msgr2=0x559a98a33b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.440+0000 7fcc09a06740 1 -- 172.21.15.37:0/187745504 shutdown_connections 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.440+0000 7fcc09a06740 1 -- 172.21.15.37:0/187745504 wait complete. 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.440+0000 7fcbff400640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcbe40a3090 0x7fcbe40a5550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcbec02c220 tx=0x7fcbec009490 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.440+0000 7fcc09a06740 1 -- start start 2024-03-21T09:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.440+0000 7fcc09a06740 1 Processor -- start 2024-03-21T09:20:50.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a390f0 0x559a98a41b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 0x559a98a3eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a42460 0x559a98a3f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a98a452d0 con 0x559a98a390f0 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a98958e80 con 0x559a98a39dd0 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a98a45450 con 0x559a98a42460 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc09a06740 10 client.0 _ll_register_callbacks cb 0x559a9896b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcbff400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 0x559a98a3eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a390f0 0x559a98a41b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcc04c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a42460 0x559a98a3f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcbff400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 0x559a98a3eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54748/0 (socket says 172.21.15.37:54748) 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.441+0000 7fcbff400640 1 -- 172.21.15.37:0/2615033169 learned_addr learned my addr 172.21.15.37:0/2615033169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbff400640 1 -- 172.21.15.37:0/2615033169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a42460 msgr2=0x559a98a3f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbff400640 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a42460 0x559a98a3f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbff400640 1 -- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a390f0 msgr2=0x559a98a41b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbff400640 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a390f0 0x559a98a41b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbff400640 1 -- 172.21.15.37:0/2615033169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a9877b7d0 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbff400640 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 0x559a98a3eea0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbec036000 tx=0x7fcbec02bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bedf65f4fdc2a8cc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbe9600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbe9600640 1 -- 172.21.15.37:0/2615033169 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec050020 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbf000fd40 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbe9600640 1 -- 172.21.15.37:0/2615033169 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcbec0348c0 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcbe9600640 1 -- 172.21.15.37:0/2615033169 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbec05b050 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcc09a06740 10 client.27500 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.442+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559a986b13f0 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcbe9600640 1 -- 172.21.15.37:0/2615033169 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbf000fd40 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcbe9600640 1 -- 172.21.15.37:0/2615033169 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcbec04b030 con 0x559a98a39dd0 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcc09a06740 10 client.27500 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcc09a06740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcc09a06740 20 client.27500 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcc09a06740 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98a52310 0x559a98a54720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcc09a06740 4 client.27500 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a989aa790 con 0x559a98a52310 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.443+0000 7fcbffe00640 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98a52310 0x559a98a54720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.444+0000 7fcbffe00640 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98a52310 0x559a98a54720 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f3e477486b9353 server_cookie=e189e3184114a66b in_seq=0 out_seq=0 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.444+0000 7fcbe9600640 10 client.27500 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcbe9600640 1 -- 172.21.15.37:0/2615033169 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559a989aa790 con 0x559a98a52310 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcbe9600640 10 client.27500 handle_command_reply: tid=0 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 client.27500 shutdown 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 msgr2=0x559a98a3eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 0x559a98a3eea0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbec036000 tx=0x7fcbec02bc10 comp rx=0 tx=0).stop 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 shutdown_connections 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a42460 0x559a98a3f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98a52310 0x559a98a54720 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a39dd0 0x559a98a3eea0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 --2- 172.21.15.37:0/2615033169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a390f0 0x559a98a41b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcbe9600640 0 client.27500 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 >> 172.21.15.37:0/2615033169 conn(0x559a98a00200 msgr2=0x559a98a35890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 shutdown_connections 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 1 -- 172.21.15.37:0/2615033169 wait complete. 2024-03-21T09:20:50.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.445+0000 7fcc09a06740 20 client.27500 trim_cache size 0 max 16384 2024-03-21T09:20:50.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.482+0000 7fcc09a06740 1 Processor -- start 2024-03-21T09:20:50.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.482+0000 7fcc09a06740 1 -- start start 2024-03-21T09:20:50.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a43850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a41b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc09a06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001b7420 con 0x559a98a42460 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc09a06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001b4f20 con 0x559a98a390f0 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc09a06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001b2fe0 con 0x559a98a39dd0 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a43850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcc04c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a41b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a43850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54760/0 (socket says 172.21.15.37:54760) 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 -- 172.21.15.37:0/1465920849 learned_addr learned my addr 172.21.15.37:0/1465920849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49488/0 (socket says 172.21.15.37:49488) 2024-03-21T09:20:50.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 -- 172.21.15.37:0/1465920849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 msgr2=0x559a98a415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 -- 172.21.15.37:0/1465920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 msgr2=0x559a98a41b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a41b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbffe00640 1 -- 172.21.15.37:0/1465920849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a986710f0 con 0x559a98a390f0 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.483+0000 7fcbff400640 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.484+0000 7fcbffe00640 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a43850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009f10 tx=0x7fcbf0019780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2a76a79301c95a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.484+0000 7fcbebe00640 1 -- 172.21.15.37:0/1465920849 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0019510 con 0x559a98a390f0 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.484+0000 7fcbebe00640 1 -- 172.21.15.37:0/1465920849 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbf00050b0 con 0x559a98a390f0 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.484+0000 7fcbebe00640 1 -- 172.21.15.37:0/1465920849 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000f820 con 0x559a98a390f0 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 -- 172.21.15.37:0/1465920849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 msgr2=0x559a98a43850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a43850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009f10 tx=0x7fcbf0019780 comp rx=0 tx=0).stop 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 -- 172.21.15.37:0/1465920849 shutdown_connections 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a41b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 --2- 172.21.15.37:0/1465920849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a43850 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 -- 172.21.15.37:0/1465920849 >> 172.21.15.37:0/1465920849 conn(0x559a98a00200 msgr2=0x559a98a33670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 -- 172.21.15.37:0/1465920849 shutdown_connections 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.485+0000 7fcc09a06740 1 -- 172.21.15.37:0/1465920849 wait complete. 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 -- start start 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 Processor -- start 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a4a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001b35e0 con 0x559a98a42460 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001b48c0 con 0x559a98a390f0 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001b3c00 con 0x559a98a39dd0 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcc09a06740 10 client.0 _ll_register_callbacks cb 0x7fcbec037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a4a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcbff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a4a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54768/0 (socket says 172.21.15.37:54768) 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcbff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49496/0 (socket says 172.21.15.37:49496) 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.486+0000 7fcbffe00640 1 -- 172.21.15.37:0/4101373814 learned_addr learned my addr 172.21.15.37:0/4101373814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcc04c00640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbffe00640 1 -- 172.21.15.37:0/4101373814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 msgr2=0x559a98a468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbffe00640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbffe00640 1 -- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 msgr2=0x559a98a46e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbffe00640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a46e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbffe00640 1 -- 172.21.15.37:0/4101373814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a98a476a0 con 0x559a98a390f0 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbff400640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbffe00640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a4a550 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcbf001b040 tx=0x7fcbf0009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bd86ebbad79166 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbe9600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.487+0000 7fcbe9600640 1 -- 172.21.15.37:0/4101373814 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00195d0 con 0x559a98a390f0 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbf00050b0 con 0x559a98a390f0 2024-03-21T09:20:50.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcbe9600640 1 -- 172.21.15.37:0/4101373814 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcbf0019c70 con 0x559a98a390f0 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcbe9600640 1 -- 172.21.15.37:0/4101373814 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf003b050 con 0x559a98a390f0 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 10 client.27506 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559a98aaeca0 con 0x559a98a390f0 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcbe9600640 1 -- 172.21.15.37:0/4101373814 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbf00050b0 con 0x559a98a390f0 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcbe9600640 1 -- 172.21.15.37:0/4101373814 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcbf002c070 con 0x559a98a390f0 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 10 client.27506 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 20 client.27506 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:50.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98aaf120 0x559a98ab1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 4 client.27506 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.488+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559a989a4ae0 con 0x559a98aaf120 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.489+0000 7fcbff400640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98aaf120 0x559a98ab1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.489+0000 7fcbff400640 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98aaf120 0x559a98ab1510 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec99acc3080ea0a server_cookie=a6fae4062737b434 in_seq=0 out_seq=0 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.489+0000 7fcbe9600640 10 client.27506 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcbe9600640 1 -- 172.21.15.37:0/4101373814 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559a989a4ae0 con 0x559a98aaf120 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcbe9600640 10 client.27506 handle_command_reply: tid=0 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 client.27506 shutdown 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 msgr2=0x559a98a4a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a4a550 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcbf001b040 tx=0x7fcbf0009d00 comp rx=0 tx=0).stop 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 shutdown_connections 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a98a42460 0x559a98a46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a98a39dd0 0x559a98a468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559a98aaf120 0x559a98ab1510 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 --2- 172.21.15.37:0/4101373814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a98a390f0 0x559a98a4a550 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcbe9600640 0 client.27506 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 >> 172.21.15.37:0/4101373814 conn(0x559a98a00200 msgr2=0x559a98a32330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 shutdown_connections 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 1 -- 172.21.15.37:0/4101373814 wait complete. 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.490+0000 7fcc09a06740 20 client.27506 trim_cache size 0 max 16384 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:20:50.491 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:20:50.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcbe40a3090 msgr2=0x7fcbe40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcbe40a3090 0x7fcbe40a5550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcbec02c220 tx=0x7fcbec009490 comp rx=0 tx=0).stop 2024-03-21T09:20:50.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00105820 msgr2=0x7fcc001a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00105820 0x7fcc001a2c10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009fd0 tx=0x7fcbf800ef10 comp rx=0 tx=0).stop 2024-03-21T09:20:50.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 shutdown_connections 2024-03-21T09:20:50.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcbe40a3090 0x7fcbe40a5550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc00105820 0x7fcc001a2c10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc00100d40 0x7fcc0019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 --2- 172.21.15.37:0/1193289137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc00100420 0x7fcc0019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 >> 172.21.15.37:0/1193289137 conn(0x7fcc00077130 msgr2=0x7fcc001031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 shutdown_connections 2024-03-21T09:20:50.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.491+0000 7fcc05600640 1 -- 172.21.15.37:0/1193289137 wait complete. 2024-03-21T09:20:50.504 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:20:50.504 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:20:50.567 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:50 smithi037 ceph-mon[31830]: pgmap v266: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:50.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:50 smithi071 ceph-mon[36597]: pgmap v266: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:50.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:50 smithi053 ceph-mon[36169]: pgmap v266: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:50.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.705+0000 7fe276c00640 1 -- 172.21.15.37:0/2375804873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe270073ad0 msgr2=0x7fe270073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.705+0000 7fe276c00640 1 --2- 172.21.15.37:0/2375804873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe270073ad0 0x7fe270073eb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe264009900 tx=0x7fe26402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 -- 172.21.15.37:0/2375804873 shutdown_connections 2024-03-21T09:20:50.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 --2- 172.21.15.37:0/2375804873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe27010dd70 0x7fe2701101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 --2- 172.21.15.37:0/2375804873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2700743f0 0x7fe27010d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 --2- 172.21.15.37:0/2375804873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe270073ad0 0x7fe270073eb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 -- 172.21.15.37:0/2375804873 >> 172.21.15.37:0/2375804873 conn(0x7fe2700fd060 msgr2=0x7fe2700ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 -- 172.21.15.37:0/2375804873 shutdown_connections 2024-03-21T09:20:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.706+0000 7fe276c00640 1 -- 172.21.15.37:0/2375804873 wait complete. 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 Processor -- start 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 -- start start 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe270073ad0 0x7fe270199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2700743f0 0x7fe27019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 0x7fe27019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270113b60 con 0x7fe2700743f0 2024-03-21T09:20:50.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701139e0 con 0x7fe270073ad0 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270113ce0 con 0x7fe27010dd70 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 0x7fe27019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe275800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe270073ad0 0x7fe270199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.707+0000 7fe276200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 0x7fe27019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49502/0 (socket says 172.21.15.37:49502) 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 -- 172.21.15.37:0/957798844 learned_addr learned my addr 172.21.15.37:0/957798844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe274e00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2700743f0 0x7fe27019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 -- 172.21.15.37:0/957798844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe270073ad0 msgr2=0x7fe270199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe270073ad0 0x7fe270199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 -- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2700743f0 msgr2=0x7fe27019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2700743f0 0x7fe27019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 -- 172.21.15.37:0/957798844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27019efd0 con 0x7fe27010dd70 2024-03-21T09:20:50.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe275800640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe270073ad0 0x7fe270199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe274e00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2700743f0 0x7fe27019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.708+0000 7fe276200640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 0x7fe27019e8a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe26800eaa0 tx=0x7fe26800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.709+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26800cdb0 con 0x7fe27010dd70 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.709+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe268005020 con 0x7fe27010dd70 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.709+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2680116b0 con 0x7fe27010dd70 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.709+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27019f2c0 con 0x7fe27010dd70 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.709+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2701a6b50 con 0x7fe27010dd70 2024-03-21T09:20:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.710+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27004f470 con 0x7fe27010dd70 2024-03-21T09:20:50.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.712+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe268005560 con 0x7fe27010dd70 2024-03-21T09:20:50.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.713+0000 7fe26e000640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe2580a3100 0x7fe2580a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:50.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.714+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2680cfe80 con 0x7fe27010dd70 2024-03-21T09:20:50.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.714+0000 7fe275800640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe2580a3100 0x7fe2580a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:50.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.715+0000 7fe275800640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe2580a3100 0x7fe2580a55c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe264005ab0 tx=0x7fe264005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:50.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.715+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe268016030 con 0x7fe27010dd70 2024-03-21T09:20:50.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.964+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe270108ef0 con 0x7fe27010dd70 2024-03-21T09:20:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.965+0000 7fe26e000640 1 -- 172.21.15.37:0/957798844 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fe26809c590 con 0x7fe27010dd70 2024-03-21T09:20:50.961 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:20:50.961 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:20:50.961 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe2580a3100 msgr2=0x7fe2580a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe2580a3100 0x7fe2580a55c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe264005ab0 tx=0x7fe264005a20 comp rx=0 tx=0).stop 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 msgr2=0x7fe27019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 0x7fe27019e8a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe26800eaa0 tx=0x7fe26800ef60 comp rx=0 tx=0).stop 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 shutdown_connections 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe27010dd70 0x7fe27019e8a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2700743f0 0x7fe27019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe2580a3100 0x7fe2580a55c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 --2- 172.21.15.37:0/957798844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe270073ad0 0x7fe270199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 >> 172.21.15.37:0/957798844 conn(0x7fe2700fd060 msgr2=0x7fe2700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 shutdown_connections 2024-03-21T09:20:50.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:50.968+0000 7fe276c00640 1 -- 172.21.15.37:0/957798844 wait complete. 2024-03-21T09:20:50.993 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:20:50.993 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:20:51.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 -- 172.21.15.37:0/3868127580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d01074e0 msgr2=0x7f89d01078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 --2- 172.21.15.37:0/3868127580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d01074e0 0x7f89d01078c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009930 tx=0x7f89c002bc30 comp rx=0 tx=0).stop 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 -- 172.21.15.37:0/3868127580 shutdown_connections 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 --2- 172.21.15.37:0/3868127580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d0101e00 0x7f89d010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 --2- 172.21.15.37:0/3868127580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101440 0x7f89d01018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 --2- 172.21.15.37:0/3868127580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d01074e0 0x7f89d01078c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 -- 172.21.15.37:0/3868127580 >> 172.21.15.37:0/3868127580 conn(0x7f89d00fcf50 msgr2=0x7f89d00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 -- 172.21.15.37:0/3868127580 shutdown_connections 2024-03-21T09:20:51.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.200+0000 7f89cf200640 1 -- 172.21.15.37:0/3868127580 wait complete. 2024-03-21T09:20:51.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.201+0000 7f89cf200640 1 Processor -- start 2024-03-21T09:20:51.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 -- start start 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 0x7f89d00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101e00 0x7f89d0073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d01074e0 0x7f89d0074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0111830 con 0x7f89d01074e0 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01116b0 con 0x7f89d0101e00 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cf200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01119b0 con 0x7f89d0101440 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 0x7f89d00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 0x7f89d00739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49512/0 (socket says 172.21.15.37:49512) 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cde00640 1 -- 172.21.15.37:0/714149110 learned_addr learned my addr 172.21.15.37:0/714149110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.202+0000 7f89cd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101e00 0x7f89d0073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89ce800640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d01074e0 0x7f89d0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cde00640 1 -- 172.21.15.37:0/714149110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101e00 msgr2=0x7f89d0073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cde00640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101e00 0x7f89d0073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cde00640 1 -- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d01074e0 msgr2=0x7f89d0074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cde00640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d01074e0 0x7f89d0074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cde00640 1 -- 172.21.15.37:0/714149110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d01a6250 con 0x7f89d0101440 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cd400640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101e00 0x7f89d0073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89ce800640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d01074e0 0x7f89d0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:51.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cde00640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 0x7f89d00739a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009900 tx=0x7f89c0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89c6a00640 1 -- 172.21.15.37:0/714149110 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0043020 con 0x7f89d0101440 2024-03-21T09:20:51.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89c6a00640 1 -- 172.21.15.37:0/714149110 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89c00054e0 con 0x7f89d0101440 2024-03-21T09:20:51.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89c6a00640 1 -- 172.21.15.37:0/714149110 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0003b80 con 0x7f89d0101440 2024-03-21T09:20:51.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.203+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d01a64e0 con 0x7f89d0101440 2024-03-21T09:20:51.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.204+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d01a6a30 con 0x7f89d0101440 2024-03-21T09:20:51.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89c6a00640 1 -- 172.21.15.37:0/714149110 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f89c0046020 con 0x7f89d0101440 2024-03-21T09:20:51.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 Processor -- start 2024-03-21T09:20:51.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 -- start start 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 0x55c8a82c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c7260 0x55c8a82cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82cfde0 0x55c8a82d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a822a2c0 con 0x55c8a82cfde0 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a81d2c90 con 0x55c8a82c68b0 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.207+0000 7f89d7e16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a8226360 con 0x55c8a82c7260 2024-03-21T09:20:51.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 0x55c8a82c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 0x55c8a82c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54802/0 (socket says 172.21.15.37:54802) 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 -- 172.21.15.37:0/3412778337 learned_addr learned my addr 172.21.15.37:0/3412778337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89cde00640 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c7260 0x55c8a82cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89cd400640 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82cfde0 0x55c8a82d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 -- 172.21.15.37:0/3412778337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c7260 msgr2=0x55c8a82cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c7260 0x55c8a82cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 -- 172.21.15.37:0/3412778337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82cfde0 msgr2=0x55c8a82d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82cfde0 0x55c8a82d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 -- 172.21.15.37:0/3412778337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a81f7080 con 0x55c8a82c68b0 2024-03-21T09:20:51.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.208+0000 7f89ce800640 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 0x55c8a82c6c90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009f30 tx=0x7f89c8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f95afb11a2db465 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89c6a00640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f89b40a3090 0x7f89b40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89bb400640 1 -- 172.21.15.37:0/3412778337 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8016070 con 0x55c8a82c68b0 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89bb400640 1 -- 172.21.15.37:0/3412778337 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89c801bb40 con 0x55c8a82c68b0 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89bb400640 1 -- 172.21.15.37:0/3412778337 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8012cb0 con 0x55c8a82c68b0 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89c6a00640 1 -- 172.21.15.37:0/714149110 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f89c00f1c80 con 0x7f89d0101440 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89cde00640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f89b40a3090 0x7f89b40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89d7e16740 1 -- 172.21.15.37:0/3412778337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 msgr2=0x55c8a82c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.209+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 0x55c8a82c6c90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009f30 tx=0x7f89c8010800 comp rx=0 tx=0).stop 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89cde00640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f89b40a3090 0x7f89b40a5550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55c8a82c6380 tx=0x7f89c00de040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 -- 172.21.15.37:0/3412778337 shutdown_connections 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82cfde0 0x55c8a82d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c7260 0x55c8a82cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3412778337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c68b0 0x55c8a82c6c90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 -- 172.21.15.37:0/3412778337 >> 172.21.15.37:0/3412778337 conn(0x55c8a82c0f80 msgr2=0x55c8a82c1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 -- 172.21.15.37:0/3412778337 shutdown_connections 2024-03-21T09:20:51.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 -- 172.21.15.37:0/3412778337 wait complete. 2024-03-21T09:20:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 -- start start 2024-03-21T09:20:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.210+0000 7f89d7e16740 1 Processor -- start 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a82d2d50 con 0x55c8a82c7260 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a81e60d0 con 0x55c8a82cfde0 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a82d2ed0 con 0x55c8a82c68b0 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49522/0 (socket says 172.21.15.37:49522) 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89ce800640 1 -- 172.21.15.37:0/1904159587 learned_addr learned my addr 172.21.15.37:0/1904159587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89d7e16740 10 client.0 _ll_register_callbacks cb 0x55c8a81eae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89cd400640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.211+0000 7f89ce800640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89ce800640 1 -- 172.21.15.37:0/1904159587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 msgr2=0x55c8a82ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89ce800640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89ce800640 1 -- 172.21.15.37:0/1904159587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 msgr2=0x55c8a82cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89ce800640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89ce800640 1 -- 172.21.15.37:0/1904159587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a7f3e3f0 con 0x55c8a82c68b0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89ce800640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82cf6a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89c801bf50 tx=0x7f89c8012890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c19f422676f9e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89afe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89afe00640 1 -- 172.21.15.37:0/1904159587 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8016070 con 0x55c8a82c68b0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89afe00640 1 -- 172.21.15.37:0/1904159587 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f89c8004d00 con 0x55c8a82c68b0 2024-03-21T09:20:51.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89c801bb40 con 0x55c8a82c68b0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89afe00640 1 -- 172.21.15.37:0/1904159587 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c8012a30 con 0x55c8a82c68b0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89d7e16740 10 client.27550 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.212+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c8a80087d0 con 0x55c8a82c68b0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89afe00640 1 -- 172.21.15.37:0/1904159587 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89c801bb40 con 0x55c8a82c68b0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89afe00640 1 -- 172.21.15.37:0/1904159587 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f89c800f040 con 0x55c8a82c68b0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89d7e16740 10 client.27550 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89d7e16740 10 client.27550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89d7e16740 20 client.27550 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a82dfbb0 0x55c8a82e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89d7e16740 4 client.27550 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8a8237800 con 0x55c8a82dfbb0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.213+0000 7f89cd400640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a82dfbb0 0x55c8a82e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.214+0000 7f89cd400640 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a82dfbb0 0x55c8a82e1fa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cabcc5f733fba3b server_cookie=eb53d0bfea11a35 in_seq=0 out_seq=0 2024-03-21T09:20:51.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.214+0000 7f89afe00640 10 client.27550 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:51.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89afe00640 1 -- 172.21.15.37:0/1904159587 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c8a8237800 con 0x55c8a82dfbb0 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89afe00640 10 client.27550 handle_command_reply: tid=0 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 client.27550 shutdown 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 msgr2=0x55c8a82cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82cf6a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89c801bf50 tx=0x7f89c8012890 comp rx=0 tx=0).stop 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 shutdown_connections 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a82dfbb0 0x55c8a82e1fa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1904159587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82cf6a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89afe00640 0 client.27550 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 >> 172.21.15.37:0/1904159587 conn(0x55c8a82c0f80 msgr2=0x55c8a82c2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 shutdown_connections 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 1 -- 172.21.15.37:0/1904159587 wait complete. 2024-03-21T09:20:51.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.215+0000 7f89d7e16740 20 client.27550 trim_cache size 0 max 16384 2024-03-21T09:20:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 Processor -- start 2024-03-21T09:20:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 -- start start 2024-03-21T09:20:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c68b0 0x55c8a82bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 0x55c8a82cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82cfde0 0x55c8a82cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01b30a0 con 0x55c8a82c68b0 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01b0ba0 con 0x55c8a82c7260 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.229+0000 7f89d7e16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01aec60 con 0x55c8a82cfde0 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 0x55c8a82cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89ce800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c68b0 0x55c8a82bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 0x55c8a82cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54818/0 (socket says 172.21.15.37:54818) 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 -- 172.21.15.37:0/1645785373 learned_addr learned my addr 172.21.15.37:0/1645785373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89ce800640 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82cfde0 0x55c8a82cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 -- 172.21.15.37:0/1645785373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82cfde0 msgr2=0x55c8a82cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82cfde0 0x55c8a82cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 -- 172.21.15.37:0/1645785373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c68b0 msgr2=0x55c8a82bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c68b0 0x55c8a82bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 -- 172.21.15.37:0/1645785373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a82c63b0 con 0x55c8a82c7260 2024-03-21T09:20:51.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.230+0000 7f89cd400640 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 0x55c8a82cf160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f89bc004700 tx=0x7f89bc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb6269d8c7028f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89baa00640 1 -- 172.21.15.37:0/1645785373 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc012110 con 0x55c8a82c7260 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89baa00640 1 -- 172.21.15.37:0/1645785373 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89bc005020 con 0x55c8a82c7260 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89baa00640 1 -- 172.21.15.37:0/1645785373 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc004a90 con 0x55c8a82c7260 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89d7e16740 1 -- 172.21.15.37:0/1645785373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 msgr2=0x55c8a82cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 0x55c8a82cf160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f89bc004700 tx=0x7f89bc005f90 comp rx=0 tx=0).stop 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89d7e16740 1 -- 172.21.15.37:0/1645785373 shutdown_connections 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82cfde0 0x55c8a82cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.231+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82c7260 0x55c8a82cf160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.232+0000 7f89d7e16740 1 --2- 172.21.15.37:0/1645785373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c68b0 0x55c8a82bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.232+0000 7f89d7e16740 1 -- 172.21.15.37:0/1645785373 >> 172.21.15.37:0/1645785373 conn(0x55c8a82c0f80 msgr2=0x55c8a82c7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.232+0000 7f89d7e16740 1 -- 172.21.15.37:0/1645785373 shutdown_connections 2024-03-21T09:20:51.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.232+0000 7f89d7e16740 1 -- 172.21.15.37:0/1645785373 wait complete. 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.232+0000 7f89d7e16740 1 -- start start 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.232+0000 7f89d7e16740 1 Processor -- start 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01af260 con 0x55c8a82c7260 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01b0540 con 0x55c8a82cfde0 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01af880 con 0x55c8a82c68b0 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89d7e16740 10 client.0 _ll_register_callbacks cb 0x7f89c8009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82d44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49530/0 (socket says 172.21.15.37:49530) 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89ce800640 1 -- 172.21.15.37:0/3689117918 learned_addr learned my addr 172.21.15.37:0/3689117918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89ce800640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.233+0000 7f89cd400640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89ce800640 1 -- 172.21.15.37:0/3689117918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 msgr2=0x55c8a82d4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89ce800640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82d4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89ce800640 1 -- 172.21.15.37:0/3689117918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 msgr2=0x55c8a82d4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89ce800640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82d4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89ce800640 1 -- 172.21.15.37:0/3689117918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a8343950 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89ce800640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82d44e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89c8010e80 tx=0x7f89c8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab156401665e699d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89afe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89afe00640 1 -- 172.21.15.37:0/3689117918 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8016040 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89afe00640 1 -- 172.21.15.37:0/3689117918 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f89c8012bd0 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89bc005020 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89afe00640 1 -- 172.21.15.37:0/3689117918 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c8049050 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89d7e16740 10 client.27556 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.234+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c8a8344bb0 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89afe00640 1 -- 172.21.15.37:0/3689117918 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89bc005020 con 0x55c8a82c68b0 2024-03-21T09:20:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89afe00640 1 -- 172.21.15.37:0/3689117918 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f89c801b460 con 0x55c8a82c68b0 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89d7e16740 10 client.27556 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89d7e16740 10 client.27556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89d7e16740 20 client.27556 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a83451c0 0x55c8a83475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89d7e16740 4 client.27556 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8a8231b50 con 0x55c8a83451c0 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.235+0000 7f89cd400640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a83451c0 0x55c8a83475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.236+0000 7f89cd400640 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a83451c0 0x55c8a83475b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d2b6b1d465d7f03 server_cookie=cfe76eb88691d094 in_seq=0 out_seq=0 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.236+0000 7f89afe00640 10 client.27556 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.236+0000 7f89afe00640 1 -- 172.21.15.37:0/3689117918 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8a8231b50 con 0x55c8a83451c0 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.236+0000 7f89afe00640 10 client.27556 handle_command_reply: tid=0 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.236+0000 7f89d7e16740 1 client.27556 shutdown 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 msgr2=0x55c8a82d44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82d44e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89c8010e80 tx=0x7f89c8000f40 comp rx=0 tx=0).stop 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 shutdown_connections 2024-03-21T09:20:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8a82cfde0 0x55c8a82d4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55c8a83451c0 0x55c8a83475b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8a82c7260 0x55c8a82d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 --2- 172.21.15.37:0/3689117918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8a82c68b0 0x55c8a82d44e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 >> 172.21.15.37:0/3689117918 conn(0x55c8a82c0f80 msgr2=0x55c8a82c2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89afe00640 0 client.27556 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 shutdown_connections 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 1 -- 172.21.15.37:0/3689117918 wait complete. 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.237+0000 7f89d7e16740 20 client.27556 trim_cache size 0 max 16384 2024-03-21T09:20:51.236 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f89b40a3090 msgr2=0x7f89b40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f89b40a3090 0x7f89b40a5550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55c8a82c6380 tx=0x7f89c00de040 comp rx=0 tx=0).stop 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 msgr2=0x7f89d00739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 0x7f89d00739a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009900 tx=0x7f89c0004360 comp rx=0 tx=0).stop 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 shutdown_connections 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d01074e0 0x7f89d0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f89b40a3090 0x7f89b40a5550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d0101e00 0x7f89d0073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 --2- 172.21.15.37:0/714149110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89d0101440 0x7f89d00739a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 >> 172.21.15.37:0/714149110 conn(0x7f89d00fcf50 msgr2=0x7f89d00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 shutdown_connections 2024-03-21T09:20:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.238+0000 7f89cf200640 1 -- 172.21.15.37:0/714149110 wait complete. 2024-03-21T09:20:51.252 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:20:51.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:20:51.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 -- 172.21.15.37:0/3291586803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 msgr2=0x7f54ec112100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 --2- 172.21.15.37:0/3291586803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 0x7f54ec112100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009900 tx=0x7f54d402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 -- 172.21.15.37:0/3291586803 shutdown_connections 2024-03-21T09:20:51.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 --2- 172.21.15.37:0/3291586803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec114a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 --2- 172.21.15.37:0/3291586803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 0x7f54ec112100 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 --2- 172.21.15.37:0/3291586803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec073a40 0x7f54ec073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 -- 172.21.15.37:0/3291586803 >> 172.21.15.37:0/3291586803 conn(0x7f54ec0fd140 msgr2=0x7f54ec0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 -- 172.21.15.37:0/3291586803 shutdown_connections 2024-03-21T09:20:51.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.479+0000 7f54f3600640 1 -- 172.21.15.37:0/3291586803 wait complete. 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.480+0000 7f54f3600640 1 Processor -- start 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 -- start start 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec073a40 0x7f54ec19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 0x7f54ec19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec1a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec118090 con 0x7f54ec073a40 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec117f10 con 0x7f54ec074360 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f3600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec118210 con 0x7f54ec112640 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec1a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec073a40 0x7f54ec19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 0x7f54ec19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec1a2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49558/0 (socket says 172.21.15.37:49558) 2024-03-21T09:20:51.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.481+0000 7f54f1600640 1 -- 172.21.15.37:0/1126348966 learned_addr learned my addr 172.21.15.37:0/1126348966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f1600640 1 -- 172.21.15.37:0/1126348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 msgr2=0x7f54ec19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f1600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 0x7f54ec19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f1600640 1 -- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec073a40 msgr2=0x7f54ec19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f1600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec073a40 0x7f54ec19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f1600640 1 -- 172.21.15.37:0/1126348966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec1a35a0 con 0x7f54ec112640 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f1600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec1a2e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f54e400e970 tx=0x7f54e400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54ea000640 1 -- 172.21.15.37:0/1126348966 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400ccc0 con 0x7f54ec112640 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54ea000640 1 -- 172.21.15.37:0/1126348966 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54e4005500 con 0x7f54ec112640 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec1a3890 con 0x7f54ec112640 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54ea000640 1 -- 172.21.15.37:0/1126348966 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4010690 con 0x7f54ec112640 2024-03-21T09:20:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.482+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec1ab120 con 0x7f54ec112640 2024-03-21T09:20:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.485+0000 7f54f63e1740 1 Processor -- start 2024-03-21T09:20:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.485+0000 7f54ea000640 1 -- 172.21.15.37:0/1126348966 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f54e40046c0 con 0x7f54ec112640 2024-03-21T09:20:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 -- start start 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887004f2c0 con 0x5588700eb6d0 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886fff7c90 con 0x5588700f4c00 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f63e1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887004b360 con 0x5588700ec080 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49570/0 (socket says 172.21.15.37:49570) 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.486+0000 7f54f0c00640 1 -- 172.21.15.37:0/1112562955 learned_addr learned my addr 172.21.15.37:0/1112562955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54e3e00640 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54f0c00640 1 -- 172.21.15.37:0/1112562955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 msgr2=0x5588700f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54f0c00640 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54f0c00640 1 -- 172.21.15.37:0/1112562955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 msgr2=0x5588700ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54f0c00640 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54f0c00640 1 -- 172.21.15.37:0/1112562955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887001c080 con 0x5588700ec080 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54e3e00640 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54ea000640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f54d00a3110 0x7f54d00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54ea000640 1 -- 172.21.15.37:0/1126348966 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f54e40cff20 con 0x7f54ec112640 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54f0c00640 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f4630 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0114f0 tx=0x7f54dc0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8f935045e647dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54e3e00640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f54d00a3110 0x7f54d00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54e1c00640 1 -- 172.21.15.37:0/1112562955 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc01e070 con 0x5588700ec080 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.487+0000 7f54e1c00640 1 -- 172.21.15.37:0/1112562955 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54dc00fb00 con 0x5588700ec080 2024-03-21T09:20:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54e1c00640 1 -- 172.21.15.37:0/1112562955 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc019b90 con 0x5588700ec080 2024-03-21T09:20:51.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 -- 172.21.15.37:0/1112562955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 msgr2=0x5588700f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54e3e00640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f54d00a3110 0x7f54d00a55d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54d4005ab0 tx=0x7f54d4005b50 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f4630 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0114f0 tx=0x7f54dc0119b0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 -- 172.21.15.37:0/1112562955 shutdown_connections 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f4630 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 --2- 172.21.15.37:0/1112562955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 -- 172.21.15.37:0/1112562955 >> 172.21.15.37:0/1112562955 conn(0x5588700e5da0 msgr2=0x5588700e6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 -- 172.21.15.37:0/1112562955 shutdown_connections 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.488+0000 7f54f63e1740 1 -- 172.21.15.37:0/1112562955 wait complete. 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.489+0000 7f54f63e1740 1 -- start start 2024-03-21T09:20:51.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.489+0000 7f54f63e1740 1 Processor -- start 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.489+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f63e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588700f7b70 con 0x5588700eb6d0 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f63e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887000b0d0 con 0x5588700f4c00 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f63e1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588700f7cf0 con 0x5588700ec080 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f63e1740 10 client.0 _ll_register_callbacks cb 0x55887000fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36036/0 (socket says 172.21.15.37:36036) 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 -- 172.21.15.37:0/3786818359 learned_addr learned my addr 172.21.15.37:0/3786818359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f1600640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54e3e00640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 -- 172.21.15.37:0/3786818359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 msgr2=0x5588700f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 -- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 msgr2=0x5588700f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f0c00640 1 -- 172.21.15.37:0/3786818359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55886fd633f0 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.490+0000 7f54f1600640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54f0c00640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700f44e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f54dc01c040 tx=0x7f54dc019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c800b182ab0c74dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54da000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54da000640 1 -- 172.21.15.37:0/3786818359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc01e040 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54da000640 1 -- 172.21.15.37:0/3786818359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f54dc004a40 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54dc00fb00 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54da000640 1 -- 172.21.15.37:0/3786818359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54dc032050 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54f63e1740 10 client.17301 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.491+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55886fe2d7d0 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54da000640 1 -- 172.21.15.37:0/3786818359 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54dc00fb00 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54da000640 1 -- 172.21.15.37:0/3786818359 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f54dc005040 con 0x5588700eb6d0 2024-03-21T09:20:51.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54f63e1740 10 client.17301 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:51.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54f63e1740 10 client.17301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:20:51.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54f63e1740 20 client.17301 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:51.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54f63e1740 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870104980 0x558870106d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54f63e1740 4 client.17301 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:51.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55887005c800 con 0x558870104980 2024-03-21T09:20:51.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.492+0000 7f54e3e00640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870104980 0x558870106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.493+0000 7f54e3e00640 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870104980 0x558870106d70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8db4d296fe0a818c server_cookie=9702e8d223fd6acb in_seq=0 out_seq=0 2024-03-21T09:20:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.493+0000 7f54da000640 10 client.17301 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54da000640 1 -- 172.21.15.37:0/3786818359 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55887005c800 con 0x558870104980 2024-03-21T09:20:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54da000640 10 client.17301 handle_command_reply: tid=0 2024-03-21T09:20:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 client.17301 shutdown 2024-03-21T09:20:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 msgr2=0x5588700f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700f44e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f54dc01c040 tx=0x7f54dc019410 comp rx=0 tx=0).stop 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 shutdown_connections 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870104980 0x558870106d70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700f4c00 0x5588700f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700ec080 0x5588700f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 --2- 172.21.15.37:0/3786818359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700f44e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54da000640 0 client.17301 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 >> 172.21.15.37:0/3786818359 conn(0x5588700e5da0 msgr2=0x5588700e7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 shutdown_connections 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 1 -- 172.21.15.37:0/3786818359 wait complete. 2024-03-21T09:20:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.494+0000 7f54f63e1740 20 client.17301 trim_cache size 0 max 16384 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 Processor -- start 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 -- start start 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700f4c00 0x5588700f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1b7760 con 0x5588700eb6d0 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1b5260 con 0x5588700ec080 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.508+0000 7f54f63e1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1b3320 con 0x5588700f4c00 2024-03-21T09:20:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54864/0 (socket says 172.21.15.37:54864) 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 -- 172.21.15.37:0/4202678654 learned_addr learned my addr 172.21.15.37:0/4202678654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54f0c00640 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54f1600640 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700f4c00 0x5588700f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 -- 172.21.15.37:0/4202678654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700f4c00 msgr2=0x5588700f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700f4c00 0x5588700f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 -- 172.21.15.37:0/4202678654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 msgr2=0x5588700e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 -- 172.21.15.37:0/4202678654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588700f7a50 con 0x5588700ec080 2024-03-21T09:20:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54f1600640 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700f4c00 0x5588700f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:51.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e3e00640 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f3fa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f54d4004500 tx=0x7f54d403b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5423e702b1c1314 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.509+0000 7f54e1200640 1 -- 172.21.15.37:0/4202678654 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d402ce20 con 0x5588700ec080 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54e1200640 1 -- 172.21.15.37:0/4202678654 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54d405c070 con 0x5588700ec080 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54e1200640 1 -- 172.21.15.37:0/4202678654 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d40053c0 con 0x5588700ec080 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 -- 172.21.15.37:0/4202678654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 msgr2=0x5588700f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f3fa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f54d4004500 tx=0x7f54d403b000 comp rx=0 tx=0).stop 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 -- 172.21.15.37:0/4202678654 shutdown_connections 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700f4c00 0x5588700f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f3fa0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 --2- 172.21.15.37:0/4202678654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700eb6d0 0x5588700e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 -- 172.21.15.37:0/4202678654 >> 172.21.15.37:0/4202678654 conn(0x5588700e5da0 msgr2=0x5588700f08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 -- 172.21.15.37:0/4202678654 shutdown_connections 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.510+0000 7f54f63e1740 1 -- 172.21.15.37:0/4202678654 wait complete. 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 -- start start 2024-03-21T09:20:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 Processor -- start 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 0x5588700f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 0x55887015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1b3920 con 0x5588700f4c00 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1b4c00 con 0x5588700ec080 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1b3f40 con 0x5588700eb6d0 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.511+0000 7f54f63e1740 10 client.0 _ll_register_callbacks cb 0x7f54dc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54f1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 0x55887015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 0x5588700f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54874/0 (socket says 172.21.15.37:54874) 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 -- 172.21.15.37:0/2418496151 learned_addr learned my addr 172.21.15.37:0/2418496151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54f0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 0x5588700f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49582/0 (socket says 172.21.15.37:49582) 2024-03-21T09:20:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54f1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 0x55887015f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36056/0 (socket says 172.21.15.37:36056) 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 -- 172.21.15.37:0/2418496151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 msgr2=0x5588700f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 0x5588700f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 -- 172.21.15.37:0/2418496151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 msgr2=0x55887015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 0x55887015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 -- 172.21.15.37:0/2418496151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588700f94e0 con 0x5588700ec080 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54f0c00640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 0x5588700f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54f1600640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 0x55887015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54e3e00640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f8c10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f54d402c200 tx=0x7f54d4005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd6e9d70dcafc3c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.512+0000 7f54da000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54da000640 1 -- 172.21.15.37:0/2418496151 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4005570 con 0x5588700ec080 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54da000640 1 -- 172.21.15.37:0/2418496151 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f54d4055070 con 0x5588700ec080 2024-03-21T09:20:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54d405c070 con 0x5588700ec080 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54da000640 1 -- 172.21.15.37:0/2418496151 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54d40046c0 con 0x5588700ec080 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 10 client.27548 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5588701606c0 con 0x5588700ec080 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54da000640 1 -- 172.21.15.37:0/2418496151 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54d405c070 con 0x5588700ec080 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54da000640 1 -- 172.21.15.37:0/2418496151 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f54d4003bf0 con 0x5588700ec080 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 10 client.27548 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 10 client.27548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 20 client.27548 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870160e10 0x558870163230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 4 client.27548 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.513+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558870056b50 con 0x558870160e10 2024-03-21T09:20:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f0c00640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870160e10 0x558870163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f0c00640 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870160e10 0x558870163230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49414966156fbb39 server_cookie=d8fc06152ffa31c7 in_seq=0 out_seq=0 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54da000640 10 client.27548 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54da000640 1 -- 172.21.15.37:0/2418496151 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558870056b50 con 0x558870160e10 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54da000640 10 client.27548 handle_command_reply: tid=0 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 client.27548 shutdown 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 msgr2=0x5588700f8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f8c10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f54d402c200 tx=0x7f54d4005140 comp rx=0 tx=0).stop 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 shutdown_connections 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x558870160e10 0x558870163230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588700f4c00 0x55887015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588700ec080 0x5588700f8c10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54da000640 0 client.27548 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:20:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 --2- 172.21.15.37:0/2418496151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588700eb6d0 0x5588700f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.514+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 >> 172.21.15.37:0/2418496151 conn(0x5588700e5da0 msgr2=0x5588700e4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 shutdown_connections 2024-03-21T09:20:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f63e1740 1 -- 172.21.15.37:0/2418496151 wait complete. 2024-03-21T09:20:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f63e1740 20 client.27548 trim_cache size 0 max 16384 2024-03-21T09:20:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f54d00a3110 msgr2=0x7f54d00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f3600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f54d00a3110 0x7f54d00a55d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54d4005ab0 tx=0x7f54d4005b50 comp rx=0 tx=0).stop 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 msgr2=0x7f54ec1a2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.515+0000 7f54f3600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec1a2e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f54e400e970 tx=0x7f54e400ee30 comp rx=0 tx=0).stop 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 shutdown_connections 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ec112640 0x7f54ec1a2e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f54d00a3110 0x7f54d00a55d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54ec074360 0x7f54ec19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 --2- 172.21.15.37:0/1126348966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec073a40 0x7f54ec19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 >> 172.21.15.37:0/1126348966 conn(0x7f54ec0fd140 msgr2=0x7f54ec0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 shutdown_connections 2024-03-21T09:20:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.516+0000 7f54f3600640 1 -- 172.21.15.37:0/1126348966 wait complete. 2024-03-21T09:20:51.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:51 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/957798844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:20:51.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:51 smithi037 ceph-mon[31830]: pgmap v267: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:51.532 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:20:51.532 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:20:51.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:51 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/957798844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:20:51.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:51 smithi071 ceph-mon[36597]: pgmap v267: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:51.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:51 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/957798844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:20:51.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:51 smithi053 ceph-mon[36169]: pgmap v267: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:51.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.741+0000 7fb4e6200640 1 -- 172.21.15.37:0/4139993322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102630 msgr2=0x7fb4e0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.741+0000 7fb4e6200640 1 --2- 172.21.15.37:0/4139993322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102630 0x7fb4e0102a10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009900 tx=0x7fb4d402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 -- 172.21.15.37:0/4139993322 shutdown_connections 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 --2- 172.21.15.37:0/4139993322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e010ba50 0x7fb4e010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 --2- 172.21.15.37:0/4139993322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102f50 0x7fb4e010b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 --2- 172.21.15.37:0/4139993322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102630 0x7fb4e0102a10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 -- 172.21.15.37:0/4139993322 >> 172.21.15.37:0/4139993322 conn(0x7fb4e0077130 msgr2=0x7fb4e00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 -- 172.21.15.37:0/4139993322 shutdown_connections 2024-03-21T09:20:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.742+0000 7fb4e6200640 1 -- 172.21.15.37:0/4139993322 wait complete. 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 Processor -- start 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 -- start start 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 0x7fb4e0100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102f50 0x7fb4e01007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e010ba50 0x7fb4e0100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e0111850 con 0x7fb4e0102630 2024-03-21T09:20:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01116d0 con 0x7fb4e0102f50 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e6200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01119d0 con 0x7fb4e010ba50 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 0x7fb4e0100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4dfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102f50 0x7fb4e01007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 0x7fb4e0100280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36086/0 (socket says 172.21.15.37:36086) 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.743+0000 7fb4e4e00640 1 -- 172.21.15.37:0/2009400849 learned_addr learned my addr 172.21.15.37:0/2009400849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4e5800640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e010ba50 0x7fb4e0100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2009400849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e010ba50 msgr2=0x7fb4e0100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e010ba50 0x7fb4e0100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 msgr2=0x7fb4e0100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 0x7fb4e0100280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2009400849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e01a6210 con 0x7fb4e0102f50 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 0x7fb4e0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102f50 0x7fb4e01007c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e9f0 tx=0x7fb4cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dd600640 1 -- 172.21.15.37:0/2009400849 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00ccc0 con 0x7fb4e0102f50 2024-03-21T09:20:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dd600640 1 -- 172.21.15.37:0/2009400849 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc010910 con 0x7fb4e0102f50 2024-03-21T09:20:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e01a6500 con 0x7fb4e0102f50 2024-03-21T09:20:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.744+0000 7fb4dd600640 1 -- 172.21.15.37:0/2009400849 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc018960 con 0x7fb4e0102f50 2024-03-21T09:20:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.745+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e01a6a50 con 0x7fb4e0102f50 2024-03-21T09:20:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 Processor -- start 2024-03-21T09:20:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 -- start start 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fbfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556de9fcb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de9f232c0 con 0x556de9fc0260 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de9ecbc90 con 0x556de9fc8de0 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de9f1f360 con 0x556de9fbf8b0 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.749+0000 7fb4e5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fbfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fbfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49606/0 (socket says 172.21.15.37:49606) 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 -- 172.21.15.37:0/3275757327 learned_addr learned my addr 172.21.15.37:0/3275757327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4dd600640 1 -- 172.21.15.37:0/2009400849 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb4cc010ac0 con 0x7fb4e0102f50 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 -- 172.21.15.37:0/3275757327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 msgr2=0x556de9fcb230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:20:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556de9fcb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 -- 172.21.15.37:0/3275757327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 msgr2=0x556de9fc8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 -- 172.21.15.37:0/3275757327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556de9ef0080 con 0x556de9fbf8b0 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.750+0000 7fb4e5800640 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fbfc90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009f50 tx=0x7fb4d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a9d99f8175a715a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4d2000640 1 -- 172.21.15.37:0/3275757327 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8004d20 con 0x556de9fbf8b0 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4d2000640 1 -- 172.21.15.37:0/3275757327 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8004050 con 0x556de9fbf8b0 2024-03-21T09:20:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4d2000640 1 -- 172.21.15.37:0/3275757327 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8005280 con 0x556de9fbf8b0 2024-03-21T09:20:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4dd600640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb4c80a3090 0x7fb4c80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 -- 172.21.15.37:0/3275757327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 msgr2=0x556de9fbfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fbfc90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009f50 tx=0x7fb4d80118d0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4dd600640 1 -- 172.21.15.37:0/2009400849 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fb4cc0cfff0 con 0x7fb4e0102f50 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 -- 172.21.15.37:0/3275757327 shutdown_connections 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556de9fcb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/3275757327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fbfc90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4ea57b740 1 -- 172.21.15.37:0/3275757327 >> 172.21.15.37:0/3275757327 conn(0x556de9fb9f80 msgr2=0x556de9fba360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.751+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb4c80a3090 0x7fb4c80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.752+0000 7fb4ea57b740 1 -- 172.21.15.37:0/3275757327 shutdown_connections 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.752+0000 7fb4ea57b740 1 -- 172.21.15.37:0/3275757327 wait complete. 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.752+0000 7fb4ea57b740 1 -- start start 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.752+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb4c80a3090 0x7fb4c80a5550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x556de9fc0660 tx=0x7fb4d4005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.752+0000 7fb4ea57b740 1 Processor -- start 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fc87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc8de0 0x556de9fc6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de9fcbd50 con 0x556de9fc0260 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de9edf0d0 con 0x556de9fbf8b0 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de9fcbed0 con 0x556de9fc8de0 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4ea57b740 10 client.0 _ll_register_callbacks cb 0x556de9ee3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fc87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fc87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54912/0 (socket says 172.21.15.37:54912) 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4e5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc8de0 0x556de9fc6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4e4e00640 1 -- 172.21.15.37:0/1052534870 learned_addr learned my addr 172.21.15.37:0/1052534870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.753+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4e4e00640 1 -- 172.21.15.37:0/1052534870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc8de0 msgr2=0x556de9fc6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc8de0 0x556de9fc6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4e4e00640 1 -- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 msgr2=0x556de9fc5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4e4e00640 1 -- 172.21.15.37:0/1052534870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556de9c373f0 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fc87e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4d402c2d0 tx=0x7fb4d4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0f4d49bcfa464f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4c6a00640 1 -- 172.21.15.37:0/1052534870 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4050020 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4d8004050 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.754+0000 7fb4c6a00640 1 -- 172.21.15.37:0/1052534870 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fb4d402cb20 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4c6a00640 1 -- 172.21.15.37:0/1052534870 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d405b050 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 10 client.27557 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556de9d017d0 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4c6a00640 1 -- 172.21.15.37:0/1052534870 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8004050 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4c6a00640 1 -- 172.21.15.37:0/1052534870 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb4d4003710 con 0x556de9fbf8b0 2024-03-21T09:20:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 10 client.27557 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 10 client.27557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 20 client.27557 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556de9fd8ce0 0x556de9fdb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 4 client.27557 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556de9f30800 con 0x556de9fd8ce0 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.755+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556de9fd8ce0 0x556de9fdb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.756+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556de9fd8ce0 0x556de9fdb0d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93aecb72a2a4c9a0 server_cookie=9d33cbabf2345083 in_seq=0 out_seq=0 2024-03-21T09:20:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.756+0000 7fb4c6a00640 10 client.27557 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4c6a00640 1 -- 172.21.15.37:0/1052534870 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fb4cc07ae50 con 0x556de9fd8ce0 2024-03-21T09:20:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4c6a00640 10 client.27557 handle_command_reply: tid=0 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 client.27557 shutdown 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 msgr2=0x556de9fc87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fc87e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4d402c2d0 tx=0x7fb4d4009260 comp rx=0 tx=0).stop 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 shutdown_connections 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc8de0 0x556de9fc6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fc5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556de9fd8ce0 0x556de9fdb0d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/1052534870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fc87e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4c6a00640 0 client.27557 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 >> 172.21.15.37:0/1052534870 conn(0x556de9fb9f80 msgr2=0x556de9fbbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 shutdown_connections 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 1 -- 172.21.15.37:0/1052534870 wait complete. 2024-03-21T09:20:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.757+0000 7fb4ea57b740 20 client.27557 trim_cache size 0 max 16384 2024-03-21T09:20:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.770+0000 7fb4ea57b740 1 Processor -- start 2024-03-21T09:20:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 -- start start 2024-03-21T09:20:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fbb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 0x556de9fc82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc8de0 0x556de9fc87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01b3100 con 0x556de9fc8de0 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01b0c00 con 0x556de9fbf8b0 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01aecc0 con 0x556de9fc0260 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 0x556de9fc82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 0x556de9fc82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49620/0 (socket says 172.21.15.37:49620) 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2793965232 learned_addr learned my addr 172.21.15.37:0/2793965232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4e5800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc8de0 0x556de9fc87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fbb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2793965232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 msgr2=0x556de9fbb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fbb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2793965232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc8de0 msgr2=0x556de9fc87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc8de0 0x556de9fc87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.771+0000 7fb4dfe00640 1 -- 172.21.15.37:0/2793965232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556de9fbffd0 con 0x556de9fc0260 2024-03-21T09:20:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 0x556de9fc82a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c6a0 tx=0x7fb4cc0bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb954f8d81203b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4d1600640 1 -- 172.21.15.37:0/2793965232 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0b5070 con 0x556de9fc0260 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4d1600640 1 -- 172.21.15.37:0/2793965232 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556de9f30800 con 0x556de9fc0260 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4d1600640 1 -- 172.21.15.37:0/2793965232 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc09b480 con 0x556de9fc0260 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 -- 172.21.15.37:0/2793965232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 msgr2=0x556de9fc82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 0x556de9fc82a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c6a0 tx=0x7fb4cc0bc040 comp rx=0 tx=0).stop 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 -- 172.21.15.37:0/2793965232 shutdown_connections 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc8de0 0x556de9fc87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fc0260 0x556de9fc82a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/2793965232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fbf8b0 0x556de9fbb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.772+0000 7fb4ea57b740 1 -- 172.21.15.37:0/2793965232 >> 172.21.15.37:0/2793965232 conn(0x556de9fb9f80 msgr2=0x556de9fb9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.773+0000 7fb4ea57b740 1 -- 172.21.15.37:0/2793965232 shutdown_connections 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.773+0000 7fb4ea57b740 1 -- 172.21.15.37:0/2793965232 wait complete. 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.773+0000 7fb4ea57b740 1 -- start start 2024-03-21T09:20:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.773+0000 7fb4ea57b740 1 Processor -- start 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.773+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fcf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.773+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fcfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4ea57b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556dea034560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01af2c0 con 0x556de9fc0260 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01b05a0 con 0x556de9fc8de0 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4ea57b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e01af8e0 con 0x556de9fbf8b0 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4ea57b740 10 client.0 _ll_register_callbacks cb 0x7fb4d4043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fcf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fcf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49622/0 (socket says 172.21.15.37:49622) 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 -- 172.21.15.37:0/129417672 learned_addr learned my addr 172.21.15.37:0/129417672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e5800640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556dea034560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fcfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 -- 172.21.15.37:0/129417672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 msgr2=0x556dea034560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556dea034560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 -- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 msgr2=0x556de9fcfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fcfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 -- 172.21.15.37:0/129417672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dea034aa0 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4e4e00640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fcf950 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40096c0 tx=0x7fb4d40045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835ea75268fda5ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.774+0000 7fb4c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4c6a00640 1 -- 172.21.15.37:0/129417672 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4064060 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4c6a00640 1 -- 172.21.15.37:0/129417672 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fb4d402cb20 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556de9f30800 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4c6a00640 1 -- 172.21.15.37:0/129417672 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d406b050 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 10 client.27592 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556dea035c50 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4c6a00640 1 -- 172.21.15.37:0/129417672 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556de9f30800 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4c6a00640 1 -- 172.21.15.37:0/129417672 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb4d404c480 con 0x556de9fbf8b0 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 10 client.27592 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:51.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 10 client.27592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 20 client.27592 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556dea036320 0x556dea038740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 4 client.27592 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.775+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556de9f2ab50 con 0x556dea036320 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.776+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556dea036320 0x556dea038740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.776+0000 7fb4dfe00640 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556dea036320 0x556dea038740 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b409fb27267e54c server_cookie=2efe7edbd07bf319 in_seq=0 out_seq=0 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.776+0000 7fb4c6a00640 10 client.27592 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.776+0000 7fb4c6a00640 1 -- 172.21.15.37:0/129417672 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556de9f2ab50 con 0x556dea036320 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.776+0000 7fb4c6a00640 10 client.27592 handle_command_reply: tid=0 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.776+0000 7fb4ea57b740 1 client.27592 shutdown 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 msgr2=0x556de9fcf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fcf950 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40096c0 tx=0x7fb4d40045e0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 shutdown_connections 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x556dea036320 0x556dea038740 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556de9fc8de0 0x556dea034560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556de9fc0260 0x556de9fcfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 --2- 172.21.15.37:0/129417672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556de9fbf8b0 0x556de9fcf950 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4c6a00640 0 client.27592 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 >> 172.21.15.37:0/129417672 conn(0x556de9fb9f80 msgr2=0x556de9fb9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 shutdown_connections 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 1 -- 172.21.15.37:0/129417672 wait complete. 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.777+0000 7fb4ea57b740 20 client.27592 trim_cache size 0 max 16384 2024-03-21T09:20:51.778 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:20:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb4c80a3090 msgr2=0x7fb4c80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb4c80a3090 0x7fb4c80a5550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x556de9fc0660 tx=0x7fb4d4005a20 comp rx=0 tx=0).stop 2024-03-21T09:20:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102f50 msgr2=0x7fb4e01007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102f50 0x7fb4e01007c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e9f0 tx=0x7fb4cc00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 shutdown_connections 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e010ba50 0x7fb4e0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb4c80a3090 0x7fb4c80a5550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4e0102f50 0x7fb4e01007c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 --2- 172.21.15.37:0/2009400849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4e0102630 0x7fb4e0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 >> 172.21.15.37:0/2009400849 conn(0x7fb4e0077130 msgr2=0x7fb4e00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 shutdown_connections 2024-03-21T09:20:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:51.778+0000 7fb4e6200640 1 -- 172.21.15.37:0/2009400849 wait complete. 2024-03-21T09:20:52.793 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:20:52.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:20:52.793 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:20:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 -- 172.21.15.37:0/3940546727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef41014c0 msgr2=0x7f5ef4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/3940546727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef41014c0 0x7f5ef4101940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009930 tx=0x7f5ee002bc20 comp rx=0 tx=0).stop 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 -- 172.21.15.37:0/3940546727 shutdown_connections 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/3940546727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef4101e80 0x7f5ef410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/3940546727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef41014c0 0x7f5ef4101940 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/3940546727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef4107580 0x7f5ef4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 -- 172.21.15.37:0/3940546727 >> 172.21.15.37:0/3940546727 conn(0x7f5ef40fcfb0 msgr2=0x7f5ef40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 -- 172.21.15.37:0/3940546727 shutdown_connections 2024-03-21T09:20:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.029+0000 7f5ef3e00640 1 -- 172.21.15.37:0/3940546727 wait complete. 2024-03-21T09:20:53.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.030+0000 7f5ef3e00640 1 Processor -- start 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.030+0000 7f5ef3e00640 1 -- start start 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 0x7f5ef4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef4101e80 0x7f5ef419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef4107580 0x7f5ef419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4111890 con 0x7f5ef4101e80 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4111710 con 0x7f5ef4107580 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4111a10 con 0x7f5ef41014c0 2024-03-21T09:20:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 0x7f5ef4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 0x7f5ef4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49646/0 (socket says 172.21.15.37:49646) 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef2a00640 1 -- 172.21.15.37:0/2651231917 learned_addr learned my addr 172.21.15.37:0/2651231917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef3400640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef4107580 0x7f5ef419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.031+0000 7f5ef2000640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef4101e80 0x7f5ef419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef2a00640 1 -- 172.21.15.37:0/2651231917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef4107580 msgr2=0x7f5ef419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef4107580 0x7f5ef419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef2a00640 1 -- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef4101e80 msgr2=0x7f5ef419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef4101e80 0x7f5ef419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef2a00640 1 -- 172.21.15.37:0/2651231917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef419ef40 con 0x7f5ef41014c0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef3400640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef4107580 0x7f5ef419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 0x7f5ef4199f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d950 tx=0x7f5ee400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5eeb400640 1 -- 172.21.15.37:0/2651231917 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4017070 con 0x7f5ef41014c0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5eeb400640 1 -- 172.21.15.37:0/2651231917 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ee4009c30 con 0x7f5ef41014c0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5eeb400640 1 -- 172.21.15.37:0/2651231917 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400f6b0 con 0x7f5ef41014c0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.032+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef419f230 con 0x7f5ef41014c0 2024-03-21T09:20:53.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.033+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef41a6ac0 con 0x7f5ef41014c0 2024-03-21T09:20:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.037+0000 7f5efc5b2740 1 Processor -- start 2024-03-21T09:20:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.037+0000 7f5efc5b2740 1 -- start start 2024-03-21T09:20:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5eeb400640 1 -- 172.21.15.37:0/2651231917 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5ee40109d0 con 0x7f5ef41014c0 2024-03-21T09:20:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c0741670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 0x5557c074a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557c06a5eb0 con 0x5557c0741290 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557c064ea60 con 0x5557c0741bb0 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557c06a1f50 con 0x5557c074a790 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 0x5557c074a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5ef3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c0741670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5ef2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 0x5557c074a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54982/0 (socket says 172.21.15.37:54982) 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5ef2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49648/0 (socket says 172.21.15.37:49648) 2024-03-21T09:20:53.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.038+0000 7f5ef2a00640 1 -- 172.21.15.37:0/998329796 learned_addr learned my addr 172.21.15.37:0/998329796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5ef2a00640 1 -- 172.21.15.37:0/998329796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 msgr2=0x5557c074cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5ef2a00640 1 -- 172.21.15.37:0/998329796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 msgr2=0x5557c0741670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c0741670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5ef2a00640 1 -- 172.21.15.37:0/998329796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557c04827e0 con 0x5557c0741bb0 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 0x5557c074a250 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40cc9c0 tx=0x7f5ee40d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4b6ba481e2766f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5edfe00640 1 -- 172.21.15.37:0/998329796 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40de070 con 0x5557c0741bb0 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5edfe00640 1 -- 172.21.15.37:0/998329796 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ee40cfd80 con 0x5557c0741bb0 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.039+0000 7f5edfe00640 1 -- 172.21.15.37:0/998329796 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40da8a0 con 0x5557c0741bb0 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 -- 172.21.15.37:0/998329796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 msgr2=0x5557c074a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 0x5557c074a250 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40cc9c0 tx=0x7f5ee40d7670 comp rx=0 tx=0).stop 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5eeb400640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5ed80a3110 0x7f5ed80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5eeb400640 1 -- 172.21.15.37:0/2651231917 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5ee40cf790 con 0x7f5ef41014c0 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 -- 172.21.15.37:0/998329796 shutdown_connections 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741bb0 0x5557c074a250 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/998329796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c0741670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 -- 172.21.15.37:0/998329796 >> 172.21.15.37:0/998329796 conn(0x5557c073b9c0 msgr2=0x5557c073bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5ef3400640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5ed80a3110 0x7f5ed80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.040+0000 7f5efc5b2740 1 -- 172.21.15.37:0/998329796 shutdown_connections 2024-03-21T09:20:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.041+0000 7f5efc5b2740 1 -- 172.21.15.37:0/998329796 wait complete. 2024-03-21T09:20:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.041+0000 7f5efc5b2740 1 -- start start 2024-03-21T09:20:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.041+0000 7f5ef3400640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5ed80a3110 0x7f5ed80a55d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009fd0 tx=0x7f5eec009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.041+0000 7f5efc5b2740 1 Processor -- start 2024-03-21T09:20:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c074a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0747500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c0747b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557c074d580 con 0x5557c0741bb0 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557c0661eb0 con 0x5557c0741290 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557c074d700 con 0x5557c074a790 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5efc5b2740 10 client.0 _ll_register_callbacks cb 0x5557c0656150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c074a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5ef2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0747500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c074a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:54988/0 (socket says 172.21.15.37:54988) 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5ef2a00640 1 -- 172.21.15.37:0/3403777237 learned_addr learned my addr 172.21.15.37:0/3403777237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.042+0000 7f5ef3400640 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c0747b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5ef2a00640 1 -- 172.21.15.37:0/3403777237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 msgr2=0x5557c0747b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c0747b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5ef2a00640 1 -- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 msgr2=0x5557c0747500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0747500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5ef2a00640 1 -- 172.21.15.37:0/3403777237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557c04e2f70 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c074a170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4097c30 tx=0x7f5ee40d5240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a704578d80c42d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5edcc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5edcc00640 1 -- 172.21.15.37:0/3403777237 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4020070 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5edcc00640 1 -- 172.21.15.37:0/3403777237 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5ee40cfb80 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee40cfd80 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5edcc00640 1 -- 172.21.15.37:0/3403777237 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee40de070 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5efc5b2740 10 client.27575 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.043+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5557c037a170 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5edcc00640 1 -- 172.21.15.37:0/3403777237 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ee40cfd80 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5edcc00640 1 -- 172.21.15.37:0/3403777237 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5ee4097e40 con 0x5557c0741290 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5efc5b2740 10 client.27575 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5efc5b2740 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5efc5b2740 20 client.27575 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c075a6f0 0x5557c075cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5efc5b2740 4 client.27575 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557c06b33f0 con 0x5557c075a6f0 2024-03-21T09:20:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.044+0000 7f5ef2000640 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c075a6f0 0x5557c075cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.045+0000 7f5ef2000640 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c075a6f0 0x5557c075cae0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abdb1ccecd16bca9 server_cookie=67353c9e1369d03 in_seq=0 out_seq=0 2024-03-21T09:20:53.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.045+0000 7f5edcc00640 10 client.27575 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5edcc00640 1 -- 172.21.15.37:0/3403777237 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5557c06b33f0 con 0x5557c075a6f0 2024-03-21T09:20:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5edcc00640 10 client.27575 handle_command_reply: tid=0 2024-03-21T09:20:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 client.27575 shutdown 2024-03-21T09:20:53.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 msgr2=0x5557c074a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c074a170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4097c30 tx=0x7f5ee40d5240 comp rx=0 tx=0).stop 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 shutdown_connections 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c0747b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0747500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c075a6f0 0x5557c075cae0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3403777237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c074a170 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5edcc00640 0 client.27575 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 >> 172.21.15.37:0/3403777237 conn(0x5557c073b9c0 msgr2=0x5557c073d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 shutdown_connections 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3403777237 wait complete. 2024-03-21T09:20:53.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.046+0000 7f5efc5b2740 20 client.27575 trim_cache size 0 max 16384 2024-03-21T09:20:53.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 Processor -- start 2024-03-21T09:20:53.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 -- start start 2024-03-21T09:20:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c073d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0749c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41b30e0 con 0x5557c0741bb0 2024-03-21T09:20:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41b0be0 con 0x5557c0741290 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.079+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41aeca0 con 0x5557c074a790 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c073d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c073d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:55002/0 (socket says 172.21.15.37:55002) 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 -- 172.21.15.37:0/1618815734 learned_addr learned my addr 172.21.15.37:0/1618815734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef3400640 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49666/0 (socket says 172.21.15.37:49666) 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2000640 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0749c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 -- 172.21.15.37:0/1618815734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 msgr2=0x5557c074a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 -- 172.21.15.37:0/1618815734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 msgr2=0x5557c0749c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0749c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.080+0000 7f5ef2a00640 1 -- 172.21.15.37:0/1618815734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557c074cef0 con 0x5557c0741290 2024-03-21T09:20:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.081+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c073d450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5ee409b4d0 tx=0x7f5ee4099ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=887146066795e8e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.081+0000 7f5edf400640 1 -- 172.21.15.37:0/1618815734 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4020030 con 0x5557c0741290 2024-03-21T09:20:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.081+0000 7f5edf400640 1 -- 172.21.15.37:0/1618815734 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ee409ac20 con 0x5557c0741290 2024-03-21T09:20:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.081+0000 7f5edf400640 1 -- 172.21.15.37:0/1618815734 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a4130 con 0x5557c0741290 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 -- 172.21.15.37:0/1618815734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 msgr2=0x5557c073d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c073d450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5ee409b4d0 tx=0x7f5ee4099ce0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 -- 172.21.15.37:0/1618815734 shutdown_connections 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c074a790 0x5557c074a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741bb0 0x5557c0749c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/1618815734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c0741290 0x5557c073d450 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 -- 172.21.15.37:0/1618815734 >> 172.21.15.37:0/1618815734 conn(0x5557c073b9c0 msgr2=0x5557c073cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 -- 172.21.15.37:0/1618815734 shutdown_connections 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 -- 172.21.15.37:0/1618815734 wait complete. 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 -- start start 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.082+0000 7f5efc5b2740 1 Processor -- start 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c075f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 0x5557c075f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c074a790 0x5557c075d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41af2a0 con 0x5557c0741290 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41b0580 con 0x5557c074a790 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41af8c0 con 0x5557c0741bb0 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5efc5b2740 10 client.0 _ll_register_callbacks cb 0x7f5eec009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 0x5557c075f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 0x5557c075f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49676/0 (socket says 172.21.15.37:49676) 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef2000640 1 -- 172.21.15.37:0/3762925136 learned_addr learned my addr 172.21.15.37:0/3762925136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef3400640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c074a790 0x5557c075d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c075f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef2000640 1 -- 172.21.15.37:0/3762925136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c074a790 msgr2=0x5557c075d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.083+0000 7f5ef2000640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c074a790 0x5557c075d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5ef2000640 1 -- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 msgr2=0x5557c075f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5ef2000640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c075f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5ef2000640 1 -- 172.21.15.37:0/3762925136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557c075df50 con 0x5557c0741bb0 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c075f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5ef2000640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 0x5557c075f810 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0000c00 tx=0x7f5ee0004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc30695b2bec1bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5edcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:53.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5edcc00640 1 -- 172.21.15.37:0/3762925136 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee002c870 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557c06ad740 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5edcc00640 1 -- 172.21.15.37:0/3762925136 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5ee002ca10 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5edcc00640 1 -- 172.21.15.37:0/3762925136 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee002cc10 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5efc5b2740 10 client.27610 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.084+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5557c07b67c0 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5edcc00640 1 -- 172.21.15.37:0/3762925136 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5557c06ad740 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5edcc00640 1 -- 172.21.15.37:0/3762925136 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5ee0005390 con 0x5557c0741bb0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5efc5b2740 10 client.27610 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5efc5b2740 10 client.27610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5efc5b2740 20 client.27610 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c07b6ef0 0x5557c07b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5efc5b2740 4 client.27610 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5557c062ba40 con 0x5557c07b6ef0 2024-03-21T09:20:53.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.085+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c07b6ef0 0x5557c07b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.086+0000 7f5ef2a00640 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c07b6ef0 0x5557c07b9310 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb51d52e85c13846 server_cookie=9e537ea47df336ff in_seq=0 out_seq=0 2024-03-21T09:20:53.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.086+0000 7f5edcc00640 10 client.27610 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.087+0000 7f5edcc00640 1 -- 172.21.15.37:0/3762925136 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5557c062ba40 con 0x5557c07b6ef0 2024-03-21T09:20:53.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.087+0000 7f5edcc00640 10 client.27610 handle_command_reply: tid=0 2024-03-21T09:20:53.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.087+0000 7f5efc5b2740 1 client.27610 shutdown 2024-03-21T09:20:53.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 msgr2=0x5557c075f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 0x5557c075f810 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0000c00 tx=0x7f5ee0004160 comp rx=0 tx=0).stop 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 shutdown_connections 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5557c07b6ef0 0x5557c07b9310 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557c074a790 0x5557c075d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557c0741bb0 0x5557c075f810 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 --2- 172.21.15.37:0/3762925136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557c0741290 0x5557c075f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 >> 172.21.15.37:0/3762925136 conn(0x5557c073b9c0 msgr2=0x5557c073d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5edcc00640 0 client.27610 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 shutdown_connections 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 1 -- 172.21.15.37:0/3762925136 wait complete. 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.088+0000 7f5efc5b2740 20 client.27610 trim_cache size 0 max 16384 2024-03-21T09:20:53.087 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:20:53.088 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:20:53.088 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "dc12d9bb-18b9-40bf-a639-a00d2ef707c0", 2024-03-21T09:20:53.088 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:20:53.088 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:20:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.089+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5ed80a3110 msgr2=0x7f5ed80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.089+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5ed80a3110 0x7f5ed80a55d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009fd0 tx=0x7f5eec009500 comp rx=0 tx=0).stop 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.089+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 msgr2=0x7f5ef4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.089+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 0x7f5ef4199f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d950 tx=0x7f5ee400de10 comp rx=0 tx=0).stop 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.089+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 shutdown_connections 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5ed80a3110 0x7f5ed80a55d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef4107580 0x7f5ef419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef4101e80 0x7f5ef419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 --2- 172.21.15.37:0/2651231917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ef41014c0 0x7f5ef4199f50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 >> 172.21.15.37:0/2651231917 conn(0x7f5ef40fcfb0 msgr2=0x7f5ef40fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 shutdown_connections 2024-03-21T09:20:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.090+0000 7f5ef3e00640 1 -- 172.21.15.37:0/2651231917 wait complete. 2024-03-21T09:20:53.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:20:53.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 -- 172.21.15.37:0/2394274019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 msgr2=0x7fbfdc110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2394274019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc110730 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009cd0 tx=0x7fbfd402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 -- 172.21.15.37:0/2394274019 shutdown_connections 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2394274019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc110730 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2394274019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2394274019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfdc109870 0x7fbfdc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 -- 172.21.15.37:0/2394274019 >> 172.21.15.37:0/2394274019 conn(0x7fbfdc0ff310 msgr2=0x7fbfdc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 -- 172.21.15.37:0/2394274019 shutdown_connections 2024-03-21T09:20:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.322+0000 7fbfe3200640 1 -- 172.21.15.37:0/2394274019 wait complete. 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.323+0000 7fbfe3200640 1 Processor -- start 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 -- start start 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfdc109870 0x7fbfdc19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc113c80 con 0x7fbfdc109870 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc113b00 con 0x7fbfdc103860 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc113e00 con 0x7fbfdc104220 2024-03-21T09:20:53.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:55048/0 (socket says 172.21.15.37:55048) 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfe1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfdc109870 0x7fbfdc19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.324+0000 7fbfdbe00640 1 -- 172.21.15.37:0/2133823258 learned_addr learned my addr 172.21.15.37:0/2133823258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfdbe00640 1 -- 172.21.15.37:0/2133823258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 msgr2=0x7fbfdc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfdbe00640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfdbe00640 1 -- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfdc109870 msgr2=0x7fbfdc19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfdbe00640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfdc109870 0x7fbfdc19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfdbe00640 1 -- 172.21.15.37:0/2133823258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc19f0f0 con 0x7fbfdc103860 2024-03-21T09:20:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfdbe00640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc19a100 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00e9f0 tx=0x7fbfcc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfd9600640 1 -- 172.21.15.37:0/2133823258 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00cd80 con 0x7fbfdc103860 2024-03-21T09:20:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfd9600640 1 -- 172.21.15.37:0/2133823258 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc010910 con 0x7fbfdc103860 2024-03-21T09:20:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfd9600640 1 -- 172.21.15.37:0/2133823258 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc018990 con 0x7fbfdc103860 2024-03-21T09:20:53.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.325+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc19f3e0 con 0x7fbfdc103860 2024-03-21T09:20:53.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.326+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc1a6c70 con 0x7fbfdc103860 2024-03-21T09:20:53.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.330+0000 7fbfe5fd1740 1 Processor -- start 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.330+0000 7fbfe5fd1740 1 -- start start 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.330+0000 7fbfd9600640 1 -- 172.21.15.37:0/2133823258 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fbfcc010ac0 con 0x7fbfdc103860 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.330+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee2a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bded8f250 con 0x559bdee2af20 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bded37a30 con 0x559bdee34290 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bded8b2f0 con 0x559bdee2bc00 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee2a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee2a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49692/0 (socket says 172.21.15.37:49692) 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 -- 172.21.15.37:0/4208404159 learned_addr learned my addr 172.21.15.37:0/4208404159 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfdbe00640 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 -- 172.21.15.37:0/4208404159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 msgr2=0x559bdee36680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee36680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 -- 172.21.15.37:0/4208404159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 msgr2=0x559bdee2b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee2b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.331+0000 7fbfe1200640 1 -- 172.21.15.37:0/4208404159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bded22690 con 0x559bdee2bc00 2024-03-21T09:20:53.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.332+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee2a950 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004af0 tx=0x7fbfd4005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f4f228f0b9b57f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.332+0000 7fbfd1600640 1 -- 172.21.15.37:0/4208404159 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4043070 con 0x559bdee2bc00 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.332+0000 7fbfd1600640 1 -- 172.21.15.37:0/4208404159 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd402cca0 con 0x559bdee2bc00 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.332+0000 7fbfd1600640 1 -- 172.21.15.37:0/4208404159 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4039040 con 0x559bdee2bc00 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfd9600640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fbfc00a3110 0x7fbfc00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4208404159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 msgr2=0x559bdee2a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee2a950 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004af0 tx=0x7fbfd4005f80 comp rx=0 tx=0).stop 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfd9600640 1 -- 172.21.15.37:0/2133823258 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fbfcc0cfea0 con 0x7fbfdc103860 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfdbe00640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fbfc00a3110 0x7fbfc00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4208404159 shutdown_connections 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee2b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4208404159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee2a950 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4208404159 >> 172.21.15.37:0/4208404159 conn(0x559bdedf2200 msgr2=0x559bdee25970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4208404159 shutdown_connections 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.333+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4208404159 wait complete. 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.334+0000 7fbfdbe00640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fbfc00a3110 0x7fbfc00a55d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x559bdee2c150 tx=0x7fbfcc000bc0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.334+0000 7fbfe5fd1740 1 -- start start 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.334+0000 7fbfe5fd1740 1 Processor -- start 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2af20 0x559bdee338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee34290 0x559bdee311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdee370a0 con 0x559bdee34290 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bded4ae80 con 0x559bdee2af20 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdee37220 con 0x559bdee2bc00 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe5fd1740 10 client.0 _ll_register_callbacks cb 0x559bded5d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee30c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49702/0 (socket says 172.21.15.37:49702) 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 -- 172.21.15.37:0/2526416374 learned_addr learned my addr 172.21.15.37:0/2526416374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe1200640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2af20 0x559bdee338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfe1200640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee34290 0x559bdee311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 -- 172.21.15.37:0/2526416374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2af20 msgr2=0x559bdee338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2af20 0x559bdee338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 -- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee34290 msgr2=0x559bdee311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee34290 0x559bdee311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.335+0000 7fbfd3e00640 1 -- 172.21.15.37:0/2526416374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bdeb6d7d0 con 0x559bdee2bc00 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.336+0000 7fbfe1200640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee34290 0x559bdee311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.336+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee30c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4012d20 tx=0x7fbfc400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a88becc252c36ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.336+0000 7fbfca000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.336+0000 7fbfca000640 1 -- 172.21.15.37:0/2526416374 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4012110 con 0x559bdee2bc00 2024-03-21T09:20:53.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.337+0000 7fbfca000640 1 -- 172.21.15.37:0/2526416374 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fbfc4004050 con 0x559bdee2bc00 2024-03-21T09:20:53.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.337+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfd402cca0 con 0x559bdee2bc00 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.338+0000 7fbfca000640 1 -- 172.21.15.37:0/2526416374 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc4025050 con 0x559bdee2bc00 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.338+0000 7fbfe5fd1740 10 client.27622 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.338+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559bdeaa33f0 con 0x559bdee2bc00 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfca000640 1 -- 172.21.15.37:0/2526416374 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd402cca0 con 0x559bdee2bc00 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfca000640 1 -- 172.21.15.37:0/2526416374 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbfc4010430 con 0x559bdee2bc00 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe5fd1740 10 client.27622 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe5fd1740 10 client.27622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe5fd1740 20 client.27622 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdee44100 0x559bdee46520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe5fd1740 4 client.27622 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bded9c790 con 0x559bdee44100 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.339+0000 7fbfe1200640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdee44100 0x559bdee46520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.340+0000 7fbfe1200640 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdee44100 0x559bdee46520 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e616d7983312f36 server_cookie=cf990c8bd3aca9e7 in_seq=0 out_seq=0 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.340+0000 7fbfca000640 10 client.27622 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfca000640 1 -- 172.21.15.37:0/2526416374 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559bded9c790 con 0x559bdee44100 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfca000640 10 client.27622 handle_command_reply: tid=0 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 client.27622 shutdown 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 msgr2=0x559bdee30c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee30c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4012d20 tx=0x7fbfc400f790 comp rx=0 tx=0).stop 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 shutdown_connections 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee34290 0x559bdee311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee30c70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdee44100 0x559bdee46520 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/2526416374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2af20 0x559bdee338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfca000640 0 client.27622 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 >> 172.21.15.37:0/2526416374 conn(0x559bdedf2200 msgr2=0x559bdee276c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 shutdown_connections 2024-03-21T09:20:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/2526416374 wait complete. 2024-03-21T09:20:53.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.341+0000 7fbfe5fd1740 20 client.27622 trim_cache size 0 max 16384 2024-03-21T09:20:53.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 Processor -- start 2024-03-21T09:20:53.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 -- start start 2024-03-21T09:20:53.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee35680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1b3270 con 0x559bdee2af20 2024-03-21T09:20:53.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1b0d70 con 0x559bdee34290 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.362+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1aee30 con 0x559bdee2bc00 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49716/0 (socket says 172.21.15.37:49716) 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfd3e00640 1 -- 172.21.15.37:0/4264559605 learned_addr learned my addr 172.21.15.37:0/4264559605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee35680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 -- 172.21.15.37:0/4264559605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 msgr2=0x559bdee338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 -- 172.21.15.37:0/4264559605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 msgr2=0x559bdee35680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee35680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfe1200640 1 -- 172.21.15.37:0/4264559605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bdea630f0 con 0x559bdee2af20 2024-03-21T09:20:53.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.363+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:20:53.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.364+0000 7fbfe1200640 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee333a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4037040 tx=0x7fbfd4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf74eb824c534083 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.364+0000 7fbfd0c00640 1 -- 172.21.15.37:0/4264559605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4055070 con 0x559bdee2af20 2024-03-21T09:20:53.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.364+0000 7fbfd0c00640 1 -- 172.21.15.37:0/4264559605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd402cca0 con 0x559bdee2af20 2024-03-21T09:20:53.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.364+0000 7fbfd0c00640 1 -- 172.21.15.37:0/4264559605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4039040 con 0x559bdee2af20 2024-03-21T09:20:53.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4264559605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 msgr2=0x559bdee333a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee333a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4037040 tx=0x7fbfd4005d80 comp rx=0 tx=0).stop 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4264559605 shutdown_connections 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee34290 0x559bdee35680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee2bc00 0x559bdee338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/4264559605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee333a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4264559605 >> 172.21.15.37:0/4264559605 conn(0x559bdedf2200 msgr2=0x559bdee2f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4264559605 shutdown_connections 2024-03-21T09:20:53.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/4264559605 wait complete. 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.365+0000 7fbfe5fd1740 1 -- start start 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 Processor -- start 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 0x559bdee381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee34290 0x559bdee9e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1af430 con 0x559bdee2af20 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1b0710 con 0x559bdee2bc00 2024-03-21T09:20:53.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1afa50 con 0x559bdee34290 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe5fd1740 10 client.0 _ll_register_callbacks cb 0x7fbfc4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 0x559bdee381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfd3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 0x559bdee381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:55094/0 (socket says 172.21.15.37:55094) 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfd3e00640 1 -- 172.21.15.37:0/95112242 learned_addr learned my addr 172.21.15.37:0/95112242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.366+0000 7fbfe1200640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfe1200640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee34290 0x559bdee9e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfd3e00640 1 -- 172.21.15.37:0/95112242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee34290 msgr2=0x559bdee9e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee34290 0x559bdee9e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfd3e00640 1 -- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 msgr2=0x559bdee37ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee37ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfd3e00640 1 -- 172.21.15.37:0/95112242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bdee9ef30 con 0x559bdee2bc00 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfe1200640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfd3e00640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 0x559bdee381e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4010880 tx=0x7fbfc4012340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53b994862ad2f8c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfca000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfca000640 1 -- 172.21.15.37:0/95112242 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc402d070 con 0x559bdee2bc00 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfd402cca0 con 0x559bdee2bc00 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.367+0000 7fbfca000640 1 -- 172.21.15.37:0/95112242 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fbfc4022040 con 0x559bdee2bc00 2024-03-21T09:20:53.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfca000640 1 -- 172.21.15.37:0/95112242 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc4039050 con 0x559bdee2bc00 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 10 client.27599 fetch_fsmap learned FSMap version 30 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559bdeea00b0 con 0x559bdee2bc00 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfca000640 1 -- 172.21.15.37:0/95112242 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd402cca0 con 0x559bdee2bc00 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfca000640 1 -- 172.21.15.37:0/95112242 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbfc40035f0 con 0x559bdee2bc00 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 10 client.27599 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 10 client.27599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 20 client.27599 populate_metadata read hostname 'smithi037' 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdeea07e0 0x559bdeea2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 4 client.27599 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559bded96ae0 con 0x559bdeea07e0 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.368+0000 7fbfe1200640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdeea07e0 0x559bdeea2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfe1200640 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdeea07e0 0x559bdeea2c00 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f891ce6e7788c6 server_cookie=5b0c17c181aa955 in_seq=0 out_seq=0 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfca000640 10 client.27599 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfca000640 1 -- 172.21.15.37:0/95112242 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559bded96ae0 con 0x559bdeea07e0 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfca000640 10 client.27599 handle_command_reply: tid=0 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfe5fd1740 1 client.27599 shutdown 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 msgr2=0x559bdee381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.369+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 0x559bdee381e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4010880 tx=0x7fbfc4012340 comp rx=0 tx=0).stop 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 shutdown_connections 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bdee34290 0x559bdee9e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x559bdeea07e0 0x559bdeea2c00 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfca000640 0 client.27599 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bdee2bc00 0x559bdee381e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 --2- 172.21.15.37:0/95112242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bdee2af20 0x559bdee37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 >> 172.21.15.37:0/95112242 conn(0x559bdedf2200 msgr2=0x559bdee24200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 shutdown_connections 2024-03-21T09:20:53.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 1 -- 172.21.15.37:0/95112242 wait complete. 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.370+0000 7fbfe5fd1740 20 client.27599 trim_cache size 0 max 16384 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: "dc12d9bb-18b9-40bf-a639-a00d2ef707c0": { 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "dc12d9bb-18b9-40bf-a639-a00d2ef707c0", 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:20:53.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:20:53.380 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:20:53.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fbfc00a3110 msgr2=0x7fbfc00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fbfc00a3110 0x7fbfc00a55d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x559bdee2c150 tx=0x7fbfcc000bc0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 msgr2=0x7fbfdc19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:20:53.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc19a100 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00e9f0 tx=0x7fbfcc00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 shutdown_connections 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfdc109870 0x7fbfdc19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc104220 0x7fbfdc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fbfc00a3110 0x7fbfc00a55d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 --2- 172.21.15.37:0/2133823258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfdc103860 0x7fbfdc19a100 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 >> 172.21.15.37:0/2133823258 conn(0x7fbfdc0ff310 msgr2=0x7fbfdc0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 shutdown_connections 2024-03-21T09:20:53.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:20:53.371+0000 7fbfe3200640 1 -- 172.21.15.37:0/2133823258 wait complete. 2024-03-21T09:20:53.386 INFO:tasks.cephfs.filesystem:scrub status for tag:dc12d9bb-18b9-40bf-a639-a00d2ef707c0 - {'path': '/', 'tag': 'dc12d9bb-18b9-40bf-a639-a00d2ef707c0', 'options': 'recursive,force'} 2024-03-21T09:20:54.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:54 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:20:54.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:54 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:54.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:54 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:20:54.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:54 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:54.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:54 smithi071 ceph-mon[36597]: pgmap v268: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:54.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:54 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:20:54.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:54 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:54.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:54 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:20:54.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:54 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:54.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:54 smithi037 ceph-mon[31830]: pgmap v268: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:54.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:54 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:20:54.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:54 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:54.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:54 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:20:54.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:54 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:20:54.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:54 smithi053 ceph-mon[36169]: pgmap v268: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:20:56.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:56 smithi071 ceph-mon[36597]: pgmap v269: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:56.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:56 smithi037 ceph-mon[31830]: pgmap v269: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:56.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:56 smithi053 ceph-mon[36169]: pgmap v269: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:20:57.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:57 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:20:57.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:57 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:20:57.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:57 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:20:58.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:58 smithi071 ceph-mon[36597]: pgmap v270: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:58.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:58 smithi037 ceph-mon[31830]: pgmap v270: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:58.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:58 smithi053 ceph-mon[36169]: pgmap v270: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:20:59.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:20:59 smithi037 ceph-mon[31830]: pgmap v271: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:59.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:20:59 smithi053 ceph-mon[36169]: pgmap v271: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:20:59.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:20:59 smithi071 ceph-mon[36597]: pgmap v271: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:02.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:02 smithi071 ceph-mon[36597]: pgmap v272: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:02.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:02 smithi037 ceph-mon[31830]: pgmap v272: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:02.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:02 smithi053 ceph-mon[36169]: pgmap v272: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:04.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:04 smithi071 ceph-mon[36597]: pgmap v273: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T09:21:04.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:04 smithi037 ceph-mon[31830]: pgmap v273: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T09:21:04.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:04 smithi053 ceph-mon[36169]: pgmap v273: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T09:21:05.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:05 smithi071 ceph-mon[36597]: pgmap v274: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-03-21T09:21:05.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:05 smithi037 ceph-mon[31830]: pgmap v274: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-03-21T09:21:05.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:05 smithi053 ceph-mon[36169]: pgmap v274: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-03-21T09:21:07.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:07 smithi071 ceph-mon[36597]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T09:21:07.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:07 smithi071 ceph-mon[36597]: pgmap v276: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:21:07.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:07 smithi037 ceph-mon[31830]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T09:21:07.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:07 smithi037 ceph-mon[31830]: pgmap v276: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:21:07.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:07 smithi053 ceph-mon[36169]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T09:21:07.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:07 smithi053 ceph-mon[36169]: pgmap v276: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:21:10.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:10 smithi071 ceph-mon[36597]: pgmap v277: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:21:10.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:10 smithi053 ceph-mon[36169]: pgmap v277: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:21:10.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:10 smithi037 ceph-mon[31830]: pgmap v277: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:21:12.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:12 smithi071 ceph-mon[36597]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T09:21:12.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:12 smithi071 ceph-mon[36597]: pgmap v279: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-21T09:21:12.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:12 smithi037 ceph-mon[31830]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T09:21:12.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:12 smithi037 ceph-mon[31830]: pgmap v279: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-21T09:21:12.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:12 smithi053 ceph-mon[36169]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T09:21:12.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:12 smithi053 ceph-mon[36169]: pgmap v279: 129 pgs: 129 active+clean; 17 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-21T09:21:14.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:14 smithi071 ceph-mon[36597]: pgmap v280: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:21:14.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:14 smithi037 ceph-mon[31830]: pgmap v280: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:21:14.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:14 smithi053 ceph-mon[36169]: pgmap v280: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-21T09:21:15.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:15 smithi037 ceph-mon[31830]: pgmap v281: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 234 B/s wr, 7 op/s 2024-03-21T09:21:15.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:15 smithi053 ceph-mon[36169]: pgmap v281: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 234 B/s wr, 7 op/s 2024-03-21T09:21:15.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:15 smithi071 ceph-mon[36597]: pgmap v281: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 234 B/s wr, 7 op/s 2024-03-21T09:21:18.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:18 smithi071 ceph-mon[36597]: pgmap v282: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:21:18.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:18 smithi037 ceph-mon[31830]: pgmap v282: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:21:18.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:18 smithi053 ceph-mon[36169]: pgmap v282: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:21:19.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:19 smithi037 ceph-mon[31830]: pgmap v283: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:21:19.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:19 smithi053 ceph-mon[36169]: pgmap v283: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:21:19.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:19 smithi071 ceph-mon[36597]: pgmap v283: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:21:22.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:22 smithi071 ceph-mon[36597]: pgmap v284: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2024-03-21T09:21:22.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:22 smithi037 ceph-mon[31830]: pgmap v284: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2024-03-21T09:21:22.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:22 smithi053 ceph-mon[36169]: pgmap v284: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2024-03-21T09:21:23.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:21:23.387 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:21:23.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 -- 172.21.15.37:0/1508880149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 msgr2=0x7f9c401057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 --2- 172.21.15.37:0/1508880149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 0x7f9c401057e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009930 tx=0x7f9c2c02bd10 comp rx=0 tx=0).stop 2024-03-21T09:21:23.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 -- 172.21.15.37:0/1508880149 shutdown_connections 2024-03-21T09:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 --2- 172.21.15.37:0/1508880149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 0x7f9c40110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 --2- 172.21.15.37:0/1508880149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 0x7f9c401057e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 --2- 172.21.15.37:0/1508880149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c40104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 -- 172.21.15.37:0/1508880149 >> 172.21.15.37:0/1508880149 conn(0x7f9c400ff350 msgr2=0x7f9c40101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 -- 172.21.15.37:0/1508880149 shutdown_connections 2024-03-21T09:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.615+0000 7f9c47800640 1 -- 172.21.15.37:0/1508880149 wait complete. 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 Processor -- start 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 -- start start 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c4019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 0x7f9c4019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 0x7f9c4019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40113de0 con 0x7f9c40105d20 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40113c60 con 0x7f9c40103c40 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c47800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40113f60 con 0x7f9c40105380 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.617+0000 7f9c44e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c4019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c4019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:51502/0 (socket says 172.21.15.37:51502) 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 -- 172.21.15.37:0/1728198016 learned_addr learned my addr 172.21.15.37:0/1728198016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 0x7f9c4019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c45800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 0x7f9c4019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 -- 172.21.15.37:0/1728198016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 msgr2=0x7f9c4019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 0x7f9c4019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 -- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 msgr2=0x7f9c4019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 0x7f9c4019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 -- 172.21.15.37:0/1728198016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4019f2f0 con 0x7f9c40103c40 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c45800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 0x7f9c4019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.618+0000 7f9c44e00640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c4019a300 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e9f0 tx=0x7f9c3400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.619+0000 7f9c3d600640 1 -- 172.21.15.37:0/1728198016 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400cd80 con 0x7f9c40103c40 2024-03-21T09:21:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.619+0000 7f9c3d600640 1 -- 172.21.15.37:0/1728198016 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c34005500 con 0x7f9c40103c40 2024-03-21T09:21:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.619+0000 7f9c3d600640 1 -- 172.21.15.37:0/1728198016 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34010690 con 0x7f9c40103c40 2024-03-21T09:21:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.619+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4019f5e0 con 0x7f9c40103c40 2024-03-21T09:21:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.619+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c401a6e70 con 0x7f9c40103c40 2024-03-21T09:21:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.622+0000 7f9c3d600640 1 -- 172.21.15.37:0/1728198016 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9c340046c0 con 0x7f9c40103c40 2024-03-21T09:21:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.622+0000 7f9c4a5f6740 1 Processor -- start 2024-03-21T09:21:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.622+0000 7f9c4a5f6740 1 -- start start 2024-03-21T09:21:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea63c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63cf20 0x5566ea63d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea646290 0x5566ea648680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea5a1250 con 0x5566ea63cf20 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea549a30 con 0x5566ea646290 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea59d2f0 con 0x5566ea63dc00 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea63c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea63c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48434/0 (socket says 172.21.15.37:48434) 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c44e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63cf20 0x5566ea63d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 -- 172.21.15.37:0/2817901504 learned_addr learned my addr 172.21.15.37:0/2817901504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea646290 0x5566ea648680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 -- 172.21.15.37:0/2817901504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea646290 msgr2=0x5566ea648680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea646290 0x5566ea648680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 -- 172.21.15.37:0/2817901504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63cf20 msgr2=0x5566ea63d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63cf20 0x5566ea63d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c45800640 1 -- 172.21.15.37:0/2817901504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ea534690 con 0x5566ea63dc00 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.623+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea646290 0x5566ea648680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c3d600640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9c280a3110 0x7f9c280a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c3d600640 1 -- 172.21.15.37:0/1728198016 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9c340d0080 con 0x7f9c40103c40 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c45800640 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea63c950 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009f30 tx=0x7f9c38010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53cbddd00b70eaad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c44e00640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9c280a3110 0x7f9c280a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c32000640 1 -- 172.21.15.37:0/2817901504 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38016070 con 0x5566ea63dc00 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c32000640 1 -- 172.21.15.37:0/2817901504 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c3801bb40 con 0x5566ea63dc00 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.624+0000 7f9c32000640 1 -- 172.21.15.37:0/2817901504 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38012cb0 con 0x5566ea63dc00 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/2817901504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 msgr2=0x5566ea63c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea63c950 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009f30 tx=0x7f9c38010800 comp rx=0 tx=0).stop 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c44e00640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9c280a3110 0x7f9c280a55d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5566ea63e150 tx=0x7f9c340d3b30 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/2817901504 shutdown_connections 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea646290 0x5566ea648680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63cf20 0x5566ea63d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/2817901504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea63c950 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/2817901504 >> 172.21.15.37:0/2817901504 conn(0x5566ea604200 msgr2=0x5566ea637970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/2817901504 shutdown_connections 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.625+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/2817901504 wait complete. 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 -- start start 2024-03-21T09:21:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 Processor -- start 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea645950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63dc00 0x5566ea642ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea646290 0x5566ea643220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea649100 con 0x5566ea63dc00 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea55ce80 con 0x5566ea63cf20 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea649280 con 0x5566ea646290 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.626+0000 7f9c4a5f6740 10 client.0 _ll_register_callbacks cb 0x5566ea56f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea645950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea645950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:51518/0 (socket says 172.21.15.37:51518) 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 -- 172.21.15.37:0/554060108 learned_addr learned my addr 172.21.15.37:0/554060108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea646290 0x5566ea643220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63dc00 0x5566ea642ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 -- 172.21.15.37:0/554060108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea646290 msgr2=0x5566ea643220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea646290 0x5566ea643220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 -- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63dc00 msgr2=0x5566ea642ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63dc00 0x5566ea642ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 -- 172.21.15.37:0/554060108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ea37f7d0 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c45800640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea645950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009c80 tx=0x7f9c38012890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f56c8f617ecacf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.627+0000 7f9c26a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c26a00640 1 -- 172.21.15.37:0/554060108 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38016070 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c26a00640 1 -- 172.21.15.37:0/554060108 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9c38012c30 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c3801bb40 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c26a00640 1 -- 172.21.15.37:0/554060108 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c3801b460 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 10 client.27611 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5566ea2b53f0 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c26a00640 1 -- 172.21.15.37:0/554060108 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c3801bb40 con 0x5566ea63cf20 2024-03-21T09:21:23.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c26a00640 1 -- 172.21.15.37:0/554060108 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9c3800f040 con 0x5566ea63cf20 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 10 client.27611 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 10 client.27611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 20 client.27611 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6560b0 0x5566ea6584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 4 client.27611 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.628+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566ea5ae790 con 0x5566ea6560b0 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.629+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6560b0 0x5566ea6584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.629+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6560b0 0x5566ea6584a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf83fb3205b497cd server_cookie=d91fe40314e8e112 in_seq=0 out_seq=0 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.629+0000 7f9c26a00640 10 client.27611 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c26a00640 1 -- 172.21.15.37:0/554060108 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5566ea5ae790 con 0x5566ea6560b0 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c26a00640 10 client.27611 handle_command_reply: tid=0 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 client.27611 shutdown 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 msgr2=0x5566ea645950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea645950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009c80 tx=0x7f9c38012890 comp rx=0 tx=0).stop 2024-03-21T09:21:23.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 shutdown_connections 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea646290 0x5566ea643220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea63dc00 0x5566ea642ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6560b0 0x5566ea6584a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/554060108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea645950 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 >> 172.21.15.37:0/554060108 conn(0x5566ea604200 msgr2=0x5566ea6396c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.630+0000 7f9c26a00640 0 client.27611 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.631+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 shutdown_connections 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.631+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/554060108 wait complete. 2024-03-21T09:21:23.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.631+0000 7f9c4a5f6740 20 client.27611 trim_cache size 0 max 16384 2024-03-21T09:21:23.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.656+0000 7f9c4a5f6740 1 Processor -- start 2024-03-21T09:21:23.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.656+0000 7f9c4a5f6740 1 -- start start 2024-03-21T09:21:23.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63cf20 0x5566ea645410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 0x5566ea645950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea647680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401b3380 con 0x5566ea646290 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401b0e80 con 0x5566ea63dc00 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401aef40 con 0x5566ea63cf20 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 0x5566ea645950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 0x5566ea645950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:51530/0 (socket says 172.21.15.37:51530) 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 -- 172.21.15.37:0/4052565733 learned_addr learned my addr 172.21.15.37:0/4052565733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c45800640 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63cf20 0x5566ea645410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c45800640 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea647680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 -- 172.21.15.37:0/4052565733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63cf20 msgr2=0x5566ea645410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63cf20 0x5566ea645410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 -- 172.21.15.37:0/4052565733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 msgr2=0x5566ea647680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea647680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c3fe00640 1 -- 172.21.15.37:0/4052565733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ea2750f0 con 0x5566ea63dc00 2024-03-21T09:21:23.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.657+0000 7f9c45800640 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea647680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:21:23.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.658+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 0x5566ea645950 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c005ab0 tx=0x7f9c2c042040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c314c8320902ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.658+0000 7f9c31600640 1 -- 172.21.15.37:0/4052565733 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0041e0 con 0x5566ea63dc00 2024-03-21T09:21:23.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.658+0000 7f9c31600640 1 -- 172.21.15.37:0/4052565733 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c2c04d070 con 0x5566ea63dc00 2024-03-21T09:21:23.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.658+0000 7f9c31600640 1 -- 172.21.15.37:0/4052565733 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c004bf0 con 0x5566ea63dc00 2024-03-21T09:21:23.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/4052565733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 msgr2=0x5566ea645950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 0x5566ea645950 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c005ab0 tx=0x7f9c2c042040 comp rx=0 tx=0).stop 2024-03-21T09:21:23.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/4052565733 shutdown_connections 2024-03-21T09:21:23.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea647680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63dc00 0x5566ea645950 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/4052565733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63cf20 0x5566ea645410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/4052565733 >> 172.21.15.37:0/4052565733 conn(0x5566ea604200 msgr2=0x5566ea636a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/4052565733 shutdown_connections 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/4052565733 wait complete. 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.659+0000 7f9c4a5f6740 1 -- start start 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 Processor -- start 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea64bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea64bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea64c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401af540 con 0x5566ea646290 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401b0820 con 0x5566ea63cf20 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401afb60 con 0x5566ea63dc00 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.660+0000 7f9c4a5f6740 10 client.0 _ll_register_callbacks cb 0x7f9c340d3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c45800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea64bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea64bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c45800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea64bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:51538/0 (socket says 172.21.15.37:51538) 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea64bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48454/0 (socket says 172.21.15.37:48454) 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c45800640 1 -- 172.21.15.37:0/237895544 learned_addr learned my addr 172.21.15.37:0/237895544 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c45800640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea64c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 -- 172.21.15.37:0/237895544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 msgr2=0x5566ea64bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea64bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 -- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 msgr2=0x5566ea64c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea64c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 -- 172.21.15.37:0/237895544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ea6b0800 con 0x5566ea63dc00 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c45800640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea64c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c3fe00640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea64bff0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5566ea549b80 tx=0x7f9c2c004890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e20c4767a01ce4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.661+0000 7f9c26a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c26a00640 1 -- 172.21.15.37:0/237895544 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c02c900 con 0x5566ea63dc00 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c26a00640 1 -- 172.21.15.37:0/237895544 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9c2c02caa0 con 0x5566ea63dc00 2024-03-21T09:21:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c2c04d070 con 0x5566ea63dc00 2024-03-21T09:21:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c26a00640 1 -- 172.21.15.37:0/237895544 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c2c02ccc0 con 0x5566ea63dc00 2024-03-21T09:21:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c4a5f6740 10 client.27646 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5566ea6b1a80 con 0x5566ea63dc00 2024-03-21T09:21:23.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c26a00640 1 -- 172.21.15.37:0/237895544 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c2c04d070 con 0x5566ea63dc00 2024-03-21T09:21:23.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c26a00640 1 -- 172.21.15.37:0/237895544 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9c2c003790 con 0x5566ea63dc00 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c4a5f6740 10 client.27646 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c4a5f6740 10 client.27646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.662+0000 7f9c4a5f6740 20 client.27646 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.663+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6b2120 0x5566ea6b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.663+0000 7f9c4a5f6740 4 client.27646 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.663+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566ea5a8ae0 con 0x5566ea6b2120 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.663+0000 7f9c45800640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6b2120 0x5566ea6b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.663+0000 7f9c45800640 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6b2120 0x5566ea6b4540 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2716dbe7ff23f475 server_cookie=9f0526951a3e1660 in_seq=0 out_seq=0 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.663+0000 7f9c26a00640 10 client.27646 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c26a00640 1 -- 172.21.15.37:0/237895544 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5566ea5a8ae0 con 0x5566ea6b2120 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c26a00640 10 client.27646 handle_command_reply: tid=0 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 client.27646 shutdown 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 msgr2=0x5566ea64bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea64bff0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5566ea549b80 tx=0x7f9c2c004890 comp rx=0 tx=0).stop 2024-03-21T09:21:23.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 shutdown_connections 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566ea646290 0x5566ea64c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566ea63dc00 0x5566ea64bff0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5566ea6b2120 0x5566ea6b4540 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 --2- 172.21.15.37:0/237895544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5566ea63cf20 0x5566ea64bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c26a00640 0 client.27646 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 >> 172.21.15.37:0/237895544 conn(0x5566ea604200 msgr2=0x5566ea636200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 shutdown_connections 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 1 -- 172.21.15.37:0/237895544 wait complete. 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.664+0000 7f9c4a5f6740 20 client.27646 trim_cache size 0 max 16384 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:21:23.665 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:21:23.666 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:21:23.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9c280a3110 msgr2=0x7f9c280a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9c280a3110 0x7f9c280a55d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5566ea63e150 tx=0x7f9c340d3b30 comp rx=0 tx=0).stop 2024-03-21T09:21:23.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 msgr2=0x7f9c4019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c4019a300 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e9f0 tx=0x7f9c3400edf0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 shutdown_connections 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40105d20 0x7f9c4019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c40105380 0x7f9c4019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f9c280a3110 0x7f9c280a55d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 --2- 172.21.15.37:0/1728198016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c40103c40 0x7f9c4019a300 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 >> 172.21.15.37:0/1728198016 conn(0x7f9c400ff350 msgr2=0x7f9c40101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 shutdown_connections 2024-03-21T09:21:23.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.665+0000 7f9c47800640 1 -- 172.21.15.37:0/1728198016 wait complete. 2024-03-21T09:21:23.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:23 smithi037 ceph-mon[31830]: pgmap v285: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:21:23.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:21:23.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:21:23.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:23 smithi053 ceph-mon[36169]: pgmap v285: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:21:23.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:23 smithi071 ceph-mon[36597]: pgmap v285: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-21T09:21:23.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.896+0000 7fcba1200640 1 -- 172.21.15.37:0/3721112014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c1028c0 msgr2=0x7fcb9c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.896+0000 7fcba1200640 1 --2- 172.21.15.37:0/3721112014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c1028c0 0x7fcb9c102ca0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009ca0 tx=0x7fcb8c02c180 comp rx=0 tx=0).stop 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 -- 172.21.15.37:0/3721112014 shutdown_connections 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 --2- 172.21.15.37:0/3721112014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 --2- 172.21.15.37:0/3721112014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c1031e0 0x7fcb9c10b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 --2- 172.21.15.37:0/3721112014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c1028c0 0x7fcb9c102ca0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 -- 172.21.15.37:0/3721112014 >> 172.21.15.37:0/3721112014 conn(0x7fcb9c077720 msgr2=0x7fcb9c077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 -- 172.21.15.37:0/3721112014 shutdown_connections 2024-03-21T09:21:23.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.897+0000 7fcba1200640 1 -- 172.21.15.37:0/3721112014 wait complete. 2024-03-21T09:21:23.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.898+0000 7fcba1200640 1 Processor -- start 2024-03-21T09:21:23.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.898+0000 7fcba1200640 1 -- start start 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcba1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c1028c0 0x7fcb9c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcba1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1031e0 0x7fcb9c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcba1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcba1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c111a30 con 0x7fcb9c1028c0 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcba1200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1118b0 con 0x7fcb9c1031e0 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcba1200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c111bb0 con 0x7fcb9c10bc90 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcb99400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1031e0 0x7fcb9c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c1028c0 0x7fcb9c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcb99400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1031e0 0x7fcb9c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44404/0 (socket says 172.21.15.37:44404) 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56788/0 (socket says 172.21.15.37:56788) 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.899+0000 7fcb99400640 1 -- 172.21.15.37:0/2089591221 learned_addr learned my addr 172.21.15.37:0/2089591221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb9a800640 1 -- 172.21.15.37:0/2089591221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1031e0 msgr2=0x7fcb9c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb9a800640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1031e0 0x7fcb9c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb9a800640 1 -- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c1028c0 msgr2=0x7fcb9c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb9a800640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c1028c0 0x7fcb9c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb9a800640 1 -- 172.21.15.37:0/2089591221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c19f120 con 0x7fcb9c10bc90 2024-03-21T09:21:23.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb9a800640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c19e9f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e970 tx=0x7fcb9400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ccc0 con 0x7fcb9c10bc90 2024-03-21T09:21:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb940035f0 con 0x7fcb9c10bc90 2024-03-21T09:21:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ce60 con 0x7fcb9c10bc90 2024-03-21T09:21:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c19f410 con 0x7fcb9c10bc90 2024-03-21T09:21:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.900+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c1a6ca0 con 0x7fcb9c10bc90 2024-03-21T09:21:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.901+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c068520 con 0x7fcb9c10bc90 2024-03-21T09:21:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.903+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcb940037a0 con 0x7fcb9c10bc90 2024-03-21T09:21:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.905+0000 7fcb92a00640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb800a3110 0x7fcb800a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.905+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcb940cfd80 con 0x7fcb9c10bc90 2024-03-21T09:21:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.905+0000 7fcb99e00640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb800a3110 0x7fcb800a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.906+0000 7fcb99e00640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb800a3110 0x7fcb800a55d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c000f80 tx=0x7fcb8c009360 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:23.909+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb9409c410 con 0x7fcb9c10bc90 2024-03-21T09:21:24.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.150+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb9c02d070 con 0x7fcb9c10bc90 2024-03-21T09:21:24.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.151+0000 7fcb92a00640 1 -- 172.21.15.37:0/2089591221 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7fcb9409bd10 con 0x7fcb9c10bc90 2024-03-21T09:21:24.149 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:21:24.149 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:21:24.149 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:21:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.153+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb800a3110 msgr2=0x7fcb800a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.153+0000 7fcba1200640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb800a3110 0x7fcb800a55d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c000f80 tx=0x7fcb8c009360 comp rx=0 tx=0).stop 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.153+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 msgr2=0x7fcb9c19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.153+0000 7fcba1200640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c19e9f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e970 tx=0x7fcb9400ee30 comp rx=0 tx=0).stop 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 shutdown_connections 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c10bc90 0x7fcb9c19e9f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb800a3110 0x7fcb800a55d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1031e0 0x7fcb9c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 --2- 172.21.15.37:0/2089591221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb9c1028c0 0x7fcb9c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 >> 172.21.15.37:0/2089591221 conn(0x7fcb9c077720 msgr2=0x7fcb9c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 shutdown_connections 2024-03-21T09:21:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.154+0000 7fcba1200640 1 -- 172.21.15.37:0/2089591221 wait complete. 2024-03-21T09:21:24.174 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:21:24.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 -- 172.21.15.37:0/3136732209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 msgr2=0x7fcc78108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3136732209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc78108220 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009930 tx=0x7fcc6802bc30 comp rx=0 tx=0).stop 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 -- 172.21.15.37:0/3136732209 shutdown_connections 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3136732209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc78108760 0x7fcc78114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3136732209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc78108220 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3136732209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78109b70 0x7fcc78109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 -- 172.21.15.37:0/3136732209 >> 172.21.15.37:0/3136732209 conn(0x7fcc78077540 msgr2=0x7fcc781064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 -- 172.21.15.37:0/3136732209 shutdown_connections 2024-03-21T09:21:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.393+0000 7fcc7f800640 1 -- 172.21.15.37:0/3136732209 wait complete. 2024-03-21T09:21:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.394+0000 7fcc7f800640 1 Processor -- start 2024-03-21T09:21:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 -- start start 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc7819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78108760 0x7fcc7819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc78109b70 0x7fcc781a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78118170 con 0x7fcc78108760 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78117ff0 con 0x7fcc78107dc0 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7f800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781182f0 con 0x7fcc78109b70 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc7819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc7819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44414/0 (socket says 172.21.15.37:44414) 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7ce00640 1 -- 172.21.15.37:0/3003389072 learned_addr learned my addr 172.21.15.37:0/3003389072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc77e00640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78108760 0x7fcc7819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.395+0000 7fcc7d800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc78109b70 0x7fcc781a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7ce00640 1 -- 172.21.15.37:0/3003389072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc78109b70 msgr2=0x7fcc781a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc78109b70 0x7fcc781a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7ce00640 1 -- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78108760 msgr2=0x7fcc7819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78108760 0x7fcc7819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7ce00640 1 -- 172.21.15.37:0/3003389072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc781a3600 con 0x7fcc78107dc0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc7819e610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d950 tx=0x7fcc6c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc75600640 1 -- 172.21.15.37:0/3003389072 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017030 con 0x7fcc78107dc0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc75600640 1 -- 172.21.15.37:0/3003389072 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c004020 con 0x7fcc78107dc0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc75600640 1 -- 172.21.15.37:0/3003389072 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0047a0 con 0x7fcc78107dc0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.396+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc781a38f0 con 0x7fcc78107dc0 2024-03-21T09:21:24.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.397+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc781ab180 con 0x7fcc78107dc0 2024-03-21T09:21:24.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.399+0000 7fcc75600640 1 -- 172.21.15.37:0/3003389072 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcc6c0041d0 con 0x7fcc78107dc0 2024-03-21T09:21:24.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 Processor -- start 2024-03-21T09:21:24.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 -- start start 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9e6d0 0x55e557e9eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 0x55e557ea7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557eaa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e557e022c0 con 0x55e557e9e6d0 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e557daac90 con 0x55e557e9f080 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.400+0000 7fcc8259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e557dfe360 con 0x55e557ea7c00 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 0x55e557ea7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9e6d0 0x55e557e9eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557eaa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 0x55e557ea7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44418/0 (socket says 172.21.15.37:44418) 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 -- 172.21.15.37:0/227367055 learned_addr learned my addr 172.21.15.37:0/227367055 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc75600640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcc600a3110 0x7fcc600a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 -- 172.21.15.37:0/227367055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 msgr2=0x55e557eaa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557eaa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 -- 172.21.15.37:0/227367055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9e6d0 msgr2=0x55e557e9eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9e6d0 0x55e557e9eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7ce00640 1 -- 172.21.15.37:0/227367055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e557dcf080 con 0x55e557e9f080 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc75600640 1 -- 172.21.15.37:0/3003389072 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcc6c0cf540 con 0x7fcc78107dc0 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.401+0000 7fcc7d800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcc600a3110 0x7fcc600a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.402+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 0x55e557ea7630 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0cc770 tx=0x7fcc6c065fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d89b8554324d23b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.402+0000 7fcc66000640 1 -- 172.21.15.37:0/227367055 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c09b010 con 0x55e557e9f080 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.402+0000 7fcc66000640 1 -- 172.21.15.37:0/227367055 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c09bbd0 con 0x55e557e9f080 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.402+0000 7fcc66000640 1 -- 172.21.15.37:0/227367055 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c097040 con 0x55e557e9f080 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 -- 172.21.15.37:0/227367055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 msgr2=0x55e557ea7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc7d800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcc600a3110 0x7fcc600a55d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005fd0 tx=0x7fcc70004a40 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 0x55e557ea7630 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0cc770 tx=0x7fcc6c065fd0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 -- 172.21.15.37:0/227367055 shutdown_connections 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557eaa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9f080 0x55e557ea7630 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 --2- 172.21.15.37:0/227367055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9e6d0 0x55e557e9eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 -- 172.21.15.37:0/227367055 >> 172.21.15.37:0/227367055 conn(0x55e557e98da0 msgr2=0x55e557e99180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 -- 172.21.15.37:0/227367055 shutdown_connections 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 -- 172.21.15.37:0/227367055 wait complete. 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.403+0000 7fcc8259a740 1 -- start start 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 Processor -- start 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9f080 0x55e557ea4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557ea4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e557eaab10 con 0x55e557e9f080 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e557dbe0d0 con 0x55e557e9e6d0 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e557eaac90 con 0x55e557ea7c00 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc8259a740 10 client.0 _ll_register_callbacks cb 0x55e557dc2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9f080 0x55e557ea4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc7d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557ea4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44420/0 (socket says 172.21.15.37:44420) 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.404+0000 7fcc7ce00640 1 -- 172.21.15.37:0/630839696 learned_addr learned my addr 172.21.15.37:0/630839696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc7ce00640 1 -- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 msgr2=0x55e557ea4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557ea4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc7ce00640 1 -- 172.21.15.37:0/630839696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9f080 msgr2=0x55e557ea4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9f080 0x55e557ea4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc7ce00640 1 -- 172.21.15.37:0/630839696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e557b163f0 con 0x55e557e9e6d0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea7410 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0a39b0 tx=0x7fcc6c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81fce155fa94c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc5aa00640 1 -- 172.21.15.37:0/630839696 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c09ae30 con 0x55e557e9e6d0 2024-03-21T09:21:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc5aa00640 1 -- 172.21.15.37:0/630839696 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcc6c0972a0 con 0x55e557e9e6d0 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.405+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc6c09bbd0 con 0x55e557e9e6d0 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc5aa00640 1 -- 172.21.15.37:0/630839696 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c097040 con 0x55e557e9e6d0 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 10 client.27635 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e557be07d0 con 0x55e557e9e6d0 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc5aa00640 1 -- 172.21.15.37:0/630839696 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c0b8020 con 0x55e557e9e6d0 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc5aa00640 1 -- 172.21.15.37:0/630839696 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcc6c0ccab0 con 0x55e557e9e6d0 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 10 client.27635 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 10 client.27635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 20 client.27635 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557eb7840 0x55e557eb9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 4 client.27635 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.406+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e557e0f800 con 0x55e557eb7840 2024-03-21T09:21:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.407+0000 7fcc77e00640 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557eb7840 0x55e557eb9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.407+0000 7fcc77e00640 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557eb7840 0x55e557eb9c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97c56f6c67192216 server_cookie=d0229949de749360 in_seq=0 out_seq=0 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.407+0000 7fcc5aa00640 10 client.27635 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc5aa00640 1 -- 172.21.15.37:0/630839696 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e557e0f800 con 0x55e557eb7840 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc5aa00640 10 client.27635 handle_command_reply: tid=0 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 client.27635 shutdown 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 msgr2=0x55e557ea7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea7410 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0a39b0 tx=0x7fcc6c000bc0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 shutdown_connections 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557ea7c00 0x55e557ea4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557eb7840 0x55e557eb9c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557e9f080 0x55e557ea4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc5aa00640 0 client.27635 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 --2- 172.21.15.37:0/630839696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea7410 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 >> 172.21.15.37:0/630839696 conn(0x55e557e98da0 msgr2=0x55e557e9aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 shutdown_connections 2024-03-21T09:21:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.408+0000 7fcc8259a740 1 -- 172.21.15.37:0/630839696 wait complete. 2024-03-21T09:21:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.409+0000 7fcc8259a740 20 client.27635 trim_cache size 0 max 16384 2024-03-21T09:21:24.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.422+0000 7fcc8259a740 1 Processor -- start 2024-03-21T09:21:24.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.422+0000 7fcc8259a740 1 -- start start 2024-03-21T09:21:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.422+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.422+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557ea6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557e9fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc8259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781b76a0 con 0x55e557ea7c00 2024-03-21T09:21:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc8259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781b51a0 con 0x55e557e9e6d0 2024-03-21T09:21:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc8259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781b3260 con 0x55e557e9f080 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557ea6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44434/0 (socket says 172.21.15.37:44434) 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc7ce00640 1 -- 172.21.15.37:0/2514751254 learned_addr learned my addr 172.21.15.37:0/2514751254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557ea6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56842/0 (socket says 172.21.15.37:56842) 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc7d800640 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557e9fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 -- 172.21.15.37:0/2514751254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 msgr2=0x55e557ea40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 -- 172.21.15.37:0/2514751254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 msgr2=0x55e557e9fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557e9fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc77e00640 1 -- 172.21.15.37:0/2514751254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e557ea2d60 con 0x55e557e9f080 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.423+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc77e00640 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557ea6f40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcc68036040 tx=0x7fcc6802df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0ffc0860974fb7b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc65600640 1 -- 172.21.15.37:0/2514751254 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680040c0 con 0x55e557e9f080 2024-03-21T09:21:24.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc65600640 1 -- 172.21.15.37:0/2514751254 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc68051030 con 0x55e557e9f080 2024-03-21T09:21:24.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc65600640 1 -- 172.21.15.37:0/2514751254 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68004b60 con 0x55e557e9f080 2024-03-21T09:21:24.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc8259a740 1 -- 172.21.15.37:0/2514751254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 msgr2=0x55e557ea6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc8259a740 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557ea6f40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcc68036040 tx=0x7fcc6802df90 comp rx=0 tx=0).stop 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc8259a740 1 -- 172.21.15.37:0/2514751254 shutdown_connections 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.424+0000 7fcc8259a740 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557e9fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557ea6f40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 --2- 172.21.15.37:0/2514751254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557ea40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 -- 172.21.15.37:0/2514751254 >> 172.21.15.37:0/2514751254 conn(0x55e557e98da0 msgr2=0x55e557e9f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 -- 172.21.15.37:0/2514751254 shutdown_connections 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 -- 172.21.15.37:0/2514751254 wait complete. 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 -- start start 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.425+0000 7fcc8259a740 1 Processor -- start 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557eab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557eabba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557f11dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781b3860 con 0x55e557ea7c00 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781b4b40 con 0x55e557e9e6d0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781b3e80 con 0x55e557e9f080 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc8259a740 10 client.0 _ll_register_callbacks cb 0x7fcc70009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557eab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557eab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44446/0 (socket says 172.21.15.37:44446) 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 -- 172.21.15.37:0/1018317994 learned_addr learned my addr 172.21.15.37:0/1018317994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557eabba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7d800640 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557f11dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 -- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 msgr2=0x55e557eabba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557eabba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 -- 172.21.15.37:0/1018317994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 msgr2=0x55e557f11dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557f11dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 -- 172.21.15.37:0/1018317994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e557eac470 con 0x55e557e9e6d0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.426+0000 7fcc7ce00640 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557eab660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009610 tx=0x7fcc6c09f620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c92e817af3d26f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc5aa00640 1 -- 172.21.15.37:0/1018317994 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c09b010 con 0x55e557e9e6d0 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc5aa00640 1 -- 172.21.15.37:0/1018317994 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcc6c0be5c0 con 0x55e557e9e6d0 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc68051030 con 0x55e557e9e6d0 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc5aa00640 1 -- 172.21.15.37:0/1018317994 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c09ae30 con 0x55e557e9e6d0 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc8259a740 10 client.27641 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e557f13400 con 0x55e557e9e6d0 2024-03-21T09:21:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc5aa00640 1 -- 172.21.15.37:0/1018317994 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc68051030 con 0x55e557e9e6d0 2024-03-21T09:21:24.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:24 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2089591221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc5aa00640 1 -- 172.21.15.37:0/1018317994 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcc6c0cf170 con 0x55e557e9e6d0 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.427+0000 7fcc8259a740 10 client.27641 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc8259a740 10 client.27641 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc8259a740 20 client.27641 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc8259a740 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557f13830 0x55e557f15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc8259a740 4 client.27641 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e557e09b50 con 0x55e557f13830 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc77e00640 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557f13830 0x55e557f15c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc5aa00640 10 client.27641 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.428+0000 7fcc77e00640 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557f13830 0x55e557f15c20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fc7a41fd8b440b0 server_cookie=f02b8c2d3555547f in_seq=0 out_seq=0 2024-03-21T09:21:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc5aa00640 1 -- 172.21.15.37:0/1018317994 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e557e09b50 con 0x55e557f13830 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc5aa00640 10 client.27641 handle_command_reply: tid=0 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 client.27641 shutdown 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 msgr2=0x55e557eab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557eab660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009610 tx=0x7fcc6c09f620 comp rx=0 tx=0).stop 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 shutdown_connections 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55e557f13830 0x55e557f15c20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e557ea7c00 0x55e557f11dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e557e9f080 0x55e557eabba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc5aa00640 0 client.27641 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 --2- 172.21.15.37:0/1018317994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e557e9e6d0 0x55e557eab660 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 >> 172.21.15.37:0/1018317994 conn(0x55e557e98da0 msgr2=0x55e557e9fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 shutdown_connections 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 1 -- 172.21.15.37:0/1018317994 wait complete. 2024-03-21T09:21:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.429+0000 7fcc8259a740 20 client.27641 trim_cache size 0 max 16384 2024-03-21T09:21:24.432 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:21:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcc600a3110 msgr2=0x7fcc600a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcc600a3110 0x7fcc600a55d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005fd0 tx=0x7fcc70004a40 comp rx=0 tx=0).stop 2024-03-21T09:21:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 msgr2=0x7fcc7819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc7819e610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d950 tx=0x7fcc6c00de10 comp rx=0 tx=0).stop 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 shutdown_connections 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc78109b70 0x7fcc781a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78108760 0x7fcc7819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcc600a3110 0x7fcc600a55d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 --2- 172.21.15.37:0/3003389072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc78107dc0 0x7fcc7819e610 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 >> 172.21.15.37:0/3003389072 conn(0x7fcc78077540 msgr2=0x7fcc78077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 shutdown_connections 2024-03-21T09:21:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.430+0000 7fcc7f800640 1 -- 172.21.15.37:0/3003389072 wait complete. 2024-03-21T09:21:24.445 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:21:24.445 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:21:24.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 -- 172.21.15.37:0/1814580205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4074360 msgr2=0x7f84e4111f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 --2- 172.21.15.37:0/1814580205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4074360 0x7f84e4111f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 -- 172.21.15.37:0/1814580205 shutdown_connections 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 --2- 172.21.15.37:0/1814580205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84e4112470 0x7f84e4114860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 --2- 172.21.15.37:0/1814580205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4074360 0x7f84e4111f30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 --2- 172.21.15.37:0/1814580205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e4073a40 0x7f84e4073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 -- 172.21.15.37:0/1814580205 >> 172.21.15.37:0/1814580205 conn(0x7f84e40fcff0 msgr2=0x7f84e40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 -- 172.21.15.37:0/1814580205 shutdown_connections 2024-03-21T09:21:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.660+0000 7f84ea200640 1 -- 172.21.15.37:0/1814580205 wait complete. 2024-03-21T09:21:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 Processor -- start 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 -- start start 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 0x7f84e419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e4074360 0x7f84e419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84e4112470 0x7f84e41a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4117ec0 con 0x7f84e4074360 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4117d40 con 0x7f84e4073a40 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.661+0000 7f84ea200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4118040 con 0x7f84e4112470 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 0x7f84e419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 0x7f84e419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44456/0 (socket says 172.21.15.37:44456) 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 -- 172.21.15.37:0/1914913806 learned_addr learned my addr 172.21.15.37:0/1914913806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e4074360 0x7f84e419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e9800640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84e4112470 0x7f84e41a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 -- 172.21.15.37:0/1914913806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84e4112470 msgr2=0x7f84e41a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84e4112470 0x7f84e41a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 -- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e4074360 msgr2=0x7f84e419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e4074360 0x7f84e419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.662+0000 7f84e8e00640 1 -- 172.21.15.37:0/1914913806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e41a33a0 con 0x7f84e4073a40 2024-03-21T09:21:24.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.663+0000 7f84e8e00640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 0x7f84e419e400 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f84d400d950 tx=0x7f84d400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.663+0000 7f84e2000640 1 -- 172.21.15.37:0/1914913806 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4017070 con 0x7f84e4073a40 2024-03-21T09:21:24.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.663+0000 7f84e2000640 1 -- 172.21.15.37:0/1914913806 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84d4009c30 con 0x7f84e4073a40 2024-03-21T09:21:24.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.663+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e41a3690 con 0x7f84e4073a40 2024-03-21T09:21:24.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.663+0000 7f84e2000640 1 -- 172.21.15.37:0/1914913806 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d400f6b0 con 0x7f84e4073a40 2024-03-21T09:21:24.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.663+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e41aaf20 con 0x7f84e4073a40 2024-03-21T09:21:24.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.666+0000 7f84e2000640 1 -- 172.21.15.37:0/1914913806 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f84d40109d0 con 0x7f84e4073a40 2024-03-21T09:21:24.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.666+0000 7f84ee5ee740 1 Processor -- start 2024-03-21T09:21:24.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 -- start start 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc69c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc72810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc75230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfcfbcd2c0 con 0x55cfcfc6a260 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfcfb75c90 con 0x55cfcfc72de0 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfcfbc9360 con 0x55cfcfc698b0 2024-03-21T09:21:24.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84e9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc69c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84e8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc72810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84e9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc69c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56870/0 (socket says 172.21.15.37:56870) 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84e9800640 1 -- 172.21.15.37:0/2093068197 learned_addr learned my addr 172.21.15.37:0/2093068197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.667+0000 7f84dbe00640 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc75230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e9800640 1 -- 172.21.15.37:0/2093068197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 msgr2=0x55cfcfc75230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e9800640 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc75230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e9800640 1 -- 172.21.15.37:0/2093068197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 msgr2=0x55cfcfc72810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e9800640 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc72810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e9800640 1 -- 172.21.15.37:0/2093068197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfcfb9a080 con 0x55cfcfc698b0 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84dbe00640 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc75230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:24.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e2000640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c80a3110 0x7f84c80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e2000640 1 -- 172.21.15.37:0/1914913806 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f84d40cf790 con 0x7f84e4073a40 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e9800640 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc69c90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009f30 tx=0x7f84dc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31d0b35c324f697 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84e8e00640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c80a3110 0x7f84c80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84da000640 1 -- 172.21.15.37:0/2093068197 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004d20 con 0x55cfcfc698b0 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84da000640 1 -- 172.21.15.37:0/2093068197 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84dc004050 con 0x55cfcfc698b0 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.668+0000 7f84da000640 1 -- 172.21.15.37:0/2093068197 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc005260 con 0x55cfcfc698b0 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 -- 172.21.15.37:0/2093068197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 msgr2=0x55cfcfc69c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84e8e00640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c80a3110 0x7f84c80a55d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55cfcfc69380 tx=0x7f84d40d5530 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc69c90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009f30 tx=0x7f84dc0118d0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 -- 172.21.15.37:0/2093068197 shutdown_connections 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc75230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc72810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/2093068197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc69c90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 -- 172.21.15.37:0/2093068197 >> 172.21.15.37:0/2093068197 conn(0x55cfcfc63f80 msgr2=0x55cfcfc64360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 -- 172.21.15.37:0/2093068197 shutdown_connections 2024-03-21T09:21:24.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.669+0000 7f84ee5ee740 1 -- 172.21.15.37:0/2093068197 wait complete. 2024-03-21T09:21:24.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.670+0000 7f84ee5ee740 1 -- start start 2024-03-21T09:21:24.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.670+0000 7f84ee5ee740 1 Processor -- start 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.670+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc698b0 0x55cfcfc726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.670+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc6a260 0x55cfcfc6fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfcfc75d50 con 0x55cfcfc698b0 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfcfb890d0 con 0x55cfcfc72de0 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfcfc75ed0 con 0x55cfcfc6a260 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84ee5ee740 10 client.0 _ll_register_callbacks cb 0x55cfcfb8de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84e9800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc698b0 0x55cfcfc726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84e9800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc698b0 0x55cfcfc726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34938/0 (socket says 172.21.15.37:34938) 2024-03-21T09:21:24.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc6a260 0x55cfcfc6fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84e9800640 1 -- 172.21.15.37:0/1320180513 learned_addr learned my addr 172.21.15.37:0/1320180513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84e9800640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84dbe00640 1 -- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 msgr2=0x55cfcfc6ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84dbe00640 1 -- 172.21.15.37:0/1320180513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc698b0 msgr2=0x55cfcfc726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc698b0 0x55cfcfc726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84dbe00640 1 -- 172.21.15.37:0/1320180513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfcf8e13f0 con 0x55cfcfc6a260 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.671+0000 7f84e9800640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc6a260 0x55cfcfc6fa50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02c200 tx=0x7f84cc005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ebe049f9f4da4a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84d2a00640 1 -- 172.21.15.37:0/1320180513 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc043070 con 0x55cfcfc6a260 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84d2a00640 1 -- 172.21.15.37:0/1320180513 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f84cc02cba0 con 0x55cfcfc6a260 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84dc004050 con 0x55cfcfc6a260 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84d2a00640 1 -- 172.21.15.37:0/1320180513 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84cc02cda0 con 0x55cfcfc6a260 2024-03-21T09:21:24.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84ee5ee740 10 client.27682 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cfcf9ab7d0 con 0x55cfcfc6a260 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.672+0000 7f84d2a00640 1 -- 172.21.15.37:0/1320180513 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84dc004050 con 0x55cfcfc6a260 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84d2a00640 1 -- 172.21.15.37:0/1320180513 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f84cc0054c0 con 0x55cfcfc6a260 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84ee5ee740 10 client.27682 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84ee5ee740 10 client.27682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84ee5ee740 20 client.27682 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfc82c20 0x55cfcfc85010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84ee5ee740 4 client.27682 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfcfbda800 con 0x55cfcfc82c20 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.673+0000 7f84e9800640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfc82c20 0x55cfcfc85010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.674+0000 7f84e9800640 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfc82c20 0x55cfcfc85010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe5863fc40baf385 server_cookie=8104b5beb45f5473 in_seq=0 out_seq=0 2024-03-21T09:21:24.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.674+0000 7f84d2a00640 10 client.27682 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.674+0000 7f84d2a00640 1 -- 172.21.15.37:0/1320180513 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfcfbda800 con 0x55cfcfc82c20 2024-03-21T09:21:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.674+0000 7f84d2a00640 10 client.27682 handle_command_reply: tid=0 2024-03-21T09:21:24.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.674+0000 7f84ee5ee740 1 client.27682 shutdown 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc6a260 msgr2=0x55cfcfc6fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc6a260 0x55cfcfc6fa50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02c200 tx=0x7f84cc005e30 comp rx=0 tx=0).stop 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 shutdown_connections 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfc82c20 0x55cfcfc85010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc6a260 0x55cfcfc6fa50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1320180513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc698b0 0x55cfcfc726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84d2a00640 0 client.27682 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 >> 172.21.15.37:0/1320180513 conn(0x55cfcfc63f80 msgr2=0x55cfcfc65cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 shutdown_connections 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1320180513 wait complete. 2024-03-21T09:21:24.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.675+0000 7f84ee5ee740 20 client.27682 trim_cache size 0 max 16384 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 Processor -- start 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 -- start start 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc72180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41b74a0 con 0x55cfcfc6a260 2024-03-21T09:21:24.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41b4fa0 con 0x55cfcfc72de0 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.689+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41b3060 con 0x55cfcfc698b0 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc72180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc72180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56894/0 (socket says 172.21.15.37:56894) 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 -- 172.21.15.37:0/269216991 learned_addr learned my addr 172.21.15.37:0/269216991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84dbe00640 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 -- 172.21.15.37:0/269216991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 msgr2=0x55cfcfc6ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 -- 172.21.15.37:0/269216991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 msgr2=0x55cfcfc726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 -- 172.21.15.37:0/269216991 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfcfc85640 con 0x55cfcfc698b0 2024-03-21T09:21:24.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84e9800640 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc72180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f84dc014040 tx=0x7f84dc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122b276f55a7c905 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84d9600640 1 -- 172.21.15.37:0/269216991 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc003970 con 0x55cfcfc698b0 2024-03-21T09:21:24.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84d9600640 1 -- 172.21.15.37:0/269216991 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84dc004940 con 0x55cfcfc698b0 2024-03-21T09:21:24.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.690+0000 7f84d9600640 1 -- 172.21.15.37:0/269216991 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc02e780 con 0x55cfcfc698b0 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 -- 172.21.15.37:0/269216991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 msgr2=0x55cfcfc72180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc72180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f84dc014040 tx=0x7f84dc005bd0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 -- 172.21.15.37:0/269216991 shutdown_connections 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc72de0 0x55cfcfc6ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc6a260 0x55cfcfc726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/269216991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc72180 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 -- 172.21.15.37:0/269216991 >> 172.21.15.37:0/269216991 conn(0x55cfcfc63f80 msgr2=0x55cfcfc63160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 -- 172.21.15.37:0/269216991 shutdown_connections 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.691+0000 7f84ee5ee740 1 -- 172.21.15.37:0/269216991 wait complete. 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 -- start start 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 Processor -- start 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc76dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 0x55cfcfc77310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc72de0 0x55cfcfcd7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41b3660 con 0x55cfcfc72de0 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41b4940 con 0x55cfcfc6a260 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41b3c80 con 0x55cfcfc698b0 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84ee5ee740 10 client.0 _ll_register_callbacks cb 0x7f84cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.692+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 0x55cfcfc77310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84e9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc76dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 0x55cfcfc77310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44506/0 (socket says 172.21.15.37:44506) 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 -- 172.21.15.37:0/1532705716 learned_addr learned my addr 172.21.15.37:0/1532705716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84e9800640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc72de0 0x55cfcfcd7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 -- 172.21.15.37:0/1532705716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 msgr2=0x55cfcfc76dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc76dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 -- 172.21.15.37:0/1532705716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc72de0 msgr2=0x55cfcfcd7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc72de0 0x55cfcfcd7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 -- 172.21.15.37:0/1532705716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfcfc693b0 con 0x55cfcfc6a260 2024-03-21T09:21:24.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84e9800640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc76dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84dbe00640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 0x55cfcfc77310 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f84cc001eb0 tx=0x7f84cc02bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee9cfdbb4f3c1198 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84d2a00640 1 -- 172.21.15.37:0/1532705716 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc043040 con 0x55cfcfc6a260 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84d2a00640 1 -- 172.21.15.37:0/1532705716 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f84cc0046c0 con 0x55cfcfc6a260 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.693+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84dc004940 con 0x55cfcfc6a260 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84d2a00640 1 -- 172.21.15.37:0/1532705716 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84cc005850 con 0x55cfcfc6a260 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 10 client.27659 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:24.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cfcfcdec80 con 0x55cfcfc6a260 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84d2a00640 1 -- 172.21.15.37:0/1532705716 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84dc004940 con 0x55cfcfc6a260 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84d2a00640 1 -- 172.21.15.37:0/1532705716 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f84cc056460 con 0x55cfcfc6a260 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 10 client.27659 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 10 client.27659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 20 client.27659 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfcdf380 0x55cfcfce17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 4 client.27659 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.694+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cfcfbd4b50 con 0x55cfcfcdf380 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84e9800640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfcdf380 0x55cfcfce17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84e9800640 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfcdf380 0x55cfcfce17a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3de1dba076d7b0b server_cookie=d64714b25026bf62 in_seq=0 out_seq=0 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84d2a00640 10 client.27659 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84d2a00640 1 -- 172.21.15.37:0/1532705716 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cfcfbd4b50 con 0x55cfcfcdf380 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84d2a00640 10 client.27659 handle_command_reply: tid=0 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84ee5ee740 1 client.27659 shutdown 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 msgr2=0x55cfcfc77310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.695+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 0x55cfcfc77310 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f84cc001eb0 tx=0x7f84cc02bcc0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 shutdown_connections 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55cfcfcdf380 0x55cfcfce17a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cfcfc72de0 0x55cfcfcd7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cfcfc6a260 0x55cfcfc77310 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 --2- 172.21.15.37:0/1532705716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cfcfc698b0 0x55cfcfc76dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84d2a00640 0 client.27659 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 >> 172.21.15.37:0/1532705716 conn(0x55cfcfc63f80 msgr2=0x55cfcfc6a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 shutdown_connections 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 1 -- 172.21.15.37:0/1532705716 wait complete. 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.696+0000 7f84ee5ee740 20 client.27659 trim_cache size 0 max 16384 2024-03-21T09:21:24.697 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:21:24.698 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:24 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2089591221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:21:24.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c80a3110 msgr2=0x7f84c80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c80a3110 0x7f84c80a55d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55cfcfc69380 tx=0x7f84d40d5530 comp rx=0 tx=0).stop 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 msgr2=0x7f84e419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 0x7f84e419e400 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f84d400d950 tx=0x7f84d400de10 comp rx=0 tx=0).stop 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 shutdown_connections 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84e4112470 0x7f84e41a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e4074360 0x7f84e419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f84c80a3110 0x7f84c80a55d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 --2- 172.21.15.37:0/1914913806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e4073a40 0x7f84e419e400 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 >> 172.21.15.37:0/1914913806 conn(0x7f84e40fcff0 msgr2=0x7f84e40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 shutdown_connections 2024-03-21T09:21:24.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.697+0000 7f84ea200640 1 -- 172.21.15.37:0/1914913806 wait complete. 2024-03-21T09:21:24.711 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:21:24.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:21:24.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:24 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2089591221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:21:24.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 -- 172.21.15.37:0/2590653024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 msgr2=0x7f8204102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 --2- 172.21.15.37:0/2590653024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204102a10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009900 tx=0x7f81f402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 -- 172.21.15.37:0/2590653024 shutdown_connections 2024-03-21T09:21:24.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 --2- 172.21.15.37:0/2590653024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f820410ba50 0x7f820410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 --2- 172.21.15.37:0/2590653024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8204102f50 0x7f820410b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 --2- 172.21.15.37:0/2590653024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204102a10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 -- 172.21.15.37:0/2590653024 >> 172.21.15.37:0/2590653024 conn(0x7f8204077130 msgr2=0x7f82040fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 -- 172.21.15.37:0/2590653024 shutdown_connections 2024-03-21T09:21:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.922+0000 7f8209a00640 1 -- 172.21.15.37:0/2590653024 wait complete. 2024-03-21T09:21:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.924+0000 7f8209a00640 1 Processor -- start 2024-03-21T09:21:24.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.924+0000 7f8209a00640 1 -- start start 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8204102f50 0x7f820419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820410ba50 0x7f820419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8204111850 con 0x7f820410ba50 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041116d0 con 0x7f8204102f50 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041119d0 con 0x7f8204102630 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8204102f50 0x7f820419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56910/0 (socket says 172.21.15.37:56910) 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 -- 172.21.15.37:0/3352173273 learned_addr learned my addr 172.21.15.37:0/3352173273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209000640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820410ba50 0x7f820419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 -- 172.21.15.37:0/3352173273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8204102f50 msgr2=0x7f820419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8204102f50 0x7f820419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 -- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820410ba50 msgr2=0x7f820419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820410ba50 0x7f820419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203e00640 1 -- 172.21.15.37:0/3352173273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820419ef40 con 0x7f8204102630 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8203400640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8204102f50 0x7f820419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.925+0000 7f8209000640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820410ba50 0x7f820419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.926+0000 7f8203e00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204199f50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000c00 tx=0x7f81f4004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.926+0000 7f8200c00640 1 -- 172.21.15.37:0/3352173273 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4042020 con 0x7f8204102630 2024-03-21T09:21:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.926+0000 7f8200c00640 1 -- 172.21.15.37:0/3352173273 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81f4005000 con 0x7f8204102630 2024-03-21T09:21:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.926+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820419f1d0 con 0x7f8204102630 2024-03-21T09:21:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.926+0000 7f8200c00640 1 -- 172.21.15.37:0/3352173273 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003b80 con 0x7f8204102630 2024-03-21T09:21:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.926+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82041a6ac0 con 0x7f8204102630 2024-03-21T09:21:24.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 Processor -- start 2024-03-21T09:21:24.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 -- start start 2024-03-21T09:21:24.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa00230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4f9582c0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4f900c90 con 0x55fc4f9fdde0 2024-03-21T09:21:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f820dda8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4f954360 con 0x55fc4f9f5260 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f8209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f8209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34990/0 (socket says 172.21.15.37:34990) 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f8209000640 1 -- 172.21.15.37:0/2421169656 learned_addr learned my addr 172.21.15.37:0/2421169656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.930+0000 7f8203400640 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa00230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8209000640 1 -- 172.21.15.37:0/2421169656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9f5260 msgr2=0x55fc4f9fd810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8209000640 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8209000640 1 -- 172.21.15.37:0/2421169656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9fdde0 msgr2=0x55fc4fa00230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8209000640 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa00230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8209000640 1 -- 172.21.15.37:0/2421169656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4f925080 con 0x55fc4f9f48b0 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8203400640 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa00230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8200c00640 1 -- 172.21.15.37:0/3352173273 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f81f4045020 con 0x7f8204102630 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.931+0000 7f8209000640 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9f4c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005f40 tx=0x7f81fc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3057872ac37ad59 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.932+0000 7f81f9600640 1 -- 172.21.15.37:0/2421169656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc011700 con 0x55fc4f9f48b0 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.932+0000 7f81f9600640 1 -- 172.21.15.37:0/2421169656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81fc011d80 con 0x55fc4f9f48b0 2024-03-21T09:21:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.932+0000 7f81f9600640 1 -- 172.21.15.37:0/2421169656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00e3c0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.932+0000 7f820dda8740 1 -- 172.21.15.37:0/2421169656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 msgr2=0x55fc4f9f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.932+0000 7f820dda8740 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9f4c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005f40 tx=0x7f81fc010860 comp rx=0 tx=0).stop 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 -- 172.21.15.37:0/2421169656 shutdown_connections 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa00230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 --2- 172.21.15.37:0/2421169656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9f4c90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 -- 172.21.15.37:0/2421169656 >> 172.21.15.37:0/2421169656 conn(0x55fc4f9eef80 msgr2=0x55fc4f9ef360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 -- 172.21.15.37:0/2421169656 shutdown_connections 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f8200c00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f81e80a3020 0x7f81e80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 -- 172.21.15.37:0/2421169656 wait complete. 2024-03-21T09:21:24.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f8200c00640 1 -- 172.21.15.37:0/3352173273 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f81f40f1760 con 0x7f8204102630 2024-03-21T09:21:24.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f8203400640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f81e80a3020 0x7f81e80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.933+0000 7f820dda8740 1 -- start start 2024-03-21T09:21:24.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 Processor -- start 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f8203400640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f81e80a3020 0x7f81e80a54e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f82040ff540 tx=0x7f81f0005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9fd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4fa00d50 con 0x55fc4f9f5260 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4f9140d0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4fa00ed0 con 0x55fc4f9fdde0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f820dda8740 10 client.0 _ll_register_callbacks cb 0x55fc4f918e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f8203400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f8203e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f8209000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56922/0 (socket says 172.21.15.37:56922) 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8203400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34998/0 (socket says 172.21.15.37:34998) 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.934+0000 7f8203e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9fd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44552/0 (socket says 172.21.15.37:44552) 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 -- 172.21.15.37:0/3600552553 learned_addr learned my addr 172.21.15.37:0/3600552553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 -- 172.21.15.37:0/3600552553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 msgr2=0x55fc4f9fd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9fd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 -- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 msgr2=0x55fc4f9fab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 -- 172.21.15.37:0/3600552553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc4f66c3f0 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f8209000640 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fb250 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f81fc01bb30 tx=0x7f81fc01b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2005fbad5184eb27 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f81ee000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.935+0000 7f81ee000640 1 -- 172.21.15.37:0/3600552553 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc003970 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f81ee000640 1 -- 172.21.15.37:0/3600552553 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f81fc003b10 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81fc011d80 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f81ee000640 1 -- 172.21.15.37:0/3600552553 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81fc003d10 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 10 client.27697 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fc4f7367d0 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f81ee000640 1 -- 172.21.15.37:0/3600552553 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81fc011d80 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f81ee000640 1 -- 172.21.15.37:0/3600552553 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f81fc004220 con 0x55fc4f9fdde0 2024-03-21T09:21:24.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 10 client.27697 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 10 client.27697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 20 client.27697 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa0dc10 0x55fc4fa10000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 4 client.27697 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.936+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc4f965800 con 0x55fc4fa0dc10 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.937+0000 7f8203e00640 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa0dc10 0x55fc4fa10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.937+0000 7f8203e00640 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa0dc10 0x55fc4fa10000 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb39a4b13df495c server_cookie=30d390bce0394304 in_seq=0 out_seq=0 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.937+0000 7f81ee000640 10 client.27697 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f81ee000640 1 -- 172.21.15.37:0/3600552553 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc4f965800 con 0x55fc4fa0dc10 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f81ee000640 10 client.27697 handle_command_reply: tid=0 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 client.27697 shutdown 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 msgr2=0x55fc4f9fb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fb250 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f81fc01bb30 tx=0x7f81fc01b660 comp rx=0 tx=0).stop 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 shutdown_connections 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fb250 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa0dc10 0x55fc4fa10000 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 --2- 172.21.15.37:0/3600552553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f81ee000640 0 client.27697 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:24.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.938+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 >> 172.21.15.37:0/3600552553 conn(0x55fc4f9eef80 msgr2=0x55fc4f9f0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.939+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 shutdown_connections 2024-03-21T09:21:24.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.939+0000 7f820dda8740 1 -- 172.21.15.37:0/3600552553 wait complete. 2024-03-21T09:21:24.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.939+0000 7f820dda8740 20 client.27697 trim_cache size 0 max 16384 2024-03-21T09:21:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.952+0000 7f820dda8740 1 Processor -- start 2024-03-21T09:21:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.952+0000 7f820dda8740 1 -- start start 2024-03-21T09:21:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f820dda8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f820dda8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f820dda8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f820dda8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041b30a0 con 0x55fc4f9f5260 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f820dda8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041b0ba0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f820dda8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041aec60 con 0x55fc4f9fdde0 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9edfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44568/0 (socket says 172.21.15.37:44568) 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 -- 172.21.15.37:0/3340699904 learned_addr learned my addr 172.21.15.37:0/3340699904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8209000640 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203400640 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 -- 172.21.15.37:0/3340699904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 msgr2=0x55fc4f9fd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 -- 172.21.15.37:0/3340699904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 msgr2=0x55fc4f9fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8203e00640 1 -- 172.21.15.37:0/3340699904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4f9f9060 con 0x55fc4f9f48b0 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.953+0000 7f8209000640 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.954+0000 7f8203e00640 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9edfe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f81f40bd500 tx=0x7f81f4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c8a45ef81e85a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.954+0000 7f81f8c00640 1 -- 172.21.15.37:0/3340699904 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40bd800 con 0x55fc4f9f48b0 2024-03-21T09:21:24.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.954+0000 7f81f8c00640 1 -- 172.21.15.37:0/3340699904 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81f40d1e50 con 0x55fc4f9f48b0 2024-03-21T09:21:24.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.954+0000 7f81f8c00640 1 -- 172.21.15.37:0/3340699904 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40a80b0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.954+0000 7f820dda8740 1 -- 172.21.15.37:0/3340699904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 msgr2=0x55fc4f9edfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.954+0000 7f820dda8740 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9edfe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f81f40bd500 tx=0x7f81f4005d50 comp rx=0 tx=0).stop 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 -- 172.21.15.37:0/3340699904 shutdown_connections 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4f9fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f5260 0x55fc4f9fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 --2- 172.21.15.37:0/3340699904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f48b0 0x55fc4f9edfe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 -- 172.21.15.37:0/3340699904 >> 172.21.15.37:0/3340699904 conn(0x55fc4f9eef80 msgr2=0x55fc4f9ed8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 -- 172.21.15.37:0/3340699904 shutdown_connections 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 -- 172.21.15.37:0/3340699904 wait complete. 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 -- start start 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.955+0000 7f820dda8740 1 Processor -- start 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4fa06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 0x55fc4fa034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa03bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041af260 con 0x55fc4f9f48b0 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041b0540 con 0x55fc4f9f5260 2024-03-21T09:21:24.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041af880 con 0x55fc4f9fdde0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f820dda8740 10 client.0 _ll_register_callbacks cb 0x7f81f0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 0x55fc4fa034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4fa06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 0x55fc4fa034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44582/0 (socket says 172.21.15.37:44582) 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203400640 1 -- 172.21.15.37:0/3710907829 learned_addr learned my addr 172.21.15.37:0/3710907829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8209000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa03bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203e00640 1 -- 172.21.15.37:0/3710907829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 msgr2=0x55fc4fa03bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203e00640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa03bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203e00640 1 -- 172.21.15.37:0/3710907829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 msgr2=0x55fc4fa034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203e00640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 0x55fc4fa034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203e00640 1 -- 172.21.15.37:0/3710907829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc4fa68310 con 0x55fc4f9f48b0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8203400640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 0x55fc4fa034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.956+0000 7f8209000640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa03bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f8203e00640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4fa06060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f81f40bd500 tx=0x7f81f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d7374e6986a056 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f81ee000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:21:24.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f81ee000640 1 -- 172.21.15.37:0/3710907829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40da020 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f81ee000640 1 -- 172.21.15.37:0/3710907829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f81f40bbb00 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81f40d1e50 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f81ee000640 1 -- 172.21.15.37:0/3710907829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81f40bd800 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f820dda8740 10 client.17436 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.957+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fc4fa694b0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f81ee000640 1 -- 172.21.15.37:0/3710907829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81f40d1e50 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f81ee000640 1 -- 172.21.15.37:0/3710907829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f81f40054f0 con 0x55fc4f9f48b0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f820dda8740 10 client.17436 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f820dda8740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f820dda8740 20 client.17436 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f820dda8740 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa698f0 0x55fc4fa6bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f820dda8740 4 client.17436 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc4f95fb50 con 0x55fc4fa698f0 2024-03-21T09:21:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.958+0000 7f8203400640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa698f0 0x55fc4fa6bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f8203400640 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa698f0 0x55fc4fa6bd10 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51354dc74036ec11 server_cookie=254c1603b742d111 in_seq=0 out_seq=0 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f81ee000640 10 client.17436 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f81ee000640 1 -- 172.21.15.37:0/3710907829 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc4f95fb50 con 0x55fc4fa698f0 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f81ee000640 10 client.17436 handle_command_reply: tid=0 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 client.17436 shutdown 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 msgr2=0x55fc4fa06060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4fa06060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f81f40bd500 tx=0x7f81f4005fd0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 shutdown_connections 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc4f9fdde0 0x55fc4fa03bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55fc4fa698f0 0x55fc4fa6bd10 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc4f9f5260 0x55fc4fa034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 --2- 172.21.15.37:0/3710907829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc4f9f48b0 0x55fc4fa06060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f81ee000640 0 client.17436 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.959+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 >> 172.21.15.37:0/3710907829 conn(0x55fc4f9eef80 msgr2=0x55fc4f9ee160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 shutdown_connections 2024-03-21T09:21:24.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f820dda8740 1 -- 172.21.15.37:0/3710907829 wait complete. 2024-03-21T09:21:24.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f820dda8740 20 client.17436 trim_cache size 0 max 16384 2024-03-21T09:21:24.961 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:21:24.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f81e80a3020 msgr2=0x7f81e80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f8209a00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f81e80a3020 0x7f81e80a54e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f82040ff540 tx=0x7f81f0005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 msgr2=0x7f8204199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.960+0000 7f8209a00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204199f50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000c00 tx=0x7f81f4004360 comp rx=0 tx=0).stop 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 shutdown_connections 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820410ba50 0x7f820419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f81e80a3020 0x7f81e80a54e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8204102f50 0x7f820419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 --2- 172.21.15.37:0/3352173273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8204102630 0x7f8204199f50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 >> 172.21.15.37:0/3352173273 conn(0x7f8204077130 msgr2=0x7f82041001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 shutdown_connections 2024-03-21T09:21:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:24.961+0000 7f8209a00640 1 -- 172.21.15.37:0/3352173273 wait complete. 2024-03-21T09:21:25.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:25 smithi037 ceph-mon[31830]: pgmap v286: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:25.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:25 smithi053 ceph-mon[36169]: pgmap v286: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:25.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:25 smithi071 ceph-mon[36597]: pgmap v286: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:25.978 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:21:25.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:21:25.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:21:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.223+0000 7f5da9e00640 1 -- 172.21.15.37:0/1674590252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4073a40 msgr2=0x7f5da4073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.223+0000 7f5da9e00640 1 --2- 172.21.15.37:0/1674590252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4073a40 0x7f5da4073e20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009be0 tx=0x7f5d9c02bf80 comp rx=0 tx=0).stop 2024-03-21T09:21:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 -- 172.21.15.37:0/1674590252 shutdown_connections 2024-03-21T09:21:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 --2- 172.21.15.37:0/1674590252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da41124b0 0x7f5da41148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 --2- 172.21.15.37:0/1674590252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da4074360 0x7f5da4111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 --2- 172.21.15.37:0/1674590252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4073a40 0x7f5da4073e20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 -- 172.21.15.37:0/1674590252 >> 172.21.15.37:0/1674590252 conn(0x7f5da40fcfb0 msgr2=0x7f5da40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 -- 172.21.15.37:0/1674590252 shutdown_connections 2024-03-21T09:21:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.224+0000 7f5da9e00640 1 -- 172.21.15.37:0/1674590252 wait complete. 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 Processor -- start 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 -- start start 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 0x7f5da419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 0x7f5da419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da41124b0 0x7f5da41a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4117eb0 con 0x7f5da41124b0 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4117d30 con 0x7f5da4074360 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.225+0000 7f5da9e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4118030 con 0x7f5da4073a40 2024-03-21T09:21:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 0x7f5da419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da41124b0 0x7f5da41a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 0x7f5da419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 0x7f5da419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56954/0 (socket says 172.21.15.37:56954) 2024-03-21T09:21:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 0x7f5da419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44598/0 (socket says 172.21.15.37:44598) 2024-03-21T09:21:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da8a00640 1 -- 172.21.15.37:0/2901397875 learned_addr learned my addr 172.21.15.37:0/2901397875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 -- 172.21.15.37:0/2901397875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 msgr2=0x7f5da419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 0x7f5da419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 -- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da41124b0 msgr2=0x7f5da41a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da41124b0 0x7f5da41a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 -- 172.21.15.37:0/2901397875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da41a3440 con 0x7f5da4074360 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da8a00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 0x7f5da419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.226+0000 7f5da3e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 0x7f5da419e990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e9f0 tx=0x7f5d9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.227+0000 7f5da1600640 1 -- 172.21.15.37:0/2901397875 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000ccc0 con 0x7f5da4074360 2024-03-21T09:21:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.227+0000 7f5da1600640 1 -- 172.21.15.37:0/2901397875 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d90010910 con 0x7f5da4074360 2024-03-21T09:21:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.227+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da41a3730 con 0x7f5da4074360 2024-03-21T09:21:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.227+0000 7f5da1600640 1 -- 172.21.15.37:0/2901397875 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90018960 con 0x7f5da4074360 2024-03-21T09:21:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.227+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da41aafc0 con 0x7f5da4074360 2024-03-21T09:21:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.231+0000 7f5da1600640 1 -- 172.21.15.37:0/2901397875 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5d90010ac0 con 0x7f5da4074360 2024-03-21T09:21:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 Processor -- start 2024-03-21T09:21:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 -- start start 2024-03-21T09:21:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c872670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c87b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c87db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80c7d6eb0 con 0x55b80c87b790 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80c77fa60 con 0x55b80c872bb0 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.232+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80c7d2f50 con 0x55b80c872290 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c872670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c872670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56970/0 (socket says 172.21.15.37:56970) 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da9400640 1 -- 172.21.15.37:0/3205542979 learned_addr learned my addr 172.21.15.37:0/3205542979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da1600640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d880a3110 0x7f5d880a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da8a00640 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c87b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da3e00640 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c87db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da1600640 1 -- 172.21.15.37:0/2901397875 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5d900cff20 con 0x7f5da4074360 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da9400640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d880a3110 0x7f5d880a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da8a00640 1 -- 172.21.15.37:0/3205542979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 msgr2=0x55b80c872670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da8a00640 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c872670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da8a00640 1 -- 172.21.15.37:0/3205542979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 msgr2=0x55b80c87db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da8a00640 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c87db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.233+0000 7f5da8a00640 1 -- 172.21.15.37:0/3205542979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b80c5b37e0 con 0x55b80c872bb0 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.234+0000 7f5da8a00640 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c87b250 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f80 tx=0x7f5d9c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8154d337cbd103 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.234+0000 7f5d99c00640 1 -- 172.21.15.37:0/3205542979 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c043030 con 0x55b80c872bb0 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.234+0000 7f5da9400640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d880a3110 0x7f5d880a55d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005a50 tx=0x7f5d94004730 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.234+0000 7f5d99c00640 1 -- 172.21.15.37:0/3205542979 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c02ce00 con 0x55b80c872bb0 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.234+0000 7f5d99c00640 1 -- 172.21.15.37:0/3205542979 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c004780 con 0x55b80c872bb0 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 -- 172.21.15.37:0/3205542979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 msgr2=0x55b80c87b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c87b250 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f80 tx=0x7f5d9c009840 comp rx=0 tx=0).stop 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 -- 172.21.15.37:0/3205542979 shutdown_connections 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c87db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c87b250 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/3205542979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c872670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 -- 172.21.15.37:0/3205542979 >> 172.21.15.37:0/3205542979 conn(0x55b80c86c9c0 msgr2=0x55b80c86cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 -- 172.21.15.37:0/3205542979 shutdown_connections 2024-03-21T09:21:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.235+0000 7f5dae1e2740 1 -- 172.21.15.37:0/3205542979 wait complete. 2024-03-21T09:21:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.236+0000 7f5dae1e2740 1 -- start start 2024-03-21T09:21:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.236+0000 7f5dae1e2740 1 Processor -- start 2024-03-21T09:21:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c878460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c8789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80c87e520 con 0x55b80c87b790 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80c792eb0 con 0x55b80c872bb0 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80c87e6a0 con 0x55b80c872290 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5dae1e2740 10 client.0 _ll_register_callbacks cb 0x55b80c787150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c878460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56978/0 (socket says 172.21.15.37:56978) 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da8a00640 1 -- 172.21.15.37:0/312809105 learned_addr learned my addr 172.21.15.37:0/312809105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da9400640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c8789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 -- 172.21.15.37:0/312809105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 msgr2=0x55b80c87b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 -- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 msgr2=0x55b80c8789a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c8789a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 -- 172.21.15.37:0/312809105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b80c613f70 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da8a00640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da9400640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c8789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.237+0000 7f5da3e00640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c878460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5d900cd210 tx=0x7f5d9000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d9e4fafd2b673a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5d8e000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5d8e000640 1 -- 172.21.15.37:0/312809105 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900cd6a0 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9c02ce00 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5d8e000640 1 -- 172.21.15.37:0/312809105 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5d9009b5b0 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5d8e000640 1 -- 172.21.15.37:0/312809105 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d900a48c0 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5dae1e2740 10 client.27689 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b80c4ab170 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5d8e000640 1 -- 172.21.15.37:0/312809105 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c02ce00 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.238+0000 7f5d8e000640 1 -- 172.21.15.37:0/312809105 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5d9009ae20 con 0x55b80c872bb0 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5dae1e2740 10 client.27689 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5dae1e2740 10 client.27689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5dae1e2740 20 client.27689 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c88b7b0 0x55b80c88dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5dae1e2740 4 client.27689 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b80c7e43f0 con 0x55b80c88b7b0 2024-03-21T09:21:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.239+0000 7f5da8a00640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c88b7b0 0x55b80c88dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.240+0000 7f5da8a00640 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c88b7b0 0x55b80c88dba0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6ade871cc40772 server_cookie=45afc57696dc05d5 in_seq=0 out_seq=0 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.240+0000 7f5d8e000640 10 client.27689 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.240+0000 7f5d8e000640 1 -- 172.21.15.37:0/312809105 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b80c7e43f0 con 0x55b80c88b7b0 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.240+0000 7f5d8e000640 10 client.27689 handle_command_reply: tid=0 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.240+0000 7f5dae1e2740 1 client.27689 shutdown 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 msgr2=0x55b80c878460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c878460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5d900cd210 tx=0x7f5d9000cf60 comp rx=0 tx=0).stop 2024-03-21T09:21:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 shutdown_connections 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c87b790 0x55b80c8789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c88b7b0 0x55b80c88dba0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872bb0 0x55b80c878460 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/312809105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5d8e000640 0 client.27689 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 >> 172.21.15.37:0/312809105 conn(0x55b80c86c9c0 msgr2=0x55b80c86e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 shutdown_connections 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 1 -- 172.21.15.37:0/312809105 wait complete. 2024-03-21T09:21:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.241+0000 7f5dae1e2740 20 client.27689 trim_cache size 0 max 16384 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 Processor -- start 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 -- start start 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c87b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c87b790 0x55b80c886db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41b73b0 con 0x55b80c872bb0 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41b4eb0 con 0x55b80c87b790 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41b2f70 con 0x55b80c872290 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56980/0 (socket says 172.21.15.37:56980) 2024-03-21T09:21:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5da8a00640 1 -- 172.21.15.37:0/27942550 learned_addr learned my addr 172.21.15.37:0/27942550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5da9400640 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c87b790 0x55b80c886db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.259+0000 7f5da3e00640 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c87b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5da8a00640 1 -- 172.21.15.37:0/27942550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c87b790 msgr2=0x55b80c886db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5da8a00640 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c87b790 0x55b80c886db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5da8a00640 1 -- 172.21.15.37:0/27942550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 msgr2=0x55b80c87b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5da8a00640 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c87b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5da8a00640 1 -- 172.21.15.37:0/27942550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b80c871c60 con 0x55b80c872290 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5da8a00640 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87ab90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0044f0 tx=0x7f5d9c004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2e82cabe60c0b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5d99200640 1 -- 172.21.15.37:0/27942550 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c050070 con 0x55b80c872290 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5d99200640 1 -- 172.21.15.37:0/27942550 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c033520 con 0x55b80c872290 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.260+0000 7f5d99200640 1 -- 172.21.15.37:0/27942550 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c043040 con 0x55b80c872290 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 -- 172.21.15.37:0/27942550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 msgr2=0x55b80c87ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87ab90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0044f0 tx=0x7f5d9c004520 comp rx=0 tx=0).stop 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 -- 172.21.15.37:0/27942550 shutdown_connections 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c87b790 0x55b80c886db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c87b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/27942550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c872290 0x55b80c87ab90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 -- 172.21.15.37:0/27942550 >> 172.21.15.37:0/27942550 conn(0x55b80c86c9c0 msgr2=0x55b80c86e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 -- 172.21.15.37:0/27942550 shutdown_connections 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.261+0000 7f5dae1e2740 1 -- 172.21.15.37:0/27942550 wait complete. 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 -- start start 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 Processor -- start 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 0x55b80c880ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c880ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c87b790 0x55b80c8e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41b3570 con 0x55b80c872bb0 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41b4850 con 0x55b80c872290 2024-03-21T09:21:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41b3b90 con 0x55b80c87b790 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5dae1e2740 10 client.0 _ll_register_callbacks cb 0x7f5d90098270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 0x55b80c880ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c880ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5da8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 0x55b80c880ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44644/0 (socket says 172.21.15.37:44644) 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c880ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35046/0 (socket says 172.21.15.37:35046) 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5da8a00640 1 -- 172.21.15.37:0/4117212175 learned_addr learned my addr 172.21.15.37:0/4117212175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.262+0000 7f5da9400640 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c87b790 0x55b80c8e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5da8a00640 1 -- 172.21.15.37:0/4117212175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c87b790 msgr2=0x55b80c8e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5da8a00640 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c87b790 0x55b80c8e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5da8a00640 1 -- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 msgr2=0x55b80c880ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5da8a00640 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c880ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5da8a00640 1 -- 172.21.15.37:0/4117212175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b80c8e7580 con 0x55b80c872290 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5da8a00640 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 0x55b80c880ab0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c036040 tx=0x7f5d9c005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec17a5345897b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5d8e000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:26.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5d8e000640 1 -- 172.21.15.37:0/4117212175 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c043040 con 0x55b80c872290 2024-03-21T09:21:26.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5d8e000640 1 -- 172.21.15.37:0/4117212175 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5d9c0338d0 con 0x55b80c872290 2024-03-21T09:21:26.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.263+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9c033520 con 0x55b80c872290 2024-03-21T09:21:26.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5d8e000640 1 -- 172.21.15.37:0/4117212175 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9c062050 con 0x55b80c872290 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 10 client.27695 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b80c8e8770 con 0x55b80c872290 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5d8e000640 1 -- 172.21.15.37:0/4117212175 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c033520 con 0x55b80c872290 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5d8e000640 1 -- 172.21.15.37:0/4117212175 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5d9c0525a0 con 0x55b80c872290 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 10 client.27695 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 10 client.27695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 20 client.27695 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c8e8af0 0x55b80c8eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 4 client.27695 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b80c7de740 con 0x55b80c8e8af0 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.264+0000 7f5da3e00640 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c8e8af0 0x55b80c8eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.265+0000 7f5da3e00640 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c8e8af0 0x55b80c8eaee0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554ad7a773fe6b01 server_cookie=7876aca28ca6d248 in_seq=0 out_seq=0 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.265+0000 7f5d8e000640 10 client.27695 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5d8e000640 1 -- 172.21.15.37:0/4117212175 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b80c7de740 con 0x55b80c8e8af0 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5d8e000640 10 client.27695 handle_command_reply: tid=0 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 client.27695 shutdown 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 msgr2=0x55b80c880ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 0x55b80c880ab0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c036040 tx=0x7f5d9c005e30 comp rx=0 tx=0).stop 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 shutdown_connections 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b80c87b790 0x55b80c8e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b80c872bb0 0x55b80c880ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55b80c8e8af0 0x55b80c8eaee0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 --2- 172.21.15.37:0/4117212175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b80c872290 0x55b80c880ab0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 >> 172.21.15.37:0/4117212175 conn(0x55b80c86c9c0 msgr2=0x55b80c892460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5d8e000640 0 client.27695 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 shutdown_connections 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.267+0000 7f5dae1e2740 1 -- 172.21.15.37:0/4117212175 wait complete. 2024-03-21T09:21:26.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.268+0000 7f5dae1e2740 20 client.27695 trim_cache size 0 max 16384 2024-03-21T09:21:26.268 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:21:26.268 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:21:26.268 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "99e3f6a5-43d1-4503-8e11-a24679e4b159", 2024-03-21T09:21:26.268 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:21:26.268 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.268+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d880a3110 msgr2=0x7f5d880a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.268+0000 7f5da9e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d880a3110 0x7f5d880a55d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005a50 tx=0x7f5d94004730 comp rx=0 tx=0).stop 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.268+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 msgr2=0x7f5da419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.268+0000 7f5da9e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 0x7f5da419e990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e9f0 tx=0x7f5d9000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 shutdown_connections 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da41124b0 0x7f5da41a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d880a3110 0x7f5d880a55d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da4074360 0x7f5da419e990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 --2- 172.21.15.37:0/2901397875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da4073a40 0x7f5da419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 >> 172.21.15.37:0/2901397875 conn(0x7f5da40fcfb0 msgr2=0x7f5da40fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 shutdown_connections 2024-03-21T09:21:26.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.269+0000 7f5da9e00640 1 -- 172.21.15.37:0/2901397875 wait complete. 2024-03-21T09:21:26.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:21:26.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 -- 172.21.15.37:0/384535736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd0010ba50 msgr2=0x7efd0010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 --2- 172.21.15.37:0/384535736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd0010ba50 0x7efd0010ded0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009860 tx=0x7efcf802bc00 comp rx=0 tx=0).stop 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 -- 172.21.15.37:0/384535736 shutdown_connections 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 --2- 172.21.15.37:0/384535736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd0010ba50 0x7efd0010ded0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 --2- 172.21.15.37:0/384535736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd00102f50 0x7efd0010b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 --2- 172.21.15.37:0/384535736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd00102630 0x7efd00102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 -- 172.21.15.37:0/384535736 >> 172.21.15.37:0/384535736 conn(0x7efd00077130 msgr2=0x7efd000fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 -- 172.21.15.37:0/384535736 shutdown_connections 2024-03-21T09:21:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.523+0000 7efd05800640 1 -- 172.21.15.37:0/384535736 wait complete. 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.524+0000 7efd05800640 1 Processor -- start 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.524+0000 7efd05800640 1 -- start start 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd05800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 0x7efd00100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd05800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd00102f50 0x7efd001007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd05800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0010ba50 0x7efd00100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd05800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001117f0 con 0x7efd0010ba50 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd05800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd00111670 con 0x7efd00102f50 2024-03-21T09:21:26.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd05800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd00111970 con 0x7efd00102630 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 0x7efd00100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcf7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd00102f50 0x7efd001007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 0x7efd00100280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57006/0 (socket says 172.21.15.37:57006) 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 -- 172.21.15.37:0/2447553722 learned_addr learned my addr 172.21.15.37:0/2447553722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efd04e00640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0010ba50 0x7efd00100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 -- 172.21.15.37:0/2447553722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd00102f50 msgr2=0x7efd001007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd00102f50 0x7efd001007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 -- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0010ba50 msgr2=0x7efd00100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.525+0000 7efcffe00640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0010ba50 0x7efd00100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efcffe00640 1 -- 172.21.15.37:0/2447553722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd001a61b0 con 0x7efd00102630 2024-03-21T09:21:26.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efcffe00640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 0x7efd00100280 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efcf000efb0 tx=0x7efcf000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efcfd600640 1 -- 172.21.15.37:0/2447553722 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf001b070 con 0x7efd00102630 2024-03-21T09:21:26.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efcfd600640 1 -- 172.21.15.37:0/2447553722 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf000ba50 con 0x7efd00102630 2024-03-21T09:21:26.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd001a64a0 con 0x7efd00102630 2024-03-21T09:21:26.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efcfd600640 1 -- 172.21.15.37:0/2447553722 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0016c00 con 0x7efd00102630 2024-03-21T09:21:26.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.526+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd001a69f0 con 0x7efd00102630 2024-03-21T09:21:26.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.529+0000 7efcfd600640 1 -- 172.21.15.37:0/2447553722 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7efcf001e070 con 0x7efd00102630 2024-03-21T09:21:26.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.531+0000 7efcfd600640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7efce40a3090 0x7efce40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.531+0000 7efcfd600640 1 -- 172.21.15.37:0/2447553722 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7efcf00cfc30 con 0x7efd00102630 2024-03-21T09:21:26.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.531+0000 7efcf7400640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7efce40a3090 0x7efce40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.531+0000 7efd09c30740 1 Processor -- start 2024-03-21T09:21:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.531+0000 7efd09c30740 1 -- start start 2024-03-21T09:21:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcf7400640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7efce40a3090 0x7efce40a5550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efd00101d10 tx=0x7efce8005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd09c30740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 0x55aef5195b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd09c30740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5196570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd09c30740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef519f460 0x55aef51a1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd09c30740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef50fa250 con 0x55aef519f460 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd09c30740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef50a2a30 con 0x55aef51960f0 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd09c30740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef50f62f0 con 0x55aef5196dd0 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 0x55aef5195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcf7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 0x55aef5195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57010/0 (socket says 172.21.15.37:57010) 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 -- 172.21.15.37:0/1855567766 learned_addr learned my addr 172.21.15.37:0/1855567766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd04e00640 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef519f460 0x55aef51a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 -- 172.21.15.37:0/1855567766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 msgr2=0x55aef5196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 -- 172.21.15.37:0/1855567766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef519f460 msgr2=0x55aef51a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef519f460 0x55aef51a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efcffe00640 1 -- 172.21.15.37:0/1855567766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef508d690 con 0x55aef5196dd0 2024-03-21T09:21:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.532+0000 7efd04e00640 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef519f460 0x55aef51a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.533+0000 7efcffe00640 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 0x55aef5195b20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efcf0098640 tx=0x7efcf009fe50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de9f5b4fe5e5e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.533+0000 7efcf5600640 1 -- 172.21.15.37:0/1855567766 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00cd1a0 con 0x55aef5196dd0 2024-03-21T09:21:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.533+0000 7efcf5600640 1 -- 172.21.15.37:0/1855567766 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf00982e0 con 0x55aef5196dd0 2024-03-21T09:21:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.533+0000 7efcf5600640 1 -- 172.21.15.37:0/1855567766 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00979c0 con 0x55aef5196dd0 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.533+0000 7efd09c30740 1 -- 172.21.15.37:0/1855567766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 msgr2=0x55aef5195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 0x55aef5195b20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efcf0098640 tx=0x7efcf009fe50 comp rx=0 tx=0).stop 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 -- 172.21.15.37:0/1855567766 shutdown_connections 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef519f460 0x55aef51a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 --2- 172.21.15.37:0/1855567766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef5196dd0 0x55aef5195b20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 -- 172.21.15.37:0/1855567766 >> 172.21.15.37:0/1855567766 conn(0x55aef515d200 msgr2=0x55aef5190b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 -- 172.21.15.37:0/1855567766 shutdown_connections 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 -- 172.21.15.37:0/1855567766 wait complete. 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 -- start start 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.534+0000 7efd09c30740 1 Processor -- start 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef519c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef51a22d0 con 0x55aef5196dd0 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef50b5e80 con 0x55aef51960f0 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef51a2450 con 0x55aef519f460 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd09c30740 10 client.0 _ll_register_callbacks cb 0x55aef50c8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd04e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef519c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efcffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd04e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef519c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57016/0 (socket says 172.21.15.37:57016) 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efd04e00640 1 -- 172.21.15.37:0/797767592 learned_addr learned my addr 172.21.15.37:0/797767592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.535+0000 7efcf7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd04e00640 1 -- 172.21.15.37:0/797767592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 msgr2=0x55aef519eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd04e00640 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd04e00640 1 -- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 msgr2=0x55aef519bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd04e00640 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd04e00640 1 -- 172.21.15.37:0/797767592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aef4ed87d0 con 0x55aef519f460 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efcf7400640 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd04e00640 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef519c410 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efcf8037040 tx=0x7efcf8005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4824742182e956 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efcee000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efcee000640 1 -- 172.21.15.37:0/797767592 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8044070 con 0x55aef519f460 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efcee000640 1 -- 172.21.15.37:0/797767592 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7efcf8034440 con 0x55aef519f460 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcf00982e0 con 0x55aef519f460 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efcee000640 1 -- 172.21.15.37:0/797767592 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf804f050 con 0x55aef519f460 2024-03-21T09:21:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd09c30740 10 client.27733 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.536+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aef4e0e3f0 con 0x55aef519f460 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efcee000640 1 -- 172.21.15.37:0/797767592 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf00982e0 con 0x55aef519f460 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efcee000640 1 -- 172.21.15.37:0/797767592 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7efcf80035f0 con 0x55aef519f460 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efd09c30740 10 client.27733 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efd09c30740 10 client.27733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efd09c30740 20 client.27733 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efd09c30740 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef51af310 0x55aef51b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efd09c30740 4 client.27733 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef5107790 con 0x55aef51af310 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.537+0000 7efcffe00640 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef51af310 0x55aef51b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.538+0000 7efcffe00640 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef51af310 0x55aef51b1730 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46644aefab578688 server_cookie=bac8fc5bd73ed361 in_seq=0 out_seq=0 2024-03-21T09:21:26.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.538+0000 7efcee000640 10 client.27733 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.538+0000 7efcee000640 1 -- 172.21.15.37:0/797767592 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aef5107790 con 0x55aef51af310 2024-03-21T09:21:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.538+0000 7efcee000640 10 client.27733 handle_command_reply: tid=0 2024-03-21T09:21:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 client.27733 shutdown 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 msgr2=0x55aef519c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef519c410 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efcf8037040 tx=0x7efcf8005da0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 shutdown_connections 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef519c410 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef51af310 0x55aef51b1730 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 --2- 172.21.15.37:0/797767592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efcee000640 0 client.27733 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 >> 172.21.15.37:0/797767592 conn(0x55aef515d200 msgr2=0x55aef5192890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 shutdown_connections 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 1 -- 172.21.15.37:0/797767592 wait complete. 2024-03-21T09:21:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.539+0000 7efd09c30740 20 client.27733 trim_cache size 0 max 16384 2024-03-21T09:21:26.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 Processor -- start 2024-03-21T09:21:26.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 -- start start 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef51a0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001b3010 con 0x55aef5196dd0 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001b0b10 con 0x55aef51960f0 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efd09c30740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001aebd0 con 0x55aef519f460 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.561+0000 7efcffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44678/0 (socket says 172.21.15.37:44678) 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efd04e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef51a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 -- 172.21.15.37:0/3890040576 learned_addr learned my addr 172.21.15.37:0/3890040576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcf7400640 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 -- 172.21.15.37:0/3890040576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 msgr2=0x55aef51a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef51a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 -- 172.21.15.37:0/3890040576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 msgr2=0x55aef519ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 -- 172.21.15.37:0/3890040576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef4dce0f0 con 0x55aef51960f0 2024-03-21T09:21:26.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcffe00640 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519e590 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efcf009bbb0 tx=0x7efcf009b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=613ee9124d754132 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.562+0000 7efcf4c00640 1 -- 172.21.15.37:0/3890040576 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf009bc70 con 0x55aef51960f0 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efcf4c00640 1 -- 172.21.15.37:0/3890040576 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf000bc00 con 0x55aef51960f0 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efcf4c00640 1 -- 172.21.15.37:0/3890040576 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00979c0 con 0x55aef51960f0 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 -- 172.21.15.37:0/3890040576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 msgr2=0x55aef519e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519e590 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efcf009bbb0 tx=0x7efcf009b8e0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 -- 172.21.15.37:0/3890040576 shutdown_connections 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef51a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef519ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 --2- 172.21.15.37:0/3890040576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef519e590 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 -- 172.21.15.37:0/3890040576 >> 172.21.15.37:0/3890040576 conn(0x55aef515d200 msgr2=0x55aef519a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 -- 172.21.15.37:0/3890040576 shutdown_connections 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.563+0000 7efd09c30740 1 -- 172.21.15.37:0/3890040576 wait complete. 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 -- start start 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 Processor -- start 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5203e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef52043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef5198d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001af1d0 con 0x55aef5196dd0 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001b04b0 con 0x55aef51960f0 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.564+0000 7efd09c30740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001af7f0 con 0x55aef519f460 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efd09c30740 10 client.0 _ll_register_callbacks cb 0x7efcf0009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5203e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5203e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44694/0 (socket says 172.21.15.37:44694) 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 -- 172.21.15.37:0/3774871144 learned_addr learned my addr 172.21.15.37:0/3774871144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcf7400640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef52043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efd04e00640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef5198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 -- 172.21.15.37:0/3774871144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 msgr2=0x55aef5198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef5198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 -- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 msgr2=0x55aef52043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef52043d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 -- 172.21.15.37:0/3774871144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aef5199440 con 0x55aef51960f0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcffe00640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5203e90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efcf0099f20 tx=0x7efcf00985b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af1785b6263b8966 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.565+0000 7efcee000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efcee000640 1 -- 172.21.15.37:0/3774871144 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00982e0 con 0x55aef51960f0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efcee000640 1 -- 172.21.15.37:0/3774871144 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7efcf009b0c0 con 0x55aef51960f0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcf000bc00 con 0x55aef51960f0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efcee000640 1 -- 172.21.15.37:0/3774871144 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf000bdb0 con 0x55aef51960f0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 10 client.27713 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aef520b650 con 0x55aef51960f0 2024-03-21T09:21:26.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efcee000640 1 -- 172.21.15.37:0/3774871144 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcf000bc00 con 0x55aef51960f0 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efcee000640 1 -- 172.21.15.37:0/3774871144 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7efcf000b040 con 0x55aef51960f0 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 10 client.27713 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 10 client.27713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 20 client.27713 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef520b870 0x55aef520dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 4 client.27713 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.566+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aef5101ae0 con 0x55aef520b870 2024-03-21T09:21:26.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.567+0000 7efcf7400640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef520b870 0x55aef520dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.567+0000 7efcf7400640 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef520b870 0x55aef520dc60 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7afd8df54cf16fde server_cookie=d3b13235dded8f3e in_seq=0 out_seq=0 2024-03-21T09:21:26.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.567+0000 7efcee000640 10 client.27713 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.567+0000 7efcee000640 1 -- 172.21.15.37:0/3774871144 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aef5101ae0 con 0x55aef520b870 2024-03-21T09:21:26.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.567+0000 7efcee000640 10 client.27713 handle_command_reply: tid=0 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.567+0000 7efd09c30740 1 client.27713 shutdown 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 msgr2=0x55aef5203e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5203e90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efcf0099f20 tx=0x7efcf00985b0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 shutdown_connections 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef519f460 0x55aef5198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aef5196dd0 0x55aef52043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55aef520b870 0x55aef520dc60 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 --2- 172.21.15.37:0/3774871144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aef51960f0 0x55aef5203e90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 >> 172.21.15.37:0/3774871144 conn(0x55aef515d200 msgr2=0x55aef5191e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efcee000640 0 client.27713 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 shutdown_connections 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 1 -- 172.21.15.37:0/3774871144 wait complete. 2024-03-21T09:21:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.568+0000 7efd09c30740 20 client.27713 trim_cache size 0 max 16384 2024-03-21T09:21:26.570 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:21:26.570 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:21:26.570 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:21:26.570 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:21:26.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7efce40a3090 msgr2=0x7efce40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7efce40a3090 0x7efce40a5550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efd00101d10 tx=0x7efce8005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 msgr2=0x7efd00100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 0x7efd00100280 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efcf000efb0 tx=0x7efcf000c800 comp rx=0 tx=0).stop 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 shutdown_connections 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0010ba50 0x7efd00100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7efce40a3090 0x7efce40a5550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd00102f50 0x7efd001007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 --2- 172.21.15.37:0/2447553722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd00102630 0x7efd00100280 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 >> 172.21.15.37:0/2447553722 conn(0x7efd00077130 msgr2=0x7efd000fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 shutdown_connections 2024-03-21T09:21:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.569+0000 7efd05800640 1 -- 172.21.15.37:0/2447553722 wait complete. 2024-03-21T09:21:26.589 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:21:26.589 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:21:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 -- 172.21.15.37:0/562199508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100d60 msgr2=0x7f3fd010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 --2- 172.21.15.37:0/562199508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100d60 0x7f3fd010dab0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009900 tx=0x7f3fbc02bc30 comp rx=0 tx=0).stop 2024-03-21T09:21:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 -- 172.21.15.37:0/562199508 shutdown_connections 2024-03-21T09:21:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 --2- 172.21.15.37:0/562199508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd010dff0 0x7f3fd01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 --2- 172.21.15.37:0/562199508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100d60 0x7f3fd010dab0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 --2- 172.21.15.37:0/562199508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100440 0x7f3fd0100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 -- 172.21.15.37:0/562199508 >> 172.21.15.37:0/562199508 conn(0x7f3fd0077720 msgr2=0x7f3fd0077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 -- 172.21.15.37:0/562199508 shutdown_connections 2024-03-21T09:21:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.807+0000 7f3fd7400640 1 -- 172.21.15.37:0/562199508 wait complete. 2024-03-21T09:21:26.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 Processor -- start 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 -- start start 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd0100440 0x7f3fd0102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 0x7f3fd0102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd010dff0 0x7f3fd01031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0113ca0 con 0x7f3fd0100440 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0113b20 con 0x7f3fd0100d60 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.809+0000 7f3fd7400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0113e20 con 0x7f3fd010dff0 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 0x7f3fd0102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 0x7f3fd0102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44708/0 (socket says 172.21.15.37:44708) 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 -- 172.21.15.37:0/1818053711 learned_addr learned my addr 172.21.15.37:0/1818053711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fd4a00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd0100440 0x7f3fd0102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fd5400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd010dff0 0x7f3fd01031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 -- 172.21.15.37:0/1818053711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd010dff0 msgr2=0x7f3fd01031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd010dff0 0x7f3fd01031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 -- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd0100440 msgr2=0x7f3fd0102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd0100440 0x7f3fd0102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 -- 172.21.15.37:0/1818053711 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd01aa800 con 0x7f3fd0100d60 2024-03-21T09:21:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fd4a00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd0100440 0x7f3fd0102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.810+0000 7f3fcfe00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 0x7f3fd0102c60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc036040 tx=0x7f3fbc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.811+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc043020 con 0x7f3fd0100d60 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.811+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fbc0054e0 con 0x7f3fd0100d60 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.811+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc003b50 con 0x7f3fd0100d60 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.811+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd01aaa90 con 0x7f3fd0100d60 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.811+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd01ab030 con 0x7f3fd0100d60 2024-03-21T09:21:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.812+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd00691b0 con 0x7f3fd0100d60 2024-03-21T09:21:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.814+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3fbc046020 con 0x7f3fd0100d60 2024-03-21T09:21:26.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.815+0000 7f3fcd600640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3fb80a3110 0x7f3fb80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:26.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.815+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3fbc0f15d0 con 0x7f3fd0100d60 2024-03-21T09:21:26.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.815+0000 7f3fd4a00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3fb80a3110 0x7f3fb80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.816+0000 7f3fd4a00640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3fb80a3110 0x7f3fb80a55d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009c20 tx=0x7f3fc4005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:26.819+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3fbc04d070 con 0x7f3fd0100d60 2024-03-21T09:21:26.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:26 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:21:26.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:26 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:26.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:26 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:21:26.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:26 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:26.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:26 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:21:26.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:26 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:21:26.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:26 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:26.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:26 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:21:26.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:26 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:26.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:26 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:21:27.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.083+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd0109180 con 0x7f3fd0100d60 2024-03-21T09:21:27.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.084+0000 7f3fcd600640 1 -- 172.21.15.37:0/1818053711 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f3fbc0bdd70 con 0x7f3fd0100d60 2024-03-21T09:21:27.081 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:21:27.081 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:21:27.082 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3fb80a3110 msgr2=0x7f3fb80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3fb80a3110 0x7f3fb80a55d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009c20 tx=0x7f3fc4005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 msgr2=0x7f3fd0102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 0x7f3fd0102c60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc036040 tx=0x7f3fbc0042f0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 shutdown_connections 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd010dff0 0x7f3fd01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3fb80a3110 0x7f3fb80a55d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd0100d60 0x7f3fd0102c60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 --2- 172.21.15.37:0/1818053711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd0100440 0x7f3fd0102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.086+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 >> 172.21.15.37:0/1818053711 conn(0x7f3fd0077720 msgr2=0x7f3fd010f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.087+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 shutdown_connections 2024-03-21T09:21:27.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.087+0000 7f3fd7400640 1 -- 172.21.15.37:0/1818053711 wait complete. 2024-03-21T09:21:27.105 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:21:27.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:21:27.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:26 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:21:27.107 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:26 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:27.107 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:26 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:21:27.107 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:26 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:27.107 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:26 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 -- 172.21.15.37:0/1909404515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 msgr2=0x7f87c0114a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 --2- 172.21.15.37:0/1909404515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c0114a30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009ca0 tx=0x7f87b802c190 comp rx=0 tx=0).stop 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 -- 172.21.15.37:0/1909404515 shutdown_connections 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 --2- 172.21.15.37:0/1909404515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c0114a30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 --2- 172.21.15.37:0/1909404515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 0x7f87c0112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 --2- 172.21.15.37:0/1909404515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c0073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 -- 172.21.15.37:0/1909404515 >> 172.21.15.37:0/1909404515 conn(0x7f87c00fd140 msgr2=0x7f87c00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 -- 172.21.15.37:0/1909404515 shutdown_connections 2024-03-21T09:21:27.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.321+0000 7f87c6000640 1 -- 172.21.15.37:0/1909404515 wait complete. 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.322+0000 7f87c6000640 1 Processor -- start 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.322+0000 7f87c6000640 1 -- start start 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87c6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87c6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 0x7f87c019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87c6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c01a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87c6000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0118040 con 0x7f87c0073a40 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87c6000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0117ec0 con 0x7f87c0074360 2024-03-21T09:21:27.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87c6000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01181c0 con 0x7f87c0112640 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 0x7f87c019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35150/0 (socket says 172.21.15.37:35150) 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87bf400640 1 -- 172.21.15.37:0/4202149319 learned_addr learned my addr 172.21.15.37:0/4202149319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87bfe00640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c01a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87b7e00640 1 -- 172.21.15.37:0/4202149319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 msgr2=0x7f87c01a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87b7e00640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c01a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87b7e00640 1 -- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 msgr2=0x7f87c019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87b7e00640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87b7e00640 1 -- 172.21.15.37:0/4202149319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c01a3550 con 0x7f87c0074360 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87bfe00640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c01a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.323+0000 7f87bf400640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:27.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.324+0000 7f87b7e00640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 0x7f87c019eaa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.324+0000 7f87bcc00640 1 -- 172.21.15.37:0/4202149319 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800cd80 con 0x7f87c0074360 2024-03-21T09:21:27.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.324+0000 7f87bcc00640 1 -- 172.21.15.37:0/4202149319 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87a80035f0 con 0x7f87c0074360 2024-03-21T09:21:27.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.324+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c01a3840 con 0x7f87c0074360 2024-03-21T09:21:27.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.324+0000 7f87bcc00640 1 -- 172.21.15.37:0/4202149319 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8010690 con 0x7f87c0074360 2024-03-21T09:21:27.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.324+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c01ab0d0 con 0x7f87c0074360 2024-03-21T09:21:27.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.327+0000 7f87bcc00640 1 -- 172.21.15.37:0/4202149319 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f87a8003b30 con 0x7f87c0074360 2024-03-21T09:21:27.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.327+0000 7f87c8bf3740 1 Processor -- start 2024-03-21T09:21:27.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 -- start start 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a73aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a743630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a746050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7a69e2c0 con 0x55cb7a743c00 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7a646c90 con 0x55cb7a73b080 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7a69a360 con 0x55cb7a73a6d0 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a73aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a73aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57064/0 (socket says 172.21.15.37:57064) 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87bfe00640 1 -- 172.21.15.37:0/944008491 learned_addr learned my addr 172.21.15.37:0/944008491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.328+0000 7f87bf400640 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a743630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87b7e00640 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a746050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bfe00640 1 -- 172.21.15.37:0/944008491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 msgr2=0x55cb7a743630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bfe00640 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a743630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bfe00640 1 -- 172.21.15.37:0/944008491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 msgr2=0x55cb7a746050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bfe00640 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a746050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bfe00640 1 -- 172.21.15.37:0/944008491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7a66b080 con 0x55cb7a73a6d0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bcc00640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f87a40a3090 0x7f87a40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bcc00640 1 -- 172.21.15.37:0/4202149319 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f87a80cfe70 con 0x7f87c0074360 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bfe00640 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a73aab0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f87c006ae00 tx=0x7f87b8005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce3b4ecb767a4d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.329+0000 7f87bf400640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f87a40a3090 0x7f87a40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.330+0000 7f87afe00640 1 -- 172.21.15.37:0/944008491 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8043070 con 0x55cb7a73a6d0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.330+0000 7f87afe00640 1 -- 172.21.15.37:0/944008491 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b8003e50 con 0x55cb7a73a6d0 2024-03-21T09:21:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.330+0000 7f87afe00640 1 -- 172.21.15.37:0/944008491 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8039040 con 0x55cb7a73a6d0 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.330+0000 7f87c8bf3740 1 -- 172.21.15.37:0/944008491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 msgr2=0x55cb7a73aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.330+0000 7f87bf400640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f87a40a3090 0x7f87a40a5550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55cb7a735e10 tx=0x7f87b0005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.330+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a73aab0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f87c006ae00 tx=0x7f87b8005f80 comp rx=0 tx=0).stop 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 -- 172.21.15.37:0/944008491 shutdown_connections 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a746050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a743630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/944008491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a73aab0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 -- 172.21.15.37:0/944008491 >> 172.21.15.37:0/944008491 conn(0x55cb7a734da0 msgr2=0x55cb7a735180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 -- 172.21.15.37:0/944008491 shutdown_connections 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 -- 172.21.15.37:0/944008491 wait complete. 2024-03-21T09:21:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 -- start start 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.331+0000 7f87c8bf3740 1 Processor -- start 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a7434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a740870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a740dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7a746b10 con 0x55cb7a73a6d0 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7a65a0d0 con 0x55cb7a73b080 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7a746c90 con 0x55cb7a743c00 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87c8bf3740 10 client.0 _ll_register_callbacks cb 0x55cb7a65ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a740870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a740dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a7434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a740dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57066/0 (socket says 172.21.15.37:57066) 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.332+0000 7f87bfe00640 1 -- 172.21.15.37:0/946715824 learned_addr learned my addr 172.21.15.37:0/946715824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87b7e00640 1 -- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 msgr2=0x55cb7a740dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87b7e00640 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a740dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87b7e00640 1 -- 172.21.15.37:0/946715824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 msgr2=0x55cb7a7434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87b7e00640 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a7434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87b7e00640 1 -- 172.21.15.37:0/946715824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb7a3b23f0 con 0x55cb7a73b080 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87bfe00640 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a740dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87b7e00640 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a740870 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f87a80cd160 tx=0x7f87a8097a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ff74fe74b968a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87acc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87acc00640 1 -- 172.21.15.37:0/946715824 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a809b760 con 0x55cb7a73b080 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87acc00640 1 -- 172.21.15.37:0/946715824 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f87a8098470 con 0x55cb7a73b080 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.333+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87b8003e50 con 0x55cb7a73b080 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87acc00640 1 -- 172.21.15.37:0/946715824 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a8009450 con 0x55cb7a73b080 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 10 client.27731 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:27.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cb7a47c7d0 con 0x55cb7a73b080 2024-03-21T09:21:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87acc00640 1 -- 172.21.15.37:0/946715824 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87a80b7030 con 0x55cb7a73b080 2024-03-21T09:21:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87acc00640 1 -- 172.21.15.37:0/946715824 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f87a80cd3e0 con 0x55cb7a73b080 2024-03-21T09:21:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 10 client.27731 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 10 client.27731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 20 client.27731 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7539f0 0x55cb7a755e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 4 client.27731 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb7a6ab800 con 0x55cb7a7539f0 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.334+0000 7f87bf400640 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7539f0 0x55cb7a755e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.335+0000 7f87bf400640 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7539f0 0x55cb7a755e30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf18be37068085d1 server_cookie=f4ae90828e6c5a57 in_seq=0 out_seq=0 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.335+0000 7f87acc00640 10 client.27731 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87acc00640 1 -- 172.21.15.37:0/946715824 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cb7a6ab800 con 0x55cb7a7539f0 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87acc00640 10 client.27731 handle_command_reply: tid=0 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 client.27731 shutdown 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 msgr2=0x55cb7a740870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a740870 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f87a80cd160 tx=0x7f87a8097a70 comp rx=0 tx=0).stop 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 shutdown_connections 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a740dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a740870 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7539f0 0x55cb7a755e30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/946715824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a7434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 >> 172.21.15.37:0/946715824 conn(0x55cb7a734da0 msgr2=0x55cb7a736af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87acc00640 0 client.27731 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 shutdown_connections 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 1 -- 172.21.15.37:0/946715824 wait complete. 2024-03-21T09:21:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.336+0000 7f87c8bf3740 20 client.27731 trim_cache size 0 max 16384 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 Processor -- start 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 -- start start 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a742fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a7434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a73dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01b76a0 con 0x55cb7a743c00 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01b51a0 con 0x55cb7a73b080 2024-03-21T09:21:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.362+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01b3260 con 0x55cb7a73a6d0 2024-03-21T09:21:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a742fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a742fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57070/0 (socket says 172.21.15.37:57070) 2024-03-21T09:21:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 -- 172.21.15.37:0/4051392267 learned_addr learned my addr 172.21.15.37:0/4051392267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a7434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bfe00640 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a73dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 -- 172.21.15.37:0/4051392267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 msgr2=0x55cb7a7434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a7434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 -- 172.21.15.37:0/4051392267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 msgr2=0x55cb7a73dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a73dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 -- 172.21.15.37:0/4051392267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7a739760 con 0x55cb7a73a6d0 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87bf400640 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a742fa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55cb7a7566a0 tx=0x7f87b001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9014379b7e4f04c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87af400640 1 -- 172.21.15.37:0/4051392267 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000fe00 con 0x55cb7a73a6d0 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87af400640 1 -- 172.21.15.37:0/4051392267 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b0005500 con 0x55cb7a73a6d0 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.363+0000 7f87af400640 1 -- 172.21.15.37:0/4051392267 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b002b680 con 0x55cb7a73a6d0 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 -- 172.21.15.37:0/4051392267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 msgr2=0x55cb7a742fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a742fa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55cb7a7566a0 tx=0x7f87b001a000 comp rx=0 tx=0).stop 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 -- 172.21.15.37:0/4051392267 shutdown_connections 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a743c00 0x55cb7a73dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a7434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/4051392267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a73a6d0 0x55cb7a742fa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 -- 172.21.15.37:0/4051392267 >> 172.21.15.37:0/4051392267 conn(0x55cb7a734da0 msgr2=0x55cb7a73f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 -- 172.21.15.37:0/4051392267 shutdown_connections 2024-03-21T09:21:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.364+0000 7f87c8bf3740 1 -- 172.21.15.37:0/4051392267 wait complete. 2024-03-21T09:21:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 -- start start 2024-03-21T09:21:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 Processor -- start 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a749aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a749fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a7ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01b3860 con 0x55cb7a73a6d0 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01b4b40 con 0x55cb7a73b080 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01b3e80 con 0x55cb7a743c00 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.365+0000 7f87c8bf3740 10 client.0 _ll_register_callbacks cb 0x7f87a8005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a749fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a7ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a749fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44748/0 (socket says 172.21.15.37:44748) 2024-03-21T09:21:27.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 -- 172.21.15.37:0/1356235832 learned_addr learned my addr 172.21.15.37:0/1356235832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87bf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a749aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 -- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 msgr2=0x55cb7a7ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a7ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 -- 172.21.15.37:0/1356235832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 msgr2=0x55cb7a749aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a749aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 -- 172.21.15.37:0/1356235832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb7a7ae840 con 0x55cb7a73b080 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87b7e00640 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a749fe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c6a0 tx=0x7f87a80a4970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e299319be4c9d2c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87acc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87acc00640 1 -- 172.21.15.37:0/1356235832 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a809b760 con 0x55cb7a73b080 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87acc00640 1 -- 172.21.15.37:0/1356235832 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f87a809b250 con 0x55cb7a73b080 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.366+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87b0005500 con 0x55cb7a73b080 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87acc00640 1 -- 172.21.15.37:0/1356235832 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a809bbd0 con 0x55cb7a73b080 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 10 client.27737 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cb7a7afb40 con 0x55cb7a73b080 2024-03-21T09:21:27.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87acc00640 1 -- 172.21.15.37:0/1356235832 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b0005500 con 0x55cb7a73b080 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87acc00640 1 -- 172.21.15.37:0/1356235832 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f87a809acc0 con 0x55cb7a73b080 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 10 client.27737 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 10 client.27737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 20 client.27737 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7b0260 0x55cb7a7b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 4 client.27737 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.367+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb7a6a5b50 con 0x55cb7a7b0260 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.368+0000 7f87bf400640 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7b0260 0x55cb7a7b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.368+0000 7f87bf400640 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7b0260 0x55cb7a7b2680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b70e2b86aa6172fc server_cookie=7b2deba79adedd29 in_seq=0 out_seq=0 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.368+0000 7f87acc00640 10 client.27737 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.368+0000 7f87acc00640 1 -- 172.21.15.37:0/1356235832 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb7a6a5b50 con 0x55cb7a7b0260 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.368+0000 7f87acc00640 10 client.27737 handle_command_reply: tid=0 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.368+0000 7f87c8bf3740 1 client.27737 shutdown 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 msgr2=0x55cb7a749fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a749fe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c6a0 tx=0x7f87a80a4970 comp rx=0 tx=0).stop 2024-03-21T09:21:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 shutdown_connections 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cb7a743c00 0x55cb7a7ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb7a73b080 0x55cb7a749fe0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x55cb7a7b0260 0x55cb7a7b2680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 --2- 172.21.15.37:0/1356235832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb7a73a6d0 0x55cb7a749aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 >> 172.21.15.37:0/1356235832 conn(0x55cb7a734da0 msgr2=0x55cb7a733620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87acc00640 0 client.27737 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 shutdown_connections 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 1 -- 172.21.15.37:0/1356235832 wait complete. 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c8bf3740 20 client.27737 trim_cache size 0 max 16384 2024-03-21T09:21:27.370 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f87a40a3090 msgr2=0x7f87a40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.369+0000 7f87c6000640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f87a40a3090 0x7f87a40a5550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55cb7a735e10 tx=0x7f87b0005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 msgr2=0x7f87c019eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 0x7f87c019eaa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 shutdown_connections 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87c0112640 0x7f87c01a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f87a40a3090 0x7f87a40a5550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0074360 0x7f87c019eaa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 --2- 172.21.15.37:0/4202149319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0073a40 0x7f87c019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 >> 172.21.15.37:0/4202149319 conn(0x7f87c00fd140 msgr2=0x7f87c00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 shutdown_connections 2024-03-21T09:21:27.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.370+0000 7f87c6000640 1 -- 172.21.15.37:0/4202149319 wait complete. 2024-03-21T09:21:27.386 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:21:27.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:21:27.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 -- 172.21.15.37:0/3599453298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c109870 msgr2=0x7fd25c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 --2- 172.21.15.37:0/3599453298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c109870 0x7fd25c109c50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd250009cd0 tx=0x7fd25002c0f0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 -- 172.21.15.37:0/3599453298 shutdown_connections 2024-03-21T09:21:27.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 --2- 172.21.15.37:0/3599453298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd25c104220 0x7fd25c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 --2- 172.21.15.37:0/3599453298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c103860 0x7fd25c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 --2- 172.21.15.37:0/3599453298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c109870 0x7fd25c109c50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 -- 172.21.15.37:0/3599453298 >> 172.21.15.37:0/3599453298 conn(0x7fd25c0ff310 msgr2=0x7fd25c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 -- 172.21.15.37:0/3599453298 shutdown_connections 2024-03-21T09:21:27.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.607+0000 7fd264000640 1 -- 172.21.15.37:0/3599453298 wait complete. 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.608+0000 7fd264000640 1 Processor -- start 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.608+0000 7fd264000640 1 -- start start 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd264000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 0x7fd25c19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd264000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd25c104220 0x7fd25c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd264000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c109870 0x7fd25c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd264000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c113c80 con 0x7fd25c109870 2024-03-21T09:21:27.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd264000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c113b00 con 0x7fd25c104220 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd264000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c113e00 con 0x7fd25c103860 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 0x7fd25c19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 0x7fd25c19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57110/0 (socket says 172.21.15.37:57110) 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 -- 172.21.15.37:0/1919852711 learned_addr learned my addr 172.21.15.37:0/1919852711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd260c00640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd25c104220 0x7fd25c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd262000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c109870 0x7fd25c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 -- 172.21.15.37:0/1919852711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd25c104220 msgr2=0x7fd25c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd25c104220 0x7fd25c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 -- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c109870 msgr2=0x7fd25c19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.609+0000 7fd261600640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c109870 0x7fd25c19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd261600640 1 -- 172.21.15.37:0/1919852711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c19f130 con 0x7fd25c103860 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd262000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c109870 0x7fd25c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd261600640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 0x7fd25c19a140 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd250004af0 tx=0x7fd250032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd25a000640 1 -- 172.21.15.37:0/1919852711 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250041020 con 0x7fd25c103860 2024-03-21T09:21:27.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd25a000640 1 -- 172.21.15.37:0/1919852711 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd250034cd0 con 0x7fd25c103860 2024-03-21T09:21:27.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c19f3c0 con 0x7fd25c103860 2024-03-21T09:21:27.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd25a000640 1 -- 172.21.15.37:0/1919852711 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250039040 con 0x7fd25c103860 2024-03-21T09:21:27.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.610+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c1a6cb0 con 0x7fd25c103860 2024-03-21T09:21:27.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.613+0000 7fd266d8d740 1 Processor -- start 2024-03-21T09:21:27.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd25a000640 1 -- 172.21.15.37:0/1919852711 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd250044020 con 0x7fd25c103860 2024-03-21T09:21:27.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 -- start start 2024-03-21T09:21:27.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 0x55b174c66ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c72050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b174bca2c0 con 0x55b174c6fc00 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b174b72c90 con 0x55b174c67080 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd266d8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b174bc6360 con 0x55b174c666d0 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd262000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 0x55b174c66ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd262000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 0x55b174c66ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57120/0 (socket says 172.21.15.37:57120) 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd262000640 1 -- 172.21.15.37:0/3630111683 learned_addr learned my addr 172.21.15.37:0/3630111683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd261600640 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.614+0000 7fd260c00640 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c72050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd262000640 1 -- 172.21.15.37:0/3630111683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 msgr2=0x55b174c6f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd262000640 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd262000640 1 -- 172.21.15.37:0/3630111683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 msgr2=0x55b174c72050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd262000640 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c72050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd262000640 1 -- 172.21.15.37:0/3630111683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b174b97080 con 0x55b174c666d0 2024-03-21T09:21:27.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd260c00640 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c72050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd262000640 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 0x55b174c66ab0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd254009c20 tx=0x7fd2540117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8984577879064aef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd25a000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2440a3110 0x7fd2440a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd24e600640 1 -- 172.21.15.37:0/3630111683 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254012450 con 0x55b174c666d0 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd24e600640 1 -- 172.21.15.37:0/3630111683 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd254012ad0 con 0x55b174c666d0 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.615+0000 7fd24e600640 1 -- 172.21.15.37:0/3630111683 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400fc10 con 0x55b174c666d0 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd25a000640 1 -- 172.21.15.37:0/1919852711 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd2500f17e0 con 0x7fd25c103860 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd261600640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2440a3110 0x7fd2440a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 -- 172.21.15.37:0/3630111683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 msgr2=0x55b174c66ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 0x55b174c66ab0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd254009c20 tx=0x7fd2540117b0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd261600640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2440a3110 0x7fd2440a55d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd2500eecf0 tx=0x7fd2500f7550 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 -- 172.21.15.37:0/3630111683 shutdown_connections 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c72050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 --2- 172.21.15.37:0/3630111683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c666d0 0x55b174c66ab0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.616+0000 7fd266d8d740 1 -- 172.21.15.37:0/3630111683 >> 172.21.15.37:0/3630111683 conn(0x55b174c60da0 msgr2=0x55b174c61180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.617+0000 7fd266d8d740 1 -- 172.21.15.37:0/3630111683 shutdown_connections 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.617+0000 7fd266d8d740 1 -- 172.21.15.37:0/3630111683 wait complete. 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.617+0000 7fd266d8d740 1 -- start start 2024-03-21T09:21:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.617+0000 7fd266d8d740 1 Processor -- start 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c6fc00 0x55b174c6cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b174c72b70 con 0x55b174c666d0 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b174b860d0 con 0x55b174c67080 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b174c72cf0 con 0x55b174c6fc00 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd266d8d740 10 client.0 _ll_register_callbacks cb 0x55b174b8ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44772/0 (socket says 172.21.15.37:44772) 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 -- 172.21.15.37:0/1605210031 learned_addr learned my addr 172.21.15.37:0/1605210031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd262000640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd262000640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c6fc00 0x55b174c6cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 -- 172.21.15.37:0/1605210031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c6fc00 msgr2=0x55b174c6cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c6fc00 0x55b174c6cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 -- 172.21.15.37:0/1605210031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 msgr2=0x55b174c6f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.618+0000 7fd260c00640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c6f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd260c00640 1 -- 172.21.15.37:0/1605210031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1748de3f0 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd262000640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c6fc00 0x55b174c6cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd260c00640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6c8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bef0 tx=0x7fd248012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81f8723bb47faff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd242a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd242a00640 1 -- 172.21.15.37:0/1605210031 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248005220 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd242a00640 1 -- 172.21.15.37:0/1605210031 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd2480053c0 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd254012ad0 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd242a00640 1 -- 172.21.15.37:0/1605210031 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd248025050 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd266d8d740 10 client.27749 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.619+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b1749a87d0 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd242a00640 1 -- 172.21.15.37:0/1605210031 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd254012ad0 con 0x55b174c67080 2024-03-21T09:21:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd242a00640 1 -- 172.21.15.37:0/1605210031 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd248010b70 con 0x55b174c67080 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd266d8d740 10 client.27749 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd266d8d740 10 client.27749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd266d8d740 20 client.27749 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174c7fb20 0x55b174c81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd266d8d740 4 client.27749 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b174bd7800 con 0x55b174c7fb20 2024-03-21T09:21:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.620+0000 7fd262000640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174c7fb20 0x55b174c81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.621+0000 7fd262000640 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174c7fb20 0x55b174c81f10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad820e9cb5150de7 server_cookie=47302f183c08348e in_seq=0 out_seq=0 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.621+0000 7fd242a00640 10 client.27749 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.621+0000 7fd242a00640 1 -- 172.21.15.37:0/1605210031 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b174bd7800 con 0x55b174c7fb20 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.621+0000 7fd242a00640 10 client.27749 handle_command_reply: tid=0 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 client.27749 shutdown 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 msgr2=0x55b174c6c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6c8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bef0 tx=0x7fd248012820 comp rx=0 tx=0).stop 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 shutdown_connections 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174c7fb20 0x55b174c81f10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c6fc00 0x55b174c6cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd242a00640 0 client.27749 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c67080 0x55b174c6c8f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1605210031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c6f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 >> 172.21.15.37:0/1605210031 conn(0x55b174c60da0 msgr2=0x55b174c62af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 shutdown_connections 2024-03-21T09:21:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 1 -- 172.21.15.37:0/1605210031 wait complete. 2024-03-21T09:21:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.622+0000 7fd266d8d740 20 client.27749 trim_cache size 0 max 16384 2024-03-21T09:21:27.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 Processor -- start 2024-03-21T09:21:27.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 -- start start 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c666d0 0x55b174c6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c69b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1b3240 con 0x55b174c6fc00 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1b0d40 con 0x55b174c666d0 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd266d8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1aee00 con 0x55b174c67080 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd262000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c666d0 0x55b174c6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd260c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd260c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c6f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57144/0 (socket says 172.21.15.37:57144) 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd260c00640 1 -- 172.21.15.37:0/39376969 learned_addr learned my addr 172.21.15.37:0/39376969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.640+0000 7fd262000640 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c69b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd260c00640 1 -- 172.21.15.37:0/39376969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c666d0 msgr2=0x55b174c6f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd260c00640 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c666d0 0x55b174c6f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd260c00640 1 -- 172.21.15.37:0/39376969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 msgr2=0x55b174c69b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd260c00640 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c69b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd260c00640 1 -- 172.21.15.37:0/39376969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b174c82540 con 0x55b174c67080 2024-03-21T09:21:27.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd262000640 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c69b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:27.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd260c00640 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c6f560 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd248006010 tx=0x7fd24800e470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb83d5cdca875725 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd24dc00640 1 -- 172.21.15.37:0/39376969 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24802d070 con 0x55b174c67080 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd24dc00640 1 -- 172.21.15.37:0/39376969 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd24800e570 con 0x55b174c67080 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.641+0000 7fd24dc00640 1 -- 172.21.15.37:0/39376969 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800e820 con 0x55b174c67080 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 -- 172.21.15.37:0/39376969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 msgr2=0x55b174c6f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c6f560 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd248006010 tx=0x7fd24800e470 comp rx=0 tx=0).stop 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 -- 172.21.15.37:0/39376969 shutdown_connections 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c6fc00 0x55b174c69b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c6f560 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 --2- 172.21.15.37:0/39376969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c666d0 0x55b174c6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 -- 172.21.15.37:0/39376969 >> 172.21.15.37:0/39376969 conn(0x55b174c60da0 msgr2=0x55b174c5feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 -- 172.21.15.37:0/39376969 shutdown_connections 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.642+0000 7fd266d8d740 1 -- 172.21.15.37:0/39376969 wait complete. 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.643+0000 7fd266d8d740 1 -- start start 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.643+0000 7fd266d8d740 1 Processor -- start 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c75b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c76060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c6fc00 0x55b174cda340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1af400 con 0x55b174c666d0 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1b06e0 con 0x55b174c6fc00 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1afa20 con 0x55b174c67080 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd266d8d740 10 client.0 _ll_register_callbacks cb 0x7fd24800ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd260c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c76060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd260c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c76060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57148/0 (socket says 172.21.15.37:57148) 2024-03-21T09:21:27.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd260c00640 1 -- 172.21.15.37:0/1749505338 learned_addr learned my addr 172.21.15.37:0/1749505338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.644+0000 7fd262000640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c75b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd260c00640 1 -- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c6fc00 msgr2=0x55b174cda340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd262000640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c6fc00 0x55b174cda340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd260c00640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c6fc00 0x55b174cda340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd260c00640 1 -- 172.21.15.37:0/1749505338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 msgr2=0x55b174c75b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd260c00640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c75b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd260c00640 1 -- 172.21.15.37:0/1749505338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b174cda930 con 0x55b174c67080 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd262000640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c6fc00 0x55b174cda340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd260c00640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c76060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bef0 tx=0x7fd24800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d891f2d54976e0da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd242a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd242a00640 1 -- 172.21.15.37:0/1749505338 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248036030 con 0x55b174c67080 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd242a00640 1 -- 172.21.15.37:0/1749505338 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd248010970 con 0x55b174c67080 2024-03-21T09:21:27.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.645+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b174bd1b50 con 0x55b174c67080 2024-03-21T09:21:27.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd242a00640 1 -- 172.21.15.37:0/1749505338 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd24802d040 con 0x55b174c67080 2024-03-21T09:21:27.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd266d8d740 10 client.27778 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:27.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b174cdbc10 con 0x55b174c67080 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd242a00640 1 -- 172.21.15.37:0/1749505338 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b174bd1b50 con 0x55b174c67080 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd242a00640 1 -- 172.21.15.37:0/1749505338 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd248032680 con 0x55b174c67080 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd266d8d740 10 client.27778 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd266d8d740 10 client.27778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.650+0000 7fd266d8d740 20 client.27778 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174cdc280 0x55b174cde6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd266d8d740 4 client.27778 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b174b4f500 con 0x55b174cdc280 2024-03-21T09:21:27.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd262000640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174cdc280 0x55b174cde6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd262000640 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174cdc280 0x55b174cde6a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99d0d83bd5079eaa server_cookie=2a0ac0ba27bb9230 in_seq=0 out_seq=0 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd242a00640 10 client.27778 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.651+0000 7fd242a00640 1 -- 172.21.15.37:0/1749505338 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b174b4f500 con 0x55b174cdc280 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd242a00640 10 client.27778 handle_command_reply: tid=0 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 client.27778 shutdown 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 msgr2=0x55b174c76060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c76060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd24800bef0 tx=0x7fd24800b810 comp rx=0 tx=0).stop 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 shutdown_connections 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55b174cdc280 0x55b174cde6a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b174c6fc00 0x55b174cda340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b174c67080 0x55b174c76060 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 --2- 172.21.15.37:0/1749505338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b174c666d0 0x55b174c75b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd242a00640 0 client.27778 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 >> 172.21.15.37:0/1749505338 conn(0x55b174c60da0 msgr2=0x55b174c62080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 shutdown_connections 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 1 -- 172.21.15.37:0/1749505338 wait complete. 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.652+0000 7fd266d8d740 20 client.27778 trim_cache size 0 max 16384 2024-03-21T09:21:27.652 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2440a3110 msgr2=0x7fd2440a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2440a3110 0x7fd2440a55d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd2500eecf0 tx=0x7fd2500f7550 comp rx=0 tx=0).stop 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 msgr2=0x7fd25c19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 0x7fd25c19a140 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd250004af0 tx=0x7fd250032ea0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 shutdown_connections 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd25c109870 0x7fd25c19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd2440a3110 0x7fd2440a55d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd25c104220 0x7fd25c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 --2- 172.21.15.37:0/1919852711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c103860 0x7fd25c19a140 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 >> 172.21.15.37:0/1919852711 conn(0x7fd25c0ff310 msgr2=0x7fd25c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 shutdown_connections 2024-03-21T09:21:27.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.653+0000 7fd264000640 1 -- 172.21.15.37:0/1919852711 wait complete. 2024-03-21T09:21:27.678 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:21:27.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 -- 172.21.15.37:0/3929690433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101e60 msgr2=0x7f199410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 --2- 172.21.15.37:0/3929690433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101e60 0x7f199410e320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f198c009930 tx=0x7f198c02bc20 comp rx=0 tx=0).stop 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 -- 172.21.15.37:0/3929690433 shutdown_connections 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 --2- 172.21.15.37:0/3929690433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101e60 0x7f199410e320 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 --2- 172.21.15.37:0/3929690433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19941014a0 0x7f1994101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 --2- 172.21.15.37:0/3929690433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 0x7f1994107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 -- 172.21.15.37:0/3929690433 >> 172.21.15.37:0/3929690433 conn(0x7f19940fcfb0 msgr2=0x7f19940ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 -- 172.21.15.37:0/3929690433 shutdown_connections 2024-03-21T09:21:27.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.925+0000 7f1993200640 1 -- 172.21.15.37:0/3929690433 wait complete. 2024-03-21T09:21:27.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.927+0000 7f1993200640 1 Processor -- start 2024-03-21T09:21:27.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.927+0000 7f1993200640 1 -- start start 2024-03-21T09:21:27.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:27 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1818053711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:21:27.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:27 smithi037 ceph-mon[31830]: pgmap v287: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.927+0000 7f1993200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 0x7f1994199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.927+0000 7f1993200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1994101e60 0x7f199419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1993200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 0x7f199419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1993200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994111870 con 0x7f1994107560 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1993200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941116f0 con 0x7f19941014a0 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1993200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941119f0 con 0x7f1994101e60 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 0x7f1994199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 0x7f1994199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44812/0 (socket says 172.21.15.37:44812) 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 -- 172.21.15.37:0/1474120332 learned_addr learned my addr 172.21.15.37:0/1474120332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1989400640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1994101e60 0x7f199419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1992800640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 0x7f199419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 -- 172.21.15.37:0/1474120332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1994101e60 msgr2=0x7f199419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1994101e60 0x7f199419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 -- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 msgr2=0x7f199419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 0x7f199419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 -- 172.21.15.37:0/1474120332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199419ef50 con 0x7f19941014a0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1992800640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 0x7f199419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.928+0000 7f1991e00640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 0x7f1994199f60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f198400ebc0 tx=0x7f198400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.929+0000 7f198b400640 1 -- 172.21.15.37:0/1474120332 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984010630 con 0x7f19941014a0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.929+0000 7f198b400640 1 -- 172.21.15.37:0/1474120332 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1984010cb0 con 0x7f19941014a0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.929+0000 7f198b400640 1 -- 172.21.15.37:0/1474120332 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984018bd0 con 0x7f19941014a0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.929+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199419f240 con 0x7f19941014a0 2024-03-21T09:21:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.929+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941a6ad0 con 0x7f19941014a0 2024-03-21T09:21:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.933+0000 7f199bf91740 1 Processor -- start 2024-03-21T09:21:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 -- start start 2024-03-21T09:21:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f198b400640 1 -- 172.21.15.37:0/1474120332 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f19840107d0 con 0x7f19941014a0 2024-03-21T09:21:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d242c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d24b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d24bde0 0x55a13d24e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13d1a62c0 con 0x55a13d243260 2024-03-21T09:21:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13d14ec90 con 0x55a13d24bde0 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f199bf91740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13d1a2360 con 0x55a13d2428b0 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f1992800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d242c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f1992800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d242c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57180/0 (socket says 172.21.15.37:57180) 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f1992800640 1 -- 172.21.15.37:0/1292394624 learned_addr learned my addr 172.21.15.37:0/1292394624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f1991e00640 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d24b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.934+0000 7f1989400640 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d24bde0 0x55a13d24e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1992800640 1 -- 172.21.15.37:0/1292394624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d24bde0 msgr2=0x55a13d24e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1992800640 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d24bde0 0x55a13d24e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1992800640 1 -- 172.21.15.37:0/1292394624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 msgr2=0x55a13d24b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1992800640 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d24b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1992800640 1 -- 172.21.15.37:0/1292394624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a13d173080 con 0x55a13d2428b0 2024-03-21T09:21:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1989400640 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d24bde0 0x55a13d24e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1992800640 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d242c90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bebb2382e55d2b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1983400640 1 -- 172.21.15.37:0/1292394624 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c044020 con 0x55a13d2428b0 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1983400640 1 -- 172.21.15.37:0/1292394624 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f198c004240 con 0x55a13d2428b0 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.935+0000 7f1983400640 1 -- 172.21.15.37:0/1292394624 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c0055f0 con 0x55a13d2428b0 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f198b400640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f19780a3110 0x7f19780a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f198b400640 1 -- 172.21.15.37:0/1474120332 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f19840d01e0 con 0x7f19941014a0 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f1991e00640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f19780a3110 0x7f19780a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 -- 172.21.15.37:0/1292394624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 msgr2=0x55a13d242c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d242c90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c005de0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 -- 172.21.15.37:0/1292394624 shutdown_connections 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d24bde0 0x55a13d24e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d24b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 --2- 172.21.15.37:0/1292394624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d242c90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 -- 172.21.15.37:0/1292394624 >> 172.21.15.37:0/1292394624 conn(0x55a13d23cf80 msgr2=0x55a13d23d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.936+0000 7f199bf91740 1 -- 172.21.15.37:0/1292394624 shutdown_connections 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.937+0000 7f199bf91740 1 -- 172.21.15.37:0/1292394624 wait complete. 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.937+0000 7f1991e00640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f19780a3110 0x7f19780a55d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55a13d242380 tx=0x7f19840d3c90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.937+0000 7f199bf91740 1 -- start start 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.937+0000 7f199bf91740 1 Processor -- start 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 0x55a13d24b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d248a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d248fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13d24ecf0 con 0x55a13d243260 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13d1620d0 con 0x55a13d2428b0 2024-03-21T09:21:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13d24ee70 con 0x55a13d24bde0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f199bf91740 10 client.0 _ll_register_callbacks cb 0x55a13d166e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f1992800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 0x55a13d24b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f1992800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 0x55a13d24b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44838/0 (socket says 172.21.15.37:44838) 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f1992800640 1 -- 172.21.15.37:0/3033118085 learned_addr learned my addr 172.21.15.37:0/3033118085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f1992800640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d248fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.938+0000 7f1989400640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d248a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1992800640 1 -- 172.21.15.37:0/3033118085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 msgr2=0x55a13d248fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1992800640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d248fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1992800640 1 -- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 msgr2=0x55a13d248a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1992800640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d248a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1992800640 1 -- 172.21.15.37:0/3033118085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a13ceba3f0 con 0x55a13d2428b0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1992800640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 0x55a13d24b6c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f198c009850 tx=0x7f198c005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6760e1f0f5a154f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1977e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1977e00640 1 -- 172.21.15.37:0/3033118085 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c04e070 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1977e00640 1 -- 172.21.15.37:0/3033118085 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f198c02cc80 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f198c004240 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.939+0000 7f1977e00640 1 -- 172.21.15.37:0/3033118085 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f198c044030 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 10 client.27764 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a13cf847d0 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f1977e00640 1 -- 172.21.15.37:0/3033118085 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f198c004240 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f1977e00640 1 -- 172.21.15.37:0/3033118085 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f198c0049c0 con 0x55a13d2428b0 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 10 client.27764 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 10 client.27764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 20 client.27764 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d25ba50 0x55a13d25de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 4 client.27764 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a13d1b3800 con 0x55a13d25ba50 2024-03-21T09:21:27.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.940+0000 7f1989400640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d25ba50 0x55a13d25de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.941+0000 7f1989400640 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d25ba50 0x55a13d25de40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a402f770f0fb02 server_cookie=24f70af89a06fe8c in_seq=0 out_seq=0 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.941+0000 7f1977e00640 10 client.27764 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f1977e00640 1 -- 172.21.15.37:0/3033118085 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a13d1b3800 con 0x55a13d25ba50 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f1977e00640 10 client.27764 handle_command_reply: tid=0 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 client.27764 shutdown 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 msgr2=0x55a13d24b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 0x55a13d24b6c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f198c009850 tx=0x7f198c005230 comp rx=0 tx=0).stop 2024-03-21T09:21:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 shutdown_connections 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d248fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d243260 0x55a13d248a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d25ba50 0x55a13d25de40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 --2- 172.21.15.37:0/3033118085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d2428b0 0x55a13d24b6c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f1977e00640 0 client.27764 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 >> 172.21.15.37:0/3033118085 conn(0x55a13d23cf80 msgr2=0x55a13d23ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 shutdown_connections 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 1 -- 172.21.15.37:0/3033118085 wait complete. 2024-03-21T09:21:27.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.942+0000 7f199bf91740 20 client.27764 trim_cache size 0 max 16384 2024-03-21T09:21:27.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:27 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1818053711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:21:27.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:27 smithi053 ceph-mon[36169]: pgmap v287: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.956+0000 7f199bf91740 1 Processor -- start 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.956+0000 7f199bf91740 1 -- start start 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f199bf91740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d24e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f199bf91740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d24b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f199bf91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d24bde0 0x55a13d24b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f199bf91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941b31c0 con 0x55a13d24bde0 2024-03-21T09:21:27.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f199bf91740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941b0cc0 con 0x55a13d243260 2024-03-21T09:21:27.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f199bf91740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941aed80 con 0x55a13d2428b0 2024-03-21T09:21:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d24b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d24b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44844/0 (socket says 172.21.15.37:44844) 2024-03-21T09:21:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 -- 172.21.15.37:0/3827507409 learned_addr learned my addr 172.21.15.37:0/3827507409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1992800640 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d24e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1992800640 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d24bde0 0x55a13d24b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 -- 172.21.15.37:0/3827507409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 msgr2=0x55a13d24e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d24e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 -- 172.21.15.37:0/3827507409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d24bde0 msgr2=0x55a13d24b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d24bde0 0x55a13d24b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1989400640 1 -- 172.21.15.37:0/3827507409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a13d246fa0 con 0x55a13d243260 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.957+0000 7f1992800640 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d24bde0 0x55a13d24b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.958+0000 7f1989400640 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d24b180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f197c009cf0 tx=0x7f197c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc385df68063bf22 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.958+0000 7f1982a00640 1 -- 172.21.15.37:0/3827507409 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c01f310 con 0x55a13d243260 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.958+0000 7f1982a00640 1 -- 172.21.15.37:0/3827507409 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f197c024070 con 0x55a13d243260 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.958+0000 7f1982a00640 1 -- 172.21.15.37:0/3827507409 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c010690 con 0x55a13d243260 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.958+0000 7f199bf91740 1 -- 172.21.15.37:0/3827507409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 msgr2=0x55a13d24b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.958+0000 7f199bf91740 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d24b180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f197c009cf0 tx=0x7f197c005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 -- 172.21.15.37:0/3827507409 shutdown_connections 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d24bde0 0x55a13d24b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d24b180 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 --2- 172.21.15.37:0/3827507409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d2428b0 0x55a13d24e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 -- 172.21.15.37:0/3827507409 >> 172.21.15.37:0/3827507409 conn(0x55a13d23cf80 msgr2=0x55a13d23c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 -- 172.21.15.37:0/3827507409 shutdown_connections 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 -- 172.21.15.37:0/3827507409 wait complete. 2024-03-21T09:21:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 -- start start 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.959+0000 7f199bf91740 1 Processor -- start 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d2428b0 0x55a13d260a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d25ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d25f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941af380 con 0x55a13d2428b0 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941b0660 con 0x55a13d243260 2024-03-21T09:21:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941af9a0 con 0x55a13d24bde0 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f199bf91740 10 client.0 _ll_register_callbacks cb 0x7f19840d3b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f1989400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d25ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f1989400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d25ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44848/0 (socket says 172.21.15.37:44848) 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f1989400640 1 -- 172.21.15.37:0/913257362 learned_addr learned my addr 172.21.15.37:0/913257362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f1992800640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d2428b0 0x55a13d260a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.960+0000 7f1992800640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d25f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1989400640 1 -- 172.21.15.37:0/913257362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 msgr2=0x55a13d25f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1989400640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d25f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1989400640 1 -- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d2428b0 msgr2=0x55a13d260a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1989400640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d2428b0 0x55a13d260a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1989400640 1 -- 172.21.15.37:0/913257362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a13d25fa50 con 0x55a13d243260 2024-03-21T09:21:27.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1992800640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d25f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1989400640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d25ee20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f197c005b90 tx=0x7f197c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb4843509d700bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1977e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1977e00640 1 -- 172.21.15.37:0/913257362 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c02d070 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f197c024070 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1977e00640 1 -- 172.21.15.37:0/913257362 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f197c003ca0 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f1977e00640 1 -- 172.21.15.37:0/913257362 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197c039050 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f199bf91740 10 client.27770 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.961+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a13d2b7ef0 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f1977e00640 1 -- 172.21.15.37:0/913257362 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f197c024070 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f1977e00640 1 -- 172.21.15.37:0/913257362 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f197c004700 con 0x55a13d243260 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f199bf91740 10 client.27770 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f199bf91740 10 client.27770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f199bf91740 20 client.27770 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f199bf91740 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d2b8600 0x55a13d2baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f199bf91740 4 client.27770 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a13d1adb50 con 0x55a13d2b8600 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.962+0000 7f1992800640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d2b8600 0x55a13d2baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.963+0000 7f1992800640 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d2b8600 0x55a13d2baa20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e532d71c44c93f54 server_cookie=5510c2d2aa8b6633 in_seq=0 out_seq=0 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.963+0000 7f1977e00640 10 client.27770 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.963+0000 7f1977e00640 1 -- 172.21.15.37:0/913257362 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a13d1adb50 con 0x55a13d2b8600 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.963+0000 7f1977e00640 10 client.27770 handle_command_reply: tid=0 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.963+0000 7f199bf91740 1 client.27770 shutdown 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 msgr2=0x55a13d25ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d25ee20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f197c005b90 tx=0x7f197c009e30 comp rx=0 tx=0).stop 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 shutdown_connections 2024-03-21T09:21:27.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a13d24bde0 0x55a13d25f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55a13d2b8600 0x55a13d2baa20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a13d243260 0x55a13d25ee20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 --2- 172.21.15.37:0/913257362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a13d2428b0 0x55a13d260a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 >> 172.21.15.37:0/913257362 conn(0x55a13d23cf80 msgr2=0x55a13d23e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f1977e00640 0 client.27770 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 shutdown_connections 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 1 -- 172.21.15.37:0/913257362 wait complete. 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.964+0000 7f199bf91740 20 client.27770 trim_cache size 0 max 16384 2024-03-21T09:21:27.966 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:21:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f19780a3110 msgr2=0x7f19780a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f19780a3110 0x7f19780a55d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55a13d242380 tx=0x7f19840d3c90 comp rx=0 tx=0).stop 2024-03-21T09:21:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 msgr2=0x7f1994199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 0x7f1994199f60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f198400ebc0 tx=0x7f198400c6a0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 shutdown_connections 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1994107560 0x7f199419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1994101e60 0x7f199419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f19780a3110 0x7f19780a55d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 --2- 172.21.15.37:0/1474120332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19941014a0 0x7f1994199f60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 >> 172.21.15.37:0/1474120332 conn(0x7f19940fcfb0 msgr2=0x7f19940fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 shutdown_connections 2024-03-21T09:21:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:27.965+0000 7f1993200640 1 -- 172.21.15.37:0/1474120332 wait complete. 2024-03-21T09:21:28.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:27 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1818053711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:21:28.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:27 smithi071 ceph-mon[36597]: pgmap v287: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:21:28.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:21:28.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:21:28.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:21:29.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 -- 172.21.15.37:0/3936999468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 msgr2=0x7f5da0111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 --2- 172.21.15.37:0/3936999468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da0111df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009890 tx=0x7f5d8c02bb70 comp rx=0 tx=0).stop 2024-03-21T09:21:29.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 -- 172.21.15.37:0/3936999468 shutdown_connections 2024-03-21T09:21:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 --2- 172.21.15.37:0/3936999468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 0x7f5da0114720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 --2- 172.21.15.37:0/3936999468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da0111df0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 --2- 172.21.15.37:0/3936999468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 0x7f5da0075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 -- 172.21.15.37:0/3936999468 >> 172.21.15.37:0/3936999468 conn(0x7f5da00ff140 msgr2=0x7f5da0101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 -- 172.21.15.37:0/3936999468 shutdown_connections 2024-03-21T09:21:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.220+0000 7f5da4a00640 1 -- 172.21.15.37:0/3936999468 wait complete. 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.221+0000 7f5da4a00640 1 Processor -- start 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 -- start start 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 0x7f5da019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 0x7f5da01a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0117e90 con 0x7f5da0112330 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0117d10 con 0x7f5da0075bd0 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5da4a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0118010 con 0x7f5da00764f0 2024-03-21T09:21:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.222+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 0x7f5da019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 0x7f5da019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44854/0 (socket says 172.21.15.37:44854) 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da019ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57244/0 (socket says 172.21.15.37:57244) 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9f400640 1 -- 172.21.15.37:0/2910194710 learned_addr learned my addr 172.21.15.37:0/2910194710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 0x7f5da01a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2910194710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 msgr2=0x7f5da019e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 0x7f5da019e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 msgr2=0x7f5da01a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 0x7f5da01a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2910194710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da01a3480 con 0x7f5da00764f0 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 0x7f5da01a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:29.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.223+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da019ea20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009780 tx=0x7f5d8c004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.224+0000 7f5d97e00640 1 -- 172.21.15.37:0/2910194710 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c041020 con 0x7f5da00764f0 2024-03-21T09:21:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.224+0000 7f5d97e00640 1 -- 172.21.15.37:0/2910194710 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c02c9f0 con 0x7f5da00764f0 2024-03-21T09:21:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.224+0000 7f5d97e00640 1 -- 172.21.15.37:0/2910194710 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c039040 con 0x7f5da00764f0 2024-03-21T09:21:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.224+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da01a3710 con 0x7f5da00764f0 2024-03-21T09:21:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.224+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da01ab000 con 0x7f5da00764f0 2024-03-21T09:21:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.227+0000 7f5da8dee740 1 Processor -- start 2024-03-21T09:21:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.227+0000 7f5d97e00640 1 -- 172.21.15.37:0/2910194710 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5d8c044020 con 0x7f5da00764f0 2024-03-21T09:21:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 -- start start 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f6b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f74250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773ecfeb0 con 0x55d773f74790 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e78a60 con 0x55d773f6bbb0 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5da8dee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773ecbf50 con 0x55d773f6b290 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5d9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f6b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f74250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5d9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f6b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57248/0 (socket says 172.21.15.37:57248) 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5d9fe00640 1 -- 172.21.15.37:0/1794784335 learned_addr learned my addr 172.21.15.37:0/1794784335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.228+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9fe00640 1 -- 172.21.15.37:0/1794784335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 msgr2=0x55d773f74250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f74250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9fe00640 1 -- 172.21.15.37:0/1794784335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 msgr2=0x55d773f76b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f76b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9fe00640 1 -- 172.21.15.37:0/1794784335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d773cac7e0 con 0x55d773f6b290 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9f400640 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f74250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f6b670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ef0 tx=0x7f5d98010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a5d4fb35ef485 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d97e00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d840a3110 0x7f5d840a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d97e00640 1 -- 172.21.15.37:0/2910194710 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5d8c0f13f0 con 0x7f5da00764f0 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d8be00640 1 -- 172.21.15.37:0/1794784335 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98016070 con 0x55d773f6b290 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d8be00640 1 -- 172.21.15.37:0/1794784335 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9801bb40 con 0x55d773f6b290 2024-03-21T09:21:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.229+0000 7f5d8be00640 1 -- 172.21.15.37:0/1794784335 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98012c40 con 0x55d773f6b290 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d840a3110 0x7f5d840a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 -- 172.21.15.37:0/1794784335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 msgr2=0x55d773f6b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f6b670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ef0 tx=0x7f5d98010740 comp rx=0 tx=0).stop 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d840a3110 0x7f5d840a55d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55d773f6ad60 tx=0x7f5d90009260 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 -- 172.21.15.37:0/1794784335 shutdown_connections 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f76b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f74250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 --2- 172.21.15.37:0/1794784335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f6b670 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 -- 172.21.15.37:0/1794784335 >> 172.21.15.37:0/1794784335 conn(0x55d773f659c0 msgr2=0x55d773f65da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.230+0000 7f5da8dee740 1 -- 172.21.15.37:0/1794784335 shutdown_connections 2024-03-21T09:21:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.231+0000 7f5da8dee740 1 -- 172.21.15.37:0/1794784335 wait complete. 2024-03-21T09:21:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.231+0000 7f5da8dee740 1 -- start start 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.231+0000 7f5da8dee740 1 Processor -- start 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f71a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773f77580 con 0x55d773f74790 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e8beb0 con 0x55d773f6bbb0 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773f77700 con 0x55d773f6b290 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5da8dee740 10 client.0 _ll_register_callbacks cb 0x55d773e80150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f71a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44886/0 (socket says 172.21.15.37:44886) 2024-03-21T09:21:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2543998289 learned_addr learned my addr 172.21.15.37:0/2543998289 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2543998289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 msgr2=0x55d773f74120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f74120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 msgr2=0x55d773f71a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f71a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.232+0000 7f5d9ea00640 1 -- 172.21.15.37:0/2543998289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d773d0cf70 con 0x55d773f6bbb0 2024-03-21T09:21:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f714b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c035fd0 tx=0x7f5d8c02bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac9c7dc3f9d6e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d88c00640 1 -- 172.21.15.37:0/2543998289 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0bcdf0 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d88c00640 1 -- 172.21.15.37:0/2543998289 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5d8c034700 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9801bb40 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d88c00640 1 -- 172.21.15.37:0/2543998289 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8c0bdb90 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5da8dee740 10 client.27782 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d773ba4170 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d88c00640 1 -- 172.21.15.37:0/2543998289 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d9801bb40 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.233+0000 7f5d88c00640 1 -- 172.21.15.37:0/2543998289 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5d8c03c030 con 0x55d773f6bbb0 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5da8dee740 10 client.27782 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5da8dee740 10 client.27782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5da8dee740 20 client.27782 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773f847a0 0x55d773f86b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5da8dee740 4 client.27782 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d773edd3f0 con 0x55d773f847a0 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.234+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773f847a0 0x55d773f86b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.235+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773f847a0 0x55d773f86b90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0a1848ef59b5b9 server_cookie=544a0f12c320718d in_seq=0 out_seq=0 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.235+0000 7f5d88c00640 10 client.27782 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.235+0000 7f5d88c00640 1 -- 172.21.15.37:0/2543998289 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d773edd3f0 con 0x55d773f847a0 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.235+0000 7f5d88c00640 10 client.27782 handle_command_reply: tid=0 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 client.27782 shutdown 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 msgr2=0x55d773f714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f714b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c035fd0 tx=0x7f5d8c02bb70 comp rx=0 tx=0).stop 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 shutdown_connections 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f74790 0x55d773f71a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773f847a0 0x55d773f86b90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f714b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2543998289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f6b290 0x55d773f74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 >> 172.21.15.37:0/2543998289 conn(0x55d773f659c0 msgr2=0x55d773f67710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5d88c00640 0 client.27782 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 shutdown_connections 2024-03-21T09:21:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 1 -- 172.21.15.37:0/2543998289 wait complete. 2024-03-21T09:21:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.236+0000 7f5da8dee740 20 client.27782 trim_cache size 0 max 16384 2024-03-21T09:21:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 Processor -- start 2024-03-21T09:21:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 -- start start 2024-03-21T09:21:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 0x55d773f73be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6bbb0 0x55d773f74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773f6c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01b7510 con 0x55d773f6bbb0 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01b5010 con 0x55d773f6b290 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.261+0000 7f5da8dee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01b30d0 con 0x55d773f74790 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 0x55d773f73be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 0x55d773f73be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44896/0 (socket says 172.21.15.37:44896) 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 -- 172.21.15.37:0/133627448 learned_addr learned my addr 172.21.15.37:0/133627448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773f6c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6bbb0 0x55d773f74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 -- 172.21.15.37:0/133627448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 msgr2=0x55d773f6c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773f6c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 -- 172.21.15.37:0/133627448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6bbb0 msgr2=0x55d773f74120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6bbb0 0x55d773f74120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 -- 172.21.15.37:0/133627448 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d773f65720 con 0x55d773f6b290 2024-03-21T09:21:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6bbb0 0x55d773f74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:21:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.262+0000 7f5d9f400640 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 0x55d773f73be0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5d90004220 tx=0x7f5d9001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f332e0c355d1d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5d8b400640 1 -- 172.21.15.37:0/133627448 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90005020 con 0x55d773f6b290 2024-03-21T09:21:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5d8b400640 1 -- 172.21.15.37:0/133627448 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d900047a0 con 0x55d773f6b290 2024-03-21T09:21:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5d8b400640 1 -- 172.21.15.37:0/133627448 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9002a8e0 con 0x55d773f6b290 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5da8dee740 1 -- 172.21.15.37:0/133627448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 msgr2=0x55d773f73be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5da8dee740 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 0x55d773f73be0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5d90004220 tx=0x7f5d9001b000 comp rx=0 tx=0).stop 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5da8dee740 1 -- 172.21.15.37:0/133627448 shutdown_connections 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.263+0000 7f5da8dee740 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773f6c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.264+0000 7f5da8dee740 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6bbb0 0x55d773f74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.264+0000 7f5da8dee740 1 --2- 172.21.15.37:0/133627448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6b290 0x55d773f73be0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.264+0000 7f5da8dee740 1 -- 172.21.15.37:0/133627448 >> 172.21.15.37:0/133627448 conn(0x55d773f659c0 msgr2=0x55d773f6c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.264+0000 7f5da8dee740 1 -- 172.21.15.37:0/133627448 shutdown_connections 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.264+0000 7f5da8dee740 1 -- 172.21.15.37:0/133627448 wait complete. 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.264+0000 7f5da8dee740 1 -- start start 2024-03-21T09:21:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 Processor -- start 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 0x55d773f79920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773fdffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01b36d0 con 0x55d773f6b290 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01b49b0 con 0x55d773f6bbb0 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01b3cf0 con 0x55d773f74790 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5da8dee740 10 client.0 _ll_register_callbacks cb 0x7f5d8c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.265+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 0x55d773f79920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 0x55d773f79920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35342/0 (socket says 172.21.15.37:35342) 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9f400640 1 -- 172.21.15.37:0/2457869611 learned_addr learned my addr 172.21.15.37:0/2457869611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9ea00640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773fdffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 -- 172.21.15.37:0/2457869611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 msgr2=0x55d773f79e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f79e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 -- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 msgr2=0x55d773f79920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 0x55d773f79920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 -- 172.21.15.37:0/2457869611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d773fe0500 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 0x55d773f79920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d9fe00640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773fdffc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ec0 tx=0x7f5d98012780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76241a701ff8b5cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.266+0000 7f5d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5d88c00640 1 -- 172.21.15.37:0/2457869611 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802b060 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5d88c00640 1 -- 172.21.15.37:0/2457869611 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5d98012890 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d900047a0 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5d88c00640 1 -- 172.21.15.37:0/2457869611 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d98032050 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 10 client.27811 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d773fe16b0 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5d88c00640 1 -- 172.21.15.37:0/2457869611 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d900047a0 con 0x55d773f74790 2024-03-21T09:21:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5d88c00640 1 -- 172.21.15.37:0/2457869611 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5d9800f040 con 0x55d773f74790 2024-03-21T09:21:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 10 client.27811 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 10 client.27811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 20 client.27811 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773fe19f0 0x55d773fe3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 4 client.27811 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:21:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.267+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d773ed7740 con 0x55d773fe19f0 2024-03-21T09:21:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.268+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773fe19f0 0x55d773fe3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.268+0000 7f5d9f400640 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773fe19f0 0x55d773fe3de0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34639f841f6d6131 server_cookie=5c9c63ce4c48eac9 in_seq=0 out_seq=0 2024-03-21T09:21:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.268+0000 7f5d88c00640 10 client.27811 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.270+0000 7f5d88c00640 1 -- 172.21.15.37:0/2457869611 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d773ed7740 con 0x55d773fe19f0 2024-03-21T09:21:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.270+0000 7f5d88c00640 10 client.27811 handle_command_reply: tid=0 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.270+0000 7f5da8dee740 1 client.27811 shutdown 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.270+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 msgr2=0x55d773fdffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.270+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773fdffc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ec0 tx=0x7f5d98012780 comp rx=0 tx=0).stop 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 shutdown_connections 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d773f74790 0x55d773fdffc0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55d773fe19f0 0x55d773fe3de0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d773f6bbb0 0x55d773f79e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 --2- 172.21.15.37:0/2457869611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d773f6b290 0x55d773f79920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 >> 172.21.15.37:0/2457869611 conn(0x55d773f659c0 msgr2=0x55d773f6c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5d88c00640 0 client.27811 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 shutdown_connections 2024-03-21T09:21:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 1 -- 172.21.15.37:0/2457869611 wait complete. 2024-03-21T09:21:29.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.271+0000 7f5da8dee740 20 client.27811 trim_cache size 0 max 16384 2024-03-21T09:21:29.271 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:21:29.271 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:21:29.271 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "5a33bf69-1ac0-4c60-909d-e0ec04ffd558", 2024-03-21T09:21:29.271 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:21:29.271 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d840a3110 msgr2=0x7f5d840a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d840a3110 0x7f5d840a55d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55d773f6ad60 tx=0x7f5d90009260 comp rx=0 tx=0).stop 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 msgr2=0x7f5da019ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da019ea20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009780 tx=0x7f5d8c004430 comp rx=0 tx=0).stop 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 shutdown_connections 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5da0112330 0x7f5da01a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da00764f0 0x7f5da019ea20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5d840a3110 0x7f5d840a55d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 --2- 172.21.15.37:0/2910194710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0075bd0 0x7f5da019e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 >> 172.21.15.37:0/2910194710 conn(0x7f5da00ff140 msgr2=0x7f5da0113a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 shutdown_connections 2024-03-21T09:21:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.272+0000 7f5da4a00640 1 -- 172.21.15.37:0/2910194710 wait complete. 2024-03-21T09:21:29.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:21:29.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:21:29.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 -- 172.21.15.37:0/1410525905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 msgr2=0x7f242010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 --2- 172.21.15.37:0/1410525905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f242010ded0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2418009ca0 tx=0x7f241802c190 comp rx=0 tx=0).stop 2024-03-21T09:21:29.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 -- 172.21.15.37:0/1410525905 shutdown_connections 2024-03-21T09:21:29.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 --2- 172.21.15.37:0/1410525905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f242010ded0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 --2- 172.21.15.37:0/1410525905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2420102ff0 0x7f242010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 --2- 172.21.15.37:0/1410525905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24201026d0 0x7f2420102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 -- 172.21.15.37:0/1410525905 >> 172.21.15.37:0/1410525905 conn(0x7f2420077130 msgr2=0x7f24200fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 -- 172.21.15.37:0/1410525905 shutdown_connections 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.512+0000 7f241f200640 1 -- 172.21.15.37:0/1410525905 wait complete. 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.514+0000 7f241f200640 1 Processor -- start 2024-03-21T09:21:29.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.514+0000 7f241f200640 1 -- start start 2024-03-21T09:21:29.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24201026d0 0x7f242019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2420102ff0 0x7f242019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f2420197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201117f0 con 0x7f24201026d0 2024-03-21T09:21:29.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420111670 con 0x7f242010baa0 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420111970 con 0x7f2420102ff0 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24201026d0 0x7f242019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24201026d0 0x7f242019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35374/0 (socket says 172.21.15.37:35374) 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241de00640 1 -- 172.21.15.37:0/2386966591 learned_addr learned my addr 172.21.15.37:0/2386966591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f2420197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241d400640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2420102ff0 0x7f242019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241e800640 1 -- 172.21.15.37:0/2386966591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2420102ff0 msgr2=0x7f242019de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241e800640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2420102ff0 0x7f242019de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241e800640 1 -- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24201026d0 msgr2=0x7f242019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241e800640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24201026d0 0x7f242019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.515+0000 7f241e800640 1 -- 172.21.15.37:0/2386966591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420198230 con 0x7f242010baa0 2024-03-21T09:21:29.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.516+0000 7f241e800640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f2420197ab0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2418009bf0 tx=0x7f2418035d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.516+0000 7f2416a00640 1 -- 172.21.15.37:0/2386966591 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418039040 con 0x7f242010baa0 2024-03-21T09:21:29.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.516+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24201984c0 con 0x7f242010baa0 2024-03-21T09:21:29.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.516+0000 7f2416a00640 1 -- 172.21.15.37:0/2386966591 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2418044070 con 0x7f242010baa0 2024-03-21T09:21:29.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.516+0000 7f2416a00640 1 -- 172.21.15.37:0/2386966591 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180328d0 con 0x7f242010baa0 2024-03-21T09:21:29.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.516+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24201a47c0 con 0x7f242010baa0 2024-03-21T09:21:29.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.519+0000 7f2416a00640 1 -- 172.21.15.37:0/2386966591 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2418032a70 con 0x7f242010baa0 2024-03-21T09:21:29.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.520+0000 7f2427f9b740 1 Processor -- start 2024-03-21T09:21:29.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.520+0000 7f2427f9b740 1 -- start start 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.520+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.520+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f70f0 0x562dec5f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec602850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f2427f9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec55b250 con 0x562dec5f70f0 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f2427f9b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec503a30 con 0x562dec5f7dd0 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f2427f9b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec5572f0 con 0x562dec600460 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44924/0 (socket says 172.21.15.37:44924) 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 -- 172.21.15.37:0/157046759 learned_addr learned my addr 172.21.15.37:0/157046759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241de00640 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f70f0 0x562dec5f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec602850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 -- 172.21.15.37:0/157046759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 msgr2=0x562dec602850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec602850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 -- 172.21.15.37:0/157046759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f70f0 msgr2=0x562dec5f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f70f0 0x562dec5f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.521+0000 7f241d400640 1 -- 172.21.15.37:0/157046759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dec4ee690 con 0x562dec5f7dd0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f241d400640 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5f6b20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f240c013740 tx=0x7f240c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6625d13b61015d13 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f2416a00640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f24040a3110 0x7f24040a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f240b400640 1 -- 172.21.15.37:0/157046759 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c013130 con 0x562dec5f7dd0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f240b400640 1 -- 172.21.15.37:0/157046759 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f240c013cc0 con 0x562dec5f7dd0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f240b400640 1 -- 172.21.15.37:0/157046759 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c0047a0 con 0x562dec5f7dd0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f2416a00640 1 -- 172.21.15.37:0/2386966591 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f24180f1bc0 con 0x7f242010baa0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.522+0000 7f241de00640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f24040a3110 0x7f24040a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 -- 172.21.15.37:0/157046759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 msgr2=0x562dec5f6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5f6b20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f240c013740 tx=0x7f240c0096c0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 -- 172.21.15.37:0/157046759 shutdown_connections 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec602850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f70f0 0x562dec5f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 --2- 172.21.15.37:0/157046759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5f6b20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 -- 172.21.15.37:0/157046759 >> 172.21.15.37:0/157046759 conn(0x562dec5be200 msgr2=0x562dec5f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 -- 172.21.15.37:0/157046759 shutdown_connections 2024-03-21T09:21:29.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f2427f9b740 1 -- 172.21.15.37:0/157046759 wait complete. 2024-03-21T09:21:29.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.523+0000 7f241de00640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f24040a3110 0x7f24040a55d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x562dec5f8320 tx=0x7f2410005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.524+0000 7f2427f9b740 1 -- start start 2024-03-21T09:21:29.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.524+0000 7f2427f9b740 1 Processor -- start 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec600460 0x562dec5fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec603270 con 0x562dec5f7dd0 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec516e80 con 0x562dec600460 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec6033f0 con 0x562dec5f70f0 2024-03-21T09:21:29.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f2427f9b740 10 client.0 _ll_register_callbacks cb 0x562dec529130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f241de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f241de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57322/0 (socket says 172.21.15.37:57322) 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f241de00640 1 -- 172.21.15.37:0/2482177842 learned_addr learned my addr 172.21.15.37:0/2482177842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f241e800640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec600460 0x562dec5fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.525+0000 7f241d400640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241de00640 1 -- 172.21.15.37:0/2482177842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec600460 msgr2=0x562dec5fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241de00640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec600460 0x562dec5fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241de00640 1 -- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 msgr2=0x562dec5fceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241de00640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5fceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241de00640 1 -- 172.21.15.37:0/2482177842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dec3397d0 con 0x562dec5f70f0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241e800640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec600460 0x562dec5fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f241de00640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5ffb20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f241001ea00 tx=0x7f2410004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d711fb325edcd4e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f23ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.526+0000 7f23ffe00640 1 -- 172.21.15.37:0/2482177842 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241002e060 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f23ffe00640 1 -- 172.21.15.37:0/2482177842 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2410018030 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f240c013cc0 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f23ffe00640 1 -- 172.21.15.37:0/2482177842 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2410004020 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f2427f9b740 10 client.27823 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562dec26f3f0 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f23ffe00640 1 -- 172.21.15.37:0/2482177842 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f240c013cc0 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f23ffe00640 1 -- 172.21.15.37:0/2482177842 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f241000f510 con 0x562dec5f70f0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f2427f9b740 10 client.27823 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.527+0000 7f2427f9b740 10 client.27823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.528+0000 7f2427f9b740 20 client.27823 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.528+0000 7f2427f9b740 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec610390 0x562dec612810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.528+0000 7f2427f9b740 4 client.27823 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.528+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dec568790 con 0x562dec610390 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.528+0000 7f241d400640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec610390 0x562dec612810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.529+0000 7f241d400640 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec610390 0x562dec612810 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3988952e05d2679c server_cookie=a3a66139f1db7dd2 in_seq=0 out_seq=0 2024-03-21T09:21:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.529+0000 7f23ffe00640 10 client.27823 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.529+0000 7f23ffe00640 1 -- 172.21.15.37:0/2482177842 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562dec568790 con 0x562dec610390 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.529+0000 7f23ffe00640 10 client.27823 handle_command_reply: tid=0 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.529+0000 7f2427f9b740 1 client.27823 shutdown 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 msgr2=0x562dec5ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5ffb20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f241001ea00 tx=0x7f2410004730 comp rx=0 tx=0).stop 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 shutdown_connections 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec610390 0x562dec612810 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec600460 0x562dec5fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f23ffe00640 0 client.27823 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 --2- 172.21.15.37:0/2482177842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5ffb20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 >> 172.21.15.37:0/2482177842 conn(0x562dec5be200 msgr2=0x562dec5f3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 shutdown_connections 2024-03-21T09:21:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 1 -- 172.21.15.37:0/2482177842 wait complete. 2024-03-21T09:21:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.530+0000 7f2427f9b740 20 client.27823 trim_cache size 0 max 16384 2024-03-21T09:21:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.548+0000 7f2427f9b740 1 Processor -- start 2024-03-21T09:21:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 -- start start 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 0x562dec5ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec601850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201b0de0 con 0x562dec5f7dd0 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201ae8e0 con 0x562dec5f70f0 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f2427f9b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201ac9a0 con 0x562dec600460 2024-03-21T09:21:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f241de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 0x562dec5ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f241de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 0x562dec5ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:44936/0 (socket says 172.21.15.37:44936) 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f241de00640 1 -- 172.21.15.37:0/1012125732 learned_addr learned my addr 172.21.15.37:0/1012125732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f241e800640 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec601850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.549+0000 7f241d400640 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241de00640 1 -- 172.21.15.37:0/1012125732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 msgr2=0x562dec601850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241de00640 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec601850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241de00640 1 -- 172.21.15.37:0/1012125732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 msgr2=0x562dec5ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241de00640 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241de00640 1 -- 172.21.15.37:0/1012125732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dec22f0f0 con 0x562dec5f70f0 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241d400640 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f241de00640 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 0x562dec5ff5e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2410013010 tx=0x7f24100092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5d4a5ace0784e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f240aa00640 1 -- 172.21.15.37:0/1012125732 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241003e070 con 0x562dec5f70f0 2024-03-21T09:21:29.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f240aa00640 1 -- 172.21.15.37:0/1012125732 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24100041c0 con 0x562dec5f70f0 2024-03-21T09:21:29.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.550+0000 7f240aa00640 1 -- 172.21.15.37:0/1012125732 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410003b00 con 0x562dec5f70f0 2024-03-21T09:21:29.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 -- 172.21.15.37:0/1012125732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 msgr2=0x562dec5ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 0x562dec5ff5e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2410013010 tx=0x7f24100092a0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 -- 172.21.15.37:0/1012125732 shutdown_connections 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec600460 0x562dec601850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec5f7dd0 0x562dec5ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1012125732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f70f0 0x562dec5ff5e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 -- 172.21.15.37:0/1012125732 >> 172.21.15.37:0/1012125732 conn(0x562dec5be200 msgr2=0x562dec5f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 -- 172.21.15.37:0/1012125732 shutdown_connections 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.551+0000 7f2427f9b740 1 -- 172.21.15.37:0/1012125732 wait complete. 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 -- start start 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 Processor -- start 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5fa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec600460 0x562dec66f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201acfa0 con 0x562dec600460 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201ae280 con 0x562dec5f7dd0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201ad5c0 con 0x562dec5f70f0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f2427f9b740 10 client.0 _ll_register_callbacks cb 0x7f241001e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f241de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f241e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec600460 0x562dec66f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f241d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f241de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57336/0 (socket says 172.21.15.37:57336) 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.552+0000 7f241de00640 1 -- 172.21.15.37:0/1160729079 learned_addr learned my addr 172.21.15.37:0/1160729079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241de00640 1 -- 172.21.15.37:0/1160729079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 msgr2=0x562dec5fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241de00640 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5fa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241de00640 1 -- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec600460 msgr2=0x562dec66f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241de00640 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec600460 0x562dec66f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241de00640 1 -- 172.21.15.37:0/1160729079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dec66f5f0 con 0x562dec5f70f0 2024-03-21T09:21:29.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241d400640 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f241de00640 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5f9c30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f24100097f0 tx=0x7f2410009cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a7fd10dd0ed575a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f23ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f23ffe00640 1 -- 172.21.15.37:0/1160729079 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241003e030 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24100041c0 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.553+0000 7f23ffe00640 1 -- 172.21.15.37:0/1160729079 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2410024040 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f23ffe00640 1 -- 172.21.15.37:0/1160729079 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241004b050 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 10 client.27829 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562dec670870 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f23ffe00640 1 -- 172.21.15.37:0/1160729079 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24100041c0 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f23ffe00640 1 -- 172.21.15.37:0/1160729079 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2410004730 con 0x562dec5f70f0 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 10 client.27829 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 10 client.27829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:29.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 20 client.27829 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec670dd0 0x562dec6731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 4 client.27829 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562dec562ae0 con 0x562dec670dd0 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.554+0000 7f241d400640 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec670dd0 0x562dec6731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.555+0000 7f241d400640 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec670dd0 0x562dec6731c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3171881663759005 server_cookie=57b3597a20a4ba in_seq=0 out_seq=0 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.555+0000 7f23ffe00640 10 client.27829 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.555+0000 7f23ffe00640 1 -- 172.21.15.37:0/1160729079 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562dec562ae0 con 0x562dec670dd0 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.555+0000 7f23ffe00640 10 client.27829 handle_command_reply: tid=0 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.555+0000 7f2427f9b740 1 client.27829 shutdown 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 msgr2=0x562dec5f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5f9c30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f24100097f0 tx=0x7f2410009cf0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 shutdown_connections 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dec600460 0x562dec66f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x562dec670dd0 0x562dec6731c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562dec5f7dd0 0x562dec5fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 --2- 172.21.15.37:0/1160729079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dec5f70f0 0x562dec5f9c30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 >> 172.21.15.37:0/1160729079 conn(0x562dec5be200 msgr2=0x562dec5fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f23ffe00640 0 client.27829 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 shutdown_connections 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 1 -- 172.21.15.37:0/1160729079 wait complete. 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.556+0000 7f2427f9b740 20 client.27829 trim_cache size 0 max 16384 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:21:29.559 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout: "5a33bf69-1ac0-4c60-909d-e0ec04ffd558": { 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "5a33bf69-1ac0-4c60-909d-e0ec04ffd558", 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:21:29.560 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f24040a3110 msgr2=0x7f24040a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f24040a3110 0x7f24040a55d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x562dec5f8320 tx=0x7f2410005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 msgr2=0x7f2420197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f2420197ab0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2418009bf0 tx=0x7f2418035d80 comp rx=0 tx=0).stop 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 shutdown_connections 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f24040a3110 0x7f24040a55d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f242010baa0 0x7f2420197ab0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2420102ff0 0x7f242019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 --2- 172.21.15.37:0/2386966591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24201026d0 0x7f242019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 >> 172.21.15.37:0/2386966591 conn(0x7f2420077130 msgr2=0x7f24200fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 shutdown_connections 2024-03-21T09:21:29.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:29.557+0000 7f241f200640 1 -- 172.21.15.37:0/2386966591 wait complete. 2024-03-21T09:21:29.572 INFO:tasks.cephfs.filesystem:scrub status for tag:5a33bf69-1ac0-4c60-909d-e0ec04ffd558 - {'path': '/', 'tag': '5a33bf69-1ac0-4c60-909d-e0ec04ffd558', 'options': 'recursive,force'} 2024-03-21T09:21:29.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:29 smithi037 ceph-mon[31830]: pgmap v288: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:29.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:29 smithi053 ceph-mon[36169]: pgmap v288: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:29.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:29 smithi071 ceph-mon[36597]: pgmap v288: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:30.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:30 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:21:30.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:30 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:30.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:30 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:21:30.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:30 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:30.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:30 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:21:30.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:30 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:30.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:30 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:21:30.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:30 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:30.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:30 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:21:30.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:30 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:30.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:30 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:21:30.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:30 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:21:32.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:32 smithi071 ceph-mon[36597]: pgmap v289: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:32.355 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:32 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:21:32.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:32 smithi037 ceph-mon[31830]: pgmap v289: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:32.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:32 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:21:32.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:32 smithi053 ceph-mon[36169]: pgmap v289: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:32.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:32 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:21:33.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:33 smithi037 ceph-mon[31830]: pgmap v290: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-03-21T09:21:33.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:33 smithi053 ceph-mon[36169]: pgmap v290: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-03-21T09:21:33.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:33 smithi071 ceph-mon[36597]: pgmap v290: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-03-21T09:21:34.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:34 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:21:34.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:34 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:34 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:34 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:34 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:34 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:21:34.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:34 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:34 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:34 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:34 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:34 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:21:34.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:34 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:34 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:34 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:34.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:34 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:35 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:35 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:35 smithi071 ceph-mon[36597]: pgmap v291: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6 op/s 2024-03-21T09:21:35.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:35 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:35 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:35 smithi037 ceph-mon[31830]: pgmap v291: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6 op/s 2024-03-21T09:21:35.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:35 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:35 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:35.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:35 smithi053 ceph-mon[36169]: pgmap v291: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6 op/s 2024-03-21T09:21:36.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:36 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:21:36.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:36 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:21:36.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:36 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:36.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:36 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:21:36.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:36 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:21:36.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:36 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:36.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:36 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:21:36.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:36 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:21:36.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:36 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:21:37.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:37 smithi071 ceph-mon[36597]: pgmap v292: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-03-21T09:21:37.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:37 smithi037 ceph-mon[31830]: pgmap v292: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-03-21T09:21:37.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:37 smithi053 ceph-mon[36169]: pgmap v292: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-03-21T09:21:39.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:39 smithi037 ceph-mon[31830]: pgmap v293: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:39.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:39 smithi053 ceph-mon[36169]: pgmap v293: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:39.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:39 smithi071 ceph-mon[36597]: pgmap v293: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:41.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:41 smithi037 ceph-mon[31830]: pgmap v294: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:41.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:41 smithi053 ceph-mon[36169]: pgmap v294: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:41.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:41 smithi071 ceph-mon[36597]: pgmap v294: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-03-21T09:21:43.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:43 smithi037 ceph-mon[31830]: pgmap v295: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:21:43.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:43 smithi053 ceph-mon[36169]: pgmap v295: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:21:43.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:43 smithi071 ceph-mon[36597]: pgmap v295: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T09:21:45.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:45 smithi037 ceph-mon[31830]: pgmap v296: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:45.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:45 smithi053 ceph-mon[36169]: pgmap v296: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:45.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:45 smithi071 ceph-mon[36597]: pgmap v296: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:47.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:47 smithi037 ceph-mon[31830]: pgmap v297: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:47.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:47 smithi053 ceph-mon[36169]: pgmap v297: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:47.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:47 smithi071 ceph-mon[36597]: pgmap v297: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:49.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:49 smithi037 ceph-mon[31830]: pgmap v298: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:49.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:49 smithi053 ceph-mon[36169]: pgmap v298: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:49.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:49 smithi071 ceph-mon[36597]: pgmap v298: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:51.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:51 smithi037 ceph-mon[31830]: pgmap v299: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:51.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:51 smithi053 ceph-mon[36169]: pgmap v299: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:51.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:51 smithi071 ceph-mon[36597]: pgmap v299: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:53.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:53 smithi037 ceph-mon[31830]: pgmap v300: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:53.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:53 smithi053 ceph-mon[36169]: pgmap v300: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:53.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:53 smithi071 ceph-mon[36597]: pgmap v300: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:55.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:55 smithi037 ceph-mon[31830]: pgmap v301: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:55.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:55 smithi053 ceph-mon[36169]: pgmap v301: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:55.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:55 smithi071 ceph-mon[36597]: pgmap v301: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:57.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:57 smithi037 ceph-mon[31830]: pgmap v302: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:57.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:57 smithi053 ceph-mon[36169]: pgmap v302: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:57.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:57 smithi071 ceph-mon[36597]: pgmap v302: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:21:59.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:21:59.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:21:59.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:21:59 smithi037 ceph-mon[31830]: pgmap v303: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:59.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:21:59 smithi053 ceph-mon[36169]: pgmap v303: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 -- 172.21.15.37:0/3661400467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 msgr2=0x7ff47c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 --2- 172.21.15.37:0/3661400467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c100460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff468009900 tx=0x7ff46802bc30 comp rx=0 tx=0).stop 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 -- 172.21.15.37:0/3661400467 shutdown_connections 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 --2- 172.21.15.37:0/3661400467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c104910 0x7ff47c104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 --2- 172.21.15.37:0/3661400467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c100460 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 --2- 172.21.15.37:0/3661400467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff47c0694d0 0x7ff47c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 -- 172.21.15.37:0/3661400467 >> 172.21.15.37:0/3661400467 conn(0x7ff47c076aa0 msgr2=0x7ff47c0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 -- 172.21.15.37:0/3661400467 shutdown_connections 2024-03-21T09:21:59.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.811+0000 7ff483e00640 1 -- 172.21.15.37:0/3661400467 wait complete. 2024-03-21T09:21:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.812+0000 7ff483e00640 1 Processor -- start 2024-03-21T09:21:59.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.812+0000 7ff483e00640 1 -- start start 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff483e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 0x7ff47c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff483e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff483e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff47c104910 0x7ff47c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff483e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c110930 con 0x7ff47c104910 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff483e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1107b0 con 0x7ff47c0694d0 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff483e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c110ab0 con 0x7ff47c069940 2024-03-21T09:21:59.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff480a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff481400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 0x7ff47c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff480a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40062/0 (socket says 172.21.15.37:40062) 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff481400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 0x7ff47c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58172/0 (socket says 172.21.15.37:58172) 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff480a00640 1 -- 172.21.15.37:0/2236323932 learned_addr learned my addr 172.21.15.37:0/2236323932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.813+0000 7ff481e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff47c104910 0x7ff47c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff481400640 1 -- 172.21.15.37:0/2236323932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 msgr2=0x7ff47c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff481400640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff481400640 1 -- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff47c104910 msgr2=0x7ff47c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff481400640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff47c104910 0x7ff47c19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff481400640 1 -- 172.21.15.37:0/2236323932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c19cf00 con 0x7ff47c0694d0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff481400640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 0x7ff47c197f10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff47000ec80 tx=0x7ff47000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff47a000640 1 -- 172.21.15.37:0/2236323932 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470003b70 con 0x7ff47c0694d0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff47a000640 1 -- 172.21.15.37:0/2236323932 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff470003d10 con 0x7ff47c0694d0 2024-03-21T09:21:59.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff47a000640 1 -- 172.21.15.37:0/2236323932 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470010a30 con 0x7ff47c0694d0 2024-03-21T09:21:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c19d1f0 con 0x7ff47c0694d0 2024-03-21T09:21:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.814+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c1a4a80 con 0x7ff47c0694d0 2024-03-21T09:21:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.817+0000 7ff47a000640 1 -- 172.21.15.37:0/2236323932 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff470003ec0 con 0x7ff47c0694d0 2024-03-21T09:21:59.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 Processor -- start 2024-03-21T09:21:59.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 -- start start 2024-03-21T09:21:59.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15bee0 0x56438a15c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15c8a0 0x56438a1732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438a0c3250 con 0x56438a15bee0 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438a06ba30 con 0x56438a15c8a0 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.818+0000 7ff486a1e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438a0bf2f0 con 0x56438a16b670 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15bee0 0x56438a15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40074/0 (socket says 172.21.15.37:40074) 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 -- 172.21.15.37:0/2224497433 learned_addr learned my addr 172.21.15.37:0/2224497433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff480a00640 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15c8a0 0x56438a1732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 -- 172.21.15.37:0/2224497433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15c8a0 msgr2=0x56438a1732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15c8a0 0x56438a1732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 -- 172.21.15.37:0/2224497433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15bee0 msgr2=0x56438a15c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15bee0 0x56438a15c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 -- 172.21.15.37:0/2224497433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56438a056690 con 0x56438a16b670 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481400640 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15bee0 0x56438a15c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff481e00640 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16ba50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff474009b90 tx=0x7ff4740117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7184e8fbd77961 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.819+0000 7ff47a000640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4640a3090 0x7ff4640a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff46e600640 1 -- 172.21.15.37:0/2224497433 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474012450 con 0x56438a16b670 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff46e600640 1 -- 172.21.15.37:0/2224497433 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff474012ad0 con 0x56438a16b670 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff46e600640 1 -- 172.21.15.37:0/2224497433 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400fab0 con 0x56438a16b670 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff47a000640 1 -- 172.21.15.37:0/2236323932 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff4700d03d0 con 0x7ff47c0694d0 2024-03-21T09:21:59.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff481400640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4640a3090 0x7ff4640a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff486a1e740 1 -- 172.21.15.37:0/2224497433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 msgr2=0x56438a16ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.820+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16ba50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff474009b90 tx=0x7ff4740117b0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff481400640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4640a3090 0x7ff4640a5550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56438a15bcf0 tx=0x7ff470004920 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 -- 172.21.15.37:0/2224497433 shutdown_connections 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15c8a0 0x56438a1732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15bee0 0x56438a15c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2224497433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16ba50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 -- 172.21.15.37:0/2224497433 >> 172.21.15.37:0/2224497433 conn(0x56438a126200 msgr2=0x56438a15add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 -- 172.21.15.37:0/2224497433 shutdown_connections 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 -- 172.21.15.37:0/2224497433 wait complete. 2024-03-21T09:21:59.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 -- start start 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.821+0000 7ff486a1e740 1 Processor -- start 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a170200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a170740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438a1739d0 con 0x56438a16b670 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438a07ee80 con 0x56438a15bee0 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438a173b50 con 0x56438a15c8a0 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff486a1e740 10 client.0 _ll_register_callbacks cb 0x56438a091130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff480a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a170200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff480a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a170200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40086/0 (socket says 172.21.15.37:40086) 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff480a00640 1 -- 172.21.15.37:0/681124641 learned_addr learned my addr 172.21.15.37:0/681124641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff481e00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.822+0000 7ff481e00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a170740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff480a00640 1 -- 172.21.15.37:0/681124641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 msgr2=0x56438a172e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff480a00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff480a00640 1 -- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 msgr2=0x56438a170740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff480a00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a170740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff480a00640 1 -- 172.21.15.37:0/681124641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564389ea17d0 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff480a00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a170200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff46802c240 tx=0x7ff46802c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978beb2667c9139f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff462a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff462a00640 1 -- 172.21.15.37:0/681124641 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46802cd40 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff474012ad0 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff462a00640 1 -- 172.21.15.37:0/681124641 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff4680046c0 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff462a00640 1 -- 172.21.15.37:0/681124641 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4680048c0 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff486a1e740 10 client.27841 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.823+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564389dd73f0 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff462a00640 1 -- 172.21.15.37:0/681124641 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff474012ad0 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff462a00640 1 -- 172.21.15.37:0/681124641 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff468046070 con 0x56438a15c8a0 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff486a1e740 10 client.27841 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff486a1e740 10 client.27841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:59.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff486a1e740 20 client.27841 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:59.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff486a1e740 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a180960 0x56438a182d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff486a1e740 4 client.27841 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:21:59.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56438a0d0790 con 0x56438a180960 2024-03-21T09:21:59.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.824+0000 7ff481e00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a180960 0x56438a182d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.825+0000 7ff481e00640 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a180960 0x56438a182d80 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5031c58ca985a4 server_cookie=380e4749829bf48b in_seq=0 out_seq=0 2024-03-21T09:21:59.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.825+0000 7ff462a00640 10 client.27841 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff462a00640 1 -- 172.21.15.37:0/681124641 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56438a0d0790 con 0x56438a180960 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff462a00640 10 client.27841 handle_command_reply: tid=0 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 client.27841 shutdown 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 msgr2=0x56438a170200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a170200 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff46802c240 tx=0x7ff46802c7e0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 shutdown_connections 2024-03-21T09:21:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a170740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a170200 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a180960 0x56438a182d80 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 --2- 172.21.15.37:0/681124641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 >> 172.21.15.37:0/681124641 conn(0x56438a126200 msgr2=0x56438a159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff462a00640 0 client.27841 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 shutdown_connections 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 1 -- 172.21.15.37:0/681124641 wait complete. 2024-03-21T09:21:59.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.826+0000 7ff486a1e740 20 client.27841 trim_cache size 0 max 16384 2024-03-21T09:21:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 Processor -- start 2024-03-21T09:21:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 -- start start 2024-03-21T09:21:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15c8a0 0x56438a172e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1aee50 con 0x56438a15c8a0 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1ac950 con 0x56438a15bee0 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.844+0000 7ff486a1e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1aaa10 con 0x56438a16b670 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58200/0 (socket says 172.21.15.37:58200) 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 -- 172.21.15.37:0/562056853 learned_addr learned my addr 172.21.15.37:0/562056853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 -- 172.21.15.37:0/562056853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 msgr2=0x56438a16d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 -- 172.21.15.37:0/562056853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15c8a0 msgr2=0x56438a172e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff480a00640 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15c8a0 0x56438a172e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15c8a0 0x56438a172e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 -- 172.21.15.37:0/562056853 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564389d970f0 con 0x56438a15bee0 2024-03-21T09:21:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff480a00640 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15c8a0 0x56438a172e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:21:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff481e00640 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172930 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff47400f720 tx=0x7ff474005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282691d557c6d30d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.845+0000 7ff46dc00640 1 -- 172.21.15.37:0/562056853 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474009e60 con 0x56438a15bee0 2024-03-21T09:21:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff46dc00640 1 -- 172.21.15.37:0/562056853 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff47402eda0 con 0x56438a15bee0 2024-03-21T09:21:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff46dc00640 1 -- 172.21.15.37:0/562056853 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400f9e0 con 0x56438a15bee0 2024-03-21T09:21:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 -- 172.21.15.37:0/562056853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 msgr2=0x56438a172930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172930 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff47400f720 tx=0x7ff474005f90 comp rx=0 tx=0).stop 2024-03-21T09:21:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 -- 172.21.15.37:0/562056853 shutdown_connections 2024-03-21T09:21:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a16b670 0x56438a16d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a15c8a0 0x56438a172e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 --2- 172.21.15.37:0/562056853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a172930 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 -- 172.21.15.37:0/562056853 >> 172.21.15.37:0/562056853 conn(0x56438a126200 msgr2=0x56438a16a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 -- 172.21.15.37:0/562056853 shutdown_connections 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.846+0000 7ff486a1e740 1 -- 172.21.15.37:0/562056853 wait complete. 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 -- start start 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 Processor -- start 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a178ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a177040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a177670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1ab010 con 0x56438a16b670 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1ac2f0 con 0x56438a15bee0 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1ab630 con 0x56438a15c8a0 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.847+0000 7ff486a1e740 10 client.0 _ll_register_callbacks cb 0x7ff468009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a177040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff481e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a178ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff481e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a177670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a177040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40096/0 (socket says 172.21.15.37:40096) 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 -- 172.21.15.37:0/2129833016 learned_addr learned my addr 172.21.15.37:0/2129833016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 -- 172.21.15.37:0/2129833016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 msgr2=0x56438a178ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a178ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 -- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 msgr2=0x56438a177670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a177670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 -- 172.21.15.37:0/2129833016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56438a177e40 con 0x56438a15c8a0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff481e00640 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a177670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff480a00640 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a177040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff468036040 tx=0x7ff468000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678afbd10d0a6fde server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.848+0000 7ff462a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff462a00640 1 -- 172.21.15.37:0/2129833016 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46805a060 con 0x56438a15c8a0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff47402eda0 con 0x56438a15c8a0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff462a00640 1 -- 172.21.15.37:0/2129833016 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff468003d90 con 0x56438a15c8a0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff462a00640 1 -- 172.21.15.37:0/2129833016 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff468061050 con 0x56438a15c8a0 2024-03-21T09:21:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff486a1e740 10 client.27847 fetch_fsmap learned FSMap version 30 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56438a1dd290 con 0x56438a15c8a0 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff462a00640 1 -- 172.21.15.37:0/2129833016 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff47402eda0 con 0x56438a15c8a0 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.849+0000 7ff462a00640 1 -- 172.21.15.37:0/2129833016 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff468046040 con 0x56438a15c8a0 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff486a1e740 10 client.27847 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff486a1e740 10 client.27847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff486a1e740 20 client.27847 populate_metadata read hostname 'smithi037' 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a1dd940 0x56438a1dfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff486a1e740 4 client.27847 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56438a0caae0 con 0x56438a1dd940 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff481e00640 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a1dd940 0x56438a1dfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff481e00640 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a1dd940 0x56438a1dfd60 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5220c183807f955a server_cookie=2c9d21b94dae5d2f in_seq=0 out_seq=0 2024-03-21T09:21:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.850+0000 7ff462a00640 10 client.27847 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff462a00640 1 -- 172.21.15.37:0/2129833016 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56438a0caae0 con 0x56438a1dd940 2024-03-21T09:21:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff462a00640 10 client.27847 handle_command_reply: tid=0 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 client.27847 shutdown 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 msgr2=0x56438a177040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a177040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff468036040 tx=0x7ff468000bc0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 shutdown_connections 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a16b670 0x56438a177670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56438a15c8a0 0x56438a177040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x56438a1dd940 0x56438a1dfd60 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 --2- 172.21.15.37:0/2129833016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56438a15bee0 0x56438a178ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 >> 172.21.15.37:0/2129833016 conn(0x56438a126200 msgr2=0x56438a159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff462a00640 0 client.27847 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 shutdown_connections 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 1 -- 172.21.15.37:0/2129833016 wait complete. 2024-03-21T09:21:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.851+0000 7ff486a1e740 20 client.27847 trim_cache size 0 max 16384 2024-03-21T09:21:59.853 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:21:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:21:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:21:59.853 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:21:59.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4640a3090 msgr2=0x7ff4640a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4640a3090 0x7ff4640a5550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x56438a15bcf0 tx=0x7ff470004920 comp rx=0 tx=0).stop 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 msgr2=0x7ff47c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 0x7ff47c197f10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff47000ec80 tx=0x7ff47000c6a0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 shutdown_connections 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff47c104910 0x7ff47c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff47c069940 0x7ff47c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4640a3090 0x7ff4640a5550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 --2- 172.21.15.37:0/2236323932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff47c0694d0 0x7ff47c197f10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 >> 172.21.15.37:0/2236323932 conn(0x7ff47c076aa0 msgr2=0x7ff47c0fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 shutdown_connections 2024-03-21T09:21:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:21:59.852+0000 7ff483e00640 1 -- 172.21.15.37:0/2236323932 wait complete. 2024-03-21T09:21:59.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:21:59 smithi071 ceph-mon[36597]: pgmap v303: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:21:59.869 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:21:59.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 -- 172.21.15.37:0/232177188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 msgr2=0x7f663c10b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 --2- 172.21.15.37:0/232177188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c10b4e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6624009900 tx=0x7f662402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 -- 172.21.15.37:0/232177188 shutdown_connections 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 --2- 172.21.15.37:0/232177188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c10bb00 0x7f663c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 --2- 172.21.15.37:0/232177188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c10b4e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 --2- 172.21.15.37:0/232177188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 0x7f663c102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 -- 172.21.15.37:0/232177188 >> 172.21.15.37:0/232177188 conn(0x7f663c0fbf40 msgr2=0x7f663c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 -- 172.21.15.37:0/232177188 shutdown_connections 2024-03-21T09:22:00.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.088+0000 7f6641000640 1 -- 172.21.15.37:0/232177188 wait complete. 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 Processor -- start 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 -- start start 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 0x7f663c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c10bb00 0x7f663c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c111910 con 0x7f663c1026e0 2024-03-21T09:22:00.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c111790 con 0x7f663c10bb00 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.089+0000 7f6641000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c111a90 con 0x7f663c103000 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f663be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c10bb00 0x7f663c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f663b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 0x7f663c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40120/0 (socket says 172.21.15.37:40120) 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 -- 172.21.15.37:0/406886910 learned_addr learned my addr 172.21.15.37:0/406886910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 -- 172.21.15.37:0/406886910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c10bb00 msgr2=0x7f663c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c10bb00 0x7f663c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 -- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 msgr2=0x7f663c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 0x7f663c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 -- 172.21.15.37:0/406886910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c19f080 con 0x7f663c103000 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f663b400640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 0x7f663c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.090+0000 7f6632a00640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c19a5d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6624000c00 tx=0x7f6624004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.091+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624042020 con 0x7f663c103000 2024-03-21T09:22:00.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.091+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66240054e0 con 0x7f663c103000 2024-03-21T09:22:00.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.091+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624003af0 con 0x7f663c103000 2024-03-21T09:22:00.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.091+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c19f310 con 0x7f663c103000 2024-03-21T09:22:00.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.091+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c1a6c00 con 0x7f663c103000 2024-03-21T09:22:00.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.092+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c06a430 con 0x7f663c103000 2024-03-21T09:22:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.094+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6624045020 con 0x7f663c103000 2024-03-21T09:22:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.096+0000 7f6638c00640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66200a3080 0x7f66200a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.096+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f66240f1710 con 0x7f663c103000 2024-03-21T09:22:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.096+0000 7f663b400640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66200a3080 0x7f66200a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.097+0000 7f663b400640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66200a3080 0x7f66200a5540 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f662c005fd0 tx=0x7f662c009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.099+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f662404c020 con 0x7f663c103000 2024-03-21T09:22:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.342+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f663c1a6dd0 con 0x7f663c103000 2024-03-21T09:22:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.343+0000 7f6638c00640 1 -- 172.21.15.37:0/406886910 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f66240bdeb0 con 0x7f663c103000 2024-03-21T09:22:00.340 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:22:00.340 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:22:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:22:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.346+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66200a3080 msgr2=0x7f66200a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.346+0000 7f6641000640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66200a3080 0x7f66200a5540 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f662c005fd0 tx=0x7f662c009500 comp rx=0 tx=0).stop 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.346+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 msgr2=0x7f663c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.346+0000 7f6641000640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c19a5d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6624000c00 tx=0x7f6624004360 comp rx=0 tx=0).stop 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 shutdown_connections 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f66200a3080 0x7f66200a5540 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c10bb00 0x7f663c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c103000 0x7f663c19a5d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 --2- 172.21.15.37:0/406886910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c1026e0 0x7f663c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 >> 172.21.15.37:0/406886910 conn(0x7f663c0fbf40 msgr2=0x7f663c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 shutdown_connections 2024-03-21T09:22:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.347+0000 7f6641000640 1 -- 172.21.15.37:0/406886910 wait complete. 2024-03-21T09:22:00.364 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:22:00.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:22:00.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 -- 172.21.15.37:0/711478506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100420 msgr2=0x7fd448100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 --2- 172.21.15.37:0/711478506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100420 0x7fd448100800 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd438009900 tx=0x7fd43802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 -- 172.21.15.37:0/711478506 shutdown_connections 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 --2- 172.21.15.37:0/711478506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd448105820 0x7fd448105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 --2- 172.21.15.37:0/711478506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100d40 0x7fd448105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 --2- 172.21.15.37:0/711478506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100420 0x7fd448100800 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 -- 172.21.15.37:0/711478506 >> 172.21.15.37:0/711478506 conn(0x7fd448077130 msgr2=0x7fd4480fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 -- 172.21.15.37:0/711478506 shutdown_connections 2024-03-21T09:22:00.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.588+0000 7fd44cc00640 1 -- 172.21.15.37:0/711478506 wait complete. 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.589+0000 7fd44cc00640 1 Processor -- start 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.589+0000 7fd44cc00640 1 -- start start 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.589+0000 7fd44cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 0x7fd44819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.589+0000 7fd44cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100d40 0x7fd44819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd44cc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd448105820 0x7fd4481a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd44cc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd448117e80 con 0x7fd448105820 2024-03-21T09:22:00.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd44cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd448117d00 con 0x7fd448100420 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd44cc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd448118000 con 0x7fd448100d40 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd447400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 0x7fd44819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd447400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 0x7fd44819e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58232/0 (socket says 172.21.15.37:58232) 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100d40 0x7fd44819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd447400640 1 -- 172.21.15.37:0/1260551917 learned_addr learned my addr 172.21.15.37:0/1260551917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd447e00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd448105820 0x7fd4481a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd43ea00640 1 -- 172.21.15.37:0/1260551917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 msgr2=0x7fd44819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd43ea00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 0x7fd44819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd43ea00640 1 -- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd448105820 msgr2=0x7fd4481a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd43ea00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd448105820 0x7fd4481a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd43ea00640 1 -- 172.21.15.37:0/1260551917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4481a3430 con 0x7fd448100d40 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.590+0000 7fd447400640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 0x7fd44819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.591+0000 7fd43ea00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100d40 0x7fd44819e980 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd430009fd0 tx=0x7fd43000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.591+0000 7fd444c00640 1 -- 172.21.15.37:0/1260551917 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430004b20 con 0x7fd448100d40 2024-03-21T09:22:00.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.591+0000 7fd444c00640 1 -- 172.21.15.37:0/1260551917 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd430004cc0 con 0x7fd448100d40 2024-03-21T09:22:00.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.591+0000 7fd444c00640 1 -- 172.21.15.37:0/1260551917 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300107b0 con 0x7fd448100d40 2024-03-21T09:22:00.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.591+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4481a3720 con 0x7fd448100d40 2024-03-21T09:22:00.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.591+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4481aafb0 con 0x7fd448100d40 2024-03-21T09:22:00.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.594+0000 7fd444c00640 1 -- 172.21.15.37:0/1260551917 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd430005020 con 0x7fd448100d40 2024-03-21T09:22:00.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.594+0000 7fd450e6e740 1 Processor -- start 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.594+0000 7fd450e6e740 1 -- start start 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596138c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd450e6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56459609c2c0 con 0x564596139260 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd450e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564596044c90 con 0x564596141de0 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd450e6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564596098360 con 0x5645961388b0 2024-03-21T09:22:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd447e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd447400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd447e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596138c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40140/0 (socket says 172.21.15.37:40140) 2024-03-21T09:22:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.595+0000 7fd447e00640 1 -- 172.21.15.37:0/3852192771 learned_addr learned my addr 172.21.15.37:0/3852192771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447e00640 1 -- 172.21.15.37:0/3852192771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 msgr2=0x564596144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447e00640 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447e00640 1 -- 172.21.15.37:0/3852192771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 msgr2=0x564596141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447e00640 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447e00640 1 -- 172.21.15.37:0/3852192771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564596069080 con 0x5645961388b0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd444c00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd42c0a3110 0x7fd42c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447e00640 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596138c90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd44000dbb0 tx=0x7fd44000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b1e58e03e56285 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd444c00640 1 -- 172.21.15.37:0/1260551917 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd4300d03e0 con 0x7fd448100d40 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd447400640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd42c0a3110 0x7fd42c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd437e00640 1 -- 172.21.15.37:0/3852192771 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44000dea0 con 0x5645961388b0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.596+0000 7fd437e00640 1 -- 172.21.15.37:0/3852192771 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4400035f0 con 0x5645961388b0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd437e00640 1 -- 172.21.15.37:0/3852192771 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400116a0 con 0x5645961388b0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 -- 172.21.15.37:0/3852192771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 msgr2=0x564596138c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596138c90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd44000dbb0 tx=0x7fd44000f730 comp rx=0 tx=0).stop 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd447400640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd42c0a3110 0x7fd42c0a55d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x564596139660 tx=0x7fd43802e040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 -- 172.21.15.37:0/3852192771 shutdown_connections 2024-03-21T09:22:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3852192771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596138c90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 -- 172.21.15.37:0/3852192771 >> 172.21.15.37:0/3852192771 conn(0x564596132f80 msgr2=0x564596133360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.597+0000 7fd450e6e740 1 -- 172.21.15.37:0/3852192771 shutdown_connections 2024-03-21T09:22:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.598+0000 7fd450e6e740 1 -- 172.21.15.37:0/3852192771 wait complete. 2024-03-21T09:22:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.598+0000 7fd450e6e740 1 -- start start 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.598+0000 7fd450e6e740 1 Processor -- start 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645961388b0 0x564596141740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x56459613ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564596141de0 0x56459613f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564596144d50 con 0x564596139260 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645960580d0 con 0x5645961388b0 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564596144ed0 con 0x564596141de0 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd450e6e740 10 client.0 _ll_register_callbacks cb 0x56459605ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645961388b0 0x564596141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645961388b0 0x564596141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58250/0 (socket says 172.21.15.37:58250) 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564596141de0 0x56459613f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x56459613ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447400640 1 -- 172.21.15.37:0/1860604666 learned_addr learned my addr 172.21.15.37:0/1860604666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447e00640 1 -- 172.21.15.37:0/1860604666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645961388b0 msgr2=0x564596141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447e00640 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645961388b0 0x564596141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.599+0000 7fd447e00640 1 -- 172.21.15.37:0/1860604666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 msgr2=0x56459613ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd447e00640 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x56459613ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd447e00640 1 -- 172.21.15.37:0/1860604666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564595db03f0 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd447e00640 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564596141de0 0x56459613f0e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd44000dbb0 tx=0x7fd44000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce2e7fcb64e47e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd434c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd434c00640 1 -- 172.21.15.37:0/1860604666 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44002b060 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd434c00640 1 -- 172.21.15.37:0/1860604666 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd4400041f0 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4400035f0 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd434c00640 1 -- 172.21.15.37:0/1860604666 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd440032050 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd450e6e740 10 client.27865 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.600+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564595e7a7d0 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd434c00640 1 -- 172.21.15.37:0/1860604666 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4400035f0 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd434c00640 1 -- 172.21.15.37:0/1860604666 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd440011770 con 0x564596141de0 2024-03-21T09:22:00.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd450e6e740 10 client.27865 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd450e6e740 10 client.27865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd450e6e740 20 client.27865 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd450e6e740 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x564596151be0 0x564596153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd450e6e740 4 client.27865 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645960a9800 con 0x564596151be0 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.601+0000 7fd447400640 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x564596151be0 0x564596153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.602+0000 7fd434c00640 10 client.27865 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.602+0000 7fd447400640 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x564596151be0 0x564596153fd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2114310e64f0fd46 server_cookie=483804420a6051c1 in_seq=0 out_seq=0 2024-03-21T09:22:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd434c00640 1 -- 172.21.15.37:0/1860604666 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5645960a9800 con 0x564596151be0 2024-03-21T09:22:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd434c00640 10 client.27865 handle_command_reply: tid=0 2024-03-21T09:22:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 client.27865 shutdown 2024-03-21T09:22:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564596141de0 msgr2=0x56459613f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564596141de0 0x56459613f0e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd44000dbb0 tx=0x7fd44000ded0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 shutdown_connections 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564596141de0 0x56459613f0e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x564596151be0 0x564596153fd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x56459613ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 --2- 172.21.15.37:0/1860604666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645961388b0 0x564596141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 >> 172.21.15.37:0/1860604666 conn(0x564596132f80 msgr2=0x564596134cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.603+0000 7fd434c00640 0 client.27865 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.604+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 shutdown_connections 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.604+0000 7fd450e6e740 1 -- 172.21.15.37:0/1860604666 wait complete. 2024-03-21T09:22:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.604+0000 7fd450e6e740 20 client.27865 trim_cache size 0 max 16384 2024-03-21T09:22:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.617+0000 7fd450e6e740 1 Processor -- start 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 -- start start 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x56459614e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596139260 0x56459612e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596141de0 0x564596141200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481b74f0 con 0x564596141de0 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481b4ff0 con 0x564596139260 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd450e6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481b30b0 con 0x5645961388b0 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd447400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x56459614e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596139260 0x56459612e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd447400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x56459614e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40160/0 (socket says 172.21.15.37:40160) 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd447400640 1 -- 172.21.15.37:0/3954107807 learned_addr learned my addr 172.21.15.37:0/3954107807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd447e00640 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596141de0 0x564596141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd43ea00640 1 -- 172.21.15.37:0/3954107807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 msgr2=0x56459614e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd43ea00640 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x56459614e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd43ea00640 1 -- 172.21.15.37:0/3954107807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596141de0 msgr2=0x564596141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd43ea00640 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596141de0 0x564596141200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd43ea00640 1 -- 172.21.15.37:0/3954107807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645961597e0 con 0x564596139260 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd447400640 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x56459614e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.618+0000 7fd447e00640 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596141de0 0x564596141200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.619+0000 7fd43ea00640 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596139260 0x56459612e060 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd4300cd6d0 tx=0x7fd430004f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab801ac920bf5635 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.619+0000 7fd437400640 1 -- 172.21.15.37:0/3954107807 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43009beb0 con 0x564596139260 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.619+0000 7fd437400640 1 -- 172.21.15.37:0/3954107807 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd43009ca70 con 0x564596139260 2024-03-21T09:22:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.619+0000 7fd437400640 1 -- 172.21.15.37:0/3954107807 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43009bcd0 con 0x564596139260 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.619+0000 7fd450e6e740 1 -- 172.21.15.37:0/3954107807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596139260 msgr2=0x56459612e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.619+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596139260 0x56459612e060 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd4300cd6d0 tx=0x7fd430004f30 comp rx=0 tx=0).stop 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 -- 172.21.15.37:0/3954107807 shutdown_connections 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596141de0 0x564596141200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596139260 0x56459612e060 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 --2- 172.21.15.37:0/3954107807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x56459614e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 -- 172.21.15.37:0/3954107807 >> 172.21.15.37:0/3954107807 conn(0x564596132f80 msgr2=0x564596132160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 -- 172.21.15.37:0/3954107807 shutdown_connections 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 -- 172.21.15.37:0/3954107807 wait complete. 2024-03-21T09:22:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 -- start start 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.620+0000 7fd450e6e740 1 Processor -- start 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596156b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596154f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596155450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481b36b0 con 0x564596139260 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481b4990 con 0x564596141de0 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481b3cd0 con 0x5645961388b0 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd450e6e740 10 client.0 _ll_register_callbacks cb 0x7fd440015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd447400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd447e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd447400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40172/0 (socket says 172.21.15.37:40172) 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.621+0000 7fd447400640 1 -- 172.21.15.37:0/2224600079 learned_addr learned my addr 172.21.15.37:0/2224600079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447400640 1 -- 172.21.15.37:0/2224600079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 msgr2=0x564596155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447400640 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447400640 1 -- 172.21.15.37:0/2224600079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 msgr2=0x564596154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447400640 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447400640 1 -- 172.21.15.37:0/2224600079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645961559e0 con 0x5645961388b0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447e00640 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd447400640 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596156b70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd43802c2e0 tx=0x7fd438009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00016e56bc467b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd434c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd434c00640 1 -- 172.21.15.37:0/2224600079 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43802ccf0 con 0x5645961388b0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd43009ca70 con 0x5645961388b0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd434c00640 1 -- 172.21.15.37:0/2224600079 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd43803a040 con 0x5645961388b0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd434c00640 1 -- 172.21.15.37:0/2224600079 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd438065050 con 0x5645961388b0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd450e6e740 10 client.27871 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.622+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5645961adf40 con 0x5645961388b0 2024-03-21T09:22:00.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd434c00640 1 -- 172.21.15.37:0/2224600079 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd43009ca70 con 0x5645961388b0 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd434c00640 1 -- 172.21.15.37:0/2224600079 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd438003660 con 0x5645961388b0 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd450e6e740 10 client.27871 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd450e6e740 10 client.27871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd450e6e740 20 client.27871 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd450e6e740 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5645961ae6a0 0x5645961b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd450e6e740 4 client.27871 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645960a3b50 con 0x5645961ae6a0 2024-03-21T09:22:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.623+0000 7fd43ea00640 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5645961ae6a0 0x5645961b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd43ea00640 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5645961ae6a0 0x5645961b0ac0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8542b7242902b0a4 server_cookie=1d86f27217130c8a in_seq=0 out_seq=0 2024-03-21T09:22:00.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd434c00640 10 client.27871 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:00.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd434c00640 1 -- 172.21.15.37:0/2224600079 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645960a3b50 con 0x5645961ae6a0 2024-03-21T09:22:00.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd434c00640 10 client.27871 handle_command_reply: tid=0 2024-03-21T09:22:00.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 client.27871 shutdown 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 msgr2=0x564596156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596156b70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd43802c2e0 tx=0x7fd438009260 comp rx=0 tx=0).stop 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 shutdown_connections 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564596141de0 0x564596155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x5645961ae6a0 0x5645961b0ac0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564596139260 0x564596154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 --2- 172.21.15.37:0/2224600079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645961388b0 0x564596156b70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 >> 172.21.15.37:0/2224600079 conn(0x564596132f80 msgr2=0x564596132160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd434c00640 0 client.27871 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 shutdown_connections 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.624+0000 7fd450e6e740 1 -- 172.21.15.37:0/2224600079 wait complete. 2024-03-21T09:22:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd450e6e740 20 client.27871 trim_cache size 0 max 16384 2024-03-21T09:22:00.626 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:22:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd42c0a3110 msgr2=0x7fd42c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd42c0a3110 0x7fd42c0a55d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x564596139660 tx=0x7fd43802e040 comp rx=0 tx=0).stop 2024-03-21T09:22:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100d40 msgr2=0x7fd44819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100d40 0x7fd44819e980 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd430009fd0 tx=0x7fd43000ef90 comp rx=0 tx=0).stop 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 shutdown_connections 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd448105820 0x7fd4481a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd448100d40 0x7fd44819e980 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fd42c0a3110 0x7fd42c0a55d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 --2- 172.21.15.37:0/1260551917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd448100420 0x7fd44819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.625+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 >> 172.21.15.37:0/1260551917 conn(0x7fd448077130 msgr2=0x7fd4481031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.626+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 shutdown_connections 2024-03-21T09:22:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.626+0000 7fd44cc00640 1 -- 172.21.15.37:0/1260551917 wait complete. 2024-03-21T09:22:00.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:00 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/406886910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:22:00.640 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:22:00.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:22:00.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:00 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/406886910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:22:00.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.851+0000 7f2d67e00640 1 -- 172.21.15.37:0/1025435991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d68102f50 msgr2=0x7f2d6810b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.851+0000 7f2d67e00640 1 --2- 172.21.15.37:0/1025435991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d68102f50 0x7f2d6810b430 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009900 tx=0x7f2d5402bc50 comp rx=0 tx=0).stop 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 -- 172.21.15.37:0/1025435991 shutdown_connections 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 --2- 172.21.15.37:0/1025435991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d6810baa0 0x7f2d6810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 --2- 172.21.15.37:0/1025435991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d68102f50 0x7f2d6810b430 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 --2- 172.21.15.37:0/1025435991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 0x7f2d68102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 -- 172.21.15.37:0/1025435991 >> 172.21.15.37:0/1025435991 conn(0x7f2d680fbe90 msgr2=0x7f2d680fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 -- 172.21.15.37:0/1025435991 shutdown_connections 2024-03-21T09:22:00.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.852+0000 7f2d67e00640 1 -- 172.21.15.37:0/1025435991 wait complete. 2024-03-21T09:22:00.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.853+0000 7f2d67e00640 1 Processor -- start 2024-03-21T09:22:00.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.853+0000 7f2d67e00640 1 -- start start 2024-03-21T09:22:00.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 0x7f2d68100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102f50 0x7f2d681007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d6810baa0 0x7f2d68100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68111890 con 0x7f2d68102f50 2024-03-21T09:22:00.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68111710 con 0x7f2d6810baa0 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68111a10 con 0x7f2d68102630 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d66000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102f50 0x7f2d681007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d6810baa0 0x7f2d68100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 0x7f2d68100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d66000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102f50 0x7f2d681007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56462/0 (socket says 172.21.15.37:56462) 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.854+0000 7f2d66000640 1 -- 172.21.15.37:0/3418604095 learned_addr learned my addr 172.21.15.37:0/3418604095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d66a00640 1 -- 172.21.15.37:0/3418604095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d6810baa0 msgr2=0x7f2d68100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d66a00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d6810baa0 0x7f2d68100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d66a00640 1 -- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102f50 msgr2=0x7f2d681007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d66a00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102f50 0x7f2d681007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d66a00640 1 -- 172.21.15.37:0/3418604095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d681a6350 con 0x7f2d68102630 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d66a00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 0x7f2d68100280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9f0 tx=0x7f2d5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d5f400640 1 -- 172.21.15.37:0/3418604095 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800ccc0 con 0x7f2d68102630 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d5f400640 1 -- 172.21.15.37:0/3418604095 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d58005020 con 0x7f2d68102630 2024-03-21T09:22:00.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d5f400640 1 -- 172.21.15.37:0/3418604095 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580049e0 con 0x7f2d68102630 2024-03-21T09:22:00.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d681a6640 con 0x7f2d68102630 2024-03-21T09:22:00.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.855+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681a6b40 con 0x7f2d68102630 2024-03-21T09:22:00.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:00 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/406886910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:22:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 Processor -- start 2024-03-21T09:22:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 -- start start 2024-03-21T09:22:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 0x56345cf6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf75de0 0x56345cf78230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345ced02c0 con 0x56345cf75de0 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345ce78c90 con 0x56345cf6c8b0 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d7067f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345cecc360 con 0x56345cf6d260 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.859+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 0x56345cf6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 0x56345cf6cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58304/0 (socket says 172.21.15.37:58304) 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 -- 172.21.15.37:0/967082835 learned_addr learned my addr 172.21.15.37:0/967082835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d66000640 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf75de0 0x56345cf78230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 -- 172.21.15.37:0/967082835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 msgr2=0x56345cf75810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf75810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 -- 172.21.15.37:0/967082835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf75de0 msgr2=0x56345cf78230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf75de0 0x56345cf78230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 -- 172.21.15.37:0/967082835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56345ce9d080 con 0x56345cf6c8b0 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d67400640 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 0x56345cf6cc90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009f90 tx=0x7f2d60010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e7d17d813982b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d5f400640 1 -- 172.21.15.37:0/3418604095 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2d58005560 con 0x7f2d68102630 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d53e00640 1 -- 172.21.15.37:0/967082835 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60011700 con 0x56345cf6c8b0 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d53e00640 1 -- 172.21.15.37:0/967082835 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d60011d80 con 0x56345cf6c8b0 2024-03-21T09:22:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.860+0000 7f2d53e00640 1 -- 172.21.15.37:0/967082835 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60018380 con 0x56345cf6c8b0 2024-03-21T09:22:00.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 -- 172.21.15.37:0/967082835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 msgr2=0x56345cf6cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 0x56345cf6cc90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009f90 tx=0x7f2d60010860 comp rx=0 tx=0).stop 2024-03-21T09:22:00.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 -- 172.21.15.37:0/967082835 shutdown_connections 2024-03-21T09:22:00.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf75de0 0x56345cf78230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf75810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 --2- 172.21.15.37:0/967082835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6c8b0 0x56345cf6cc90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 -- 172.21.15.37:0/967082835 >> 172.21.15.37:0/967082835 conn(0x56345cf66f80 msgr2=0x56345cf67360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 -- 172.21.15.37:0/967082835 shutdown_connections 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.861+0000 7f2d7067f740 1 -- 172.21.15.37:0/967082835 wait complete. 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.862+0000 7f2d7067f740 1 -- start start 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.862+0000 7f2d7067f740 1 Processor -- start 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.862+0000 7f2d5f400640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2d4c0a3020 0x7f2d4c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.862+0000 7f2d5f400640 1 -- 172.21.15.37:0/3418604095 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2d580cfad0 con 0x7f2d68102630 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.862+0000 7f2d67400640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2d4c0a3020 0x7f2d4c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf75730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345cf78d50 con 0x56345cf6c8b0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345ce8c0d0 con 0x56345cf75de0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345cf78ed0 con 0x56345cf6d260 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d7067f740 10 client.0 _ll_register_callbacks cb 0x56345ce90e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58318/0 (socket says 172.21.15.37:58318) 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d67400640 1 -- 172.21.15.37:0/658401251 learned_addr learned my addr 172.21.15.37:0/658401251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66a00640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf75730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66000640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66a00640 1 -- 172.21.15.37:0/658401251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 msgr2=0x56345cf72ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66a00640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf72ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66a00640 1 -- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 msgr2=0x56345cf730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66a00640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.863+0000 7f2d66a00640 1 -- 172.21.15.37:0/658401251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56345cbe43f0 con 0x56345cf6c8b0 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d67400640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d67400640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2d4c0a3020 0x7f2d4c0a54e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d6001a000 tx=0x7f2d60019a60 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d66a00640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf75730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d580cce20 tx=0x7f2d5809b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63f4e9413853ae9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d50c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d50c00640 1 -- 172.21.15.37:0/658401251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5801c040 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d50c00640 1 -- 172.21.15.37:0/658401251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2d5809bd20 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d60011d80 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d50c00640 1 -- 172.21.15.37:0/658401251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5809bac0 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d7067f740 10 client.17616 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.864+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56345ccae7d0 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d50c00640 1 -- 172.21.15.37:0/658401251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d60011d80 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d50c00640 1 -- 172.21.15.37:0/658401251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2d5809af10 con 0x56345cf6c8b0 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d7067f740 10 client.17616 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d7067f740 10 client.17616 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d7067f740 20 client.17616 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d7067f740 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cf85c30 0x56345cf88020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d7067f740 4 client.17616 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:00.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56345cedd800 con 0x56345cf85c30 2024-03-21T09:22:00.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.865+0000 7f2d66000640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cf85c30 0x56345cf88020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.866+0000 7f2d66000640 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cf85c30 0x56345cf88020 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb87e86aa13b7204 server_cookie=620c6731b065d5a0 in_seq=0 out_seq=0 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.866+0000 7f2d50c00640 10 client.17616 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d50c00640 1 -- 172.21.15.37:0/658401251 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56345cedd800 con 0x56345cf85c30 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d50c00640 10 client.17616 handle_command_reply: tid=0 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 client.17616 shutdown 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 msgr2=0x56345cf75730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf75730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d580cce20 tx=0x7f2d5809b510 comp rx=0 tx=0).stop 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 shutdown_connections 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cf85c30 0x56345cf88020 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6d260 0x56345cf72ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 --2- 172.21.15.37:0/658401251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf75730 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d50c00640 0 client.17616 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:00.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.867+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 >> 172.21.15.37:0/658401251 conn(0x56345cf66f80 msgr2=0x56345cf68cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.868+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 shutdown_connections 2024-03-21T09:22:00.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.868+0000 7f2d7067f740 1 -- 172.21.15.37:0/658401251 wait complete. 2024-03-21T09:22:00.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.868+0000 7f2d7067f740 20 client.17616 trim_cache size 0 max 16384 2024-03-21T09:22:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 Processor -- start 2024-03-21T09:22:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 -- start start 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6c8b0 0x56345cf723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6d260 0x56345cf751f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf75730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681b2fd0 con 0x56345cf6d260 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681b0ad0 con 0x56345cf75de0 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.885+0000 7f2d7067f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681aeb90 con 0x56345cf6c8b0 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf75730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6c8b0 0x56345cf723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d67400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf75730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58322/0 (socket says 172.21.15.37:58322) 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d67400640 1 -- 172.21.15.37:0/1754480045 learned_addr learned my addr 172.21.15.37:0/1754480045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6d260 0x56345cf751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 -- 172.21.15.37:0/1754480045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 msgr2=0x56345cf75730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf75730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 -- 172.21.15.37:0/1754480045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6d260 msgr2=0x56345cf751f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6d260 0x56345cf751f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 -- 172.21.15.37:0/1754480045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56345cf88650 con 0x56345cf6c8b0 2024-03-21T09:22:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d67400640 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf75730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.886+0000 7f2d66a00640 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6c8b0 0x56345cf723a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2d58097f20 tx=0x7f2d58005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2643f71ad718a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d53400640 1 -- 172.21.15.37:0/1754480045 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5801c070 con 0x56345cf6c8b0 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d53400640 1 -- 172.21.15.37:0/1754480045 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d580a44d0 con 0x56345cf6c8b0 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d53400640 1 -- 172.21.15.37:0/1754480045 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5809ab20 con 0x56345cf6c8b0 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 -- 172.21.15.37:0/1754480045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6c8b0 msgr2=0x56345cf723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6c8b0 0x56345cf723a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2d58097f20 tx=0x7f2d58005440 comp rx=0 tx=0).stop 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 -- 172.21.15.37:0/1754480045 shutdown_connections 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf75de0 0x56345cf75730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6d260 0x56345cf751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 --2- 172.21.15.37:0/1754480045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf6c8b0 0x56345cf723a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 -- 172.21.15.37:0/1754480045 >> 172.21.15.37:0/1754480045 conn(0x56345cf66f80 msgr2=0x56345cf66160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.887+0000 7f2d7067f740 1 -- 172.21.15.37:0/1754480045 shutdown_connections 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.888+0000 7f2d7067f740 1 -- 172.21.15.37:0/1754480045 wait complete. 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.888+0000 7f2d7067f740 1 -- start start 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.888+0000 7f2d7067f740 1 Processor -- start 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf7d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6d260 0x56345cf7ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf75de0 0x56345cf7bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681af190 con 0x56345cf6c8b0 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681b0470 con 0x56345cf6d260 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681af7b0 con 0x56345cf75de0 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d7067f740 10 client.0 _ll_register_callbacks cb 0x7f2d580046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf7d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf7d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56480/0 (socket says 172.21.15.37:56480) 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 -- 172.21.15.37:0/2881398993 learned_addr learned my addr 172.21.15.37:0/2881398993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:00.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66000640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6d260 0x56345cf7ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d67400640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf75de0 0x56345cf7bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 -- 172.21.15.37:0/2881398993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf75de0 msgr2=0x56345cf7bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf75de0 0x56345cf7bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 -- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6d260 msgr2=0x56345cf7ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6d260 0x56345cf7ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66a00640 1 -- 172.21.15.37:0/2881398993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56345cf7c3d0 con 0x56345cf6c8b0 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.889+0000 7f2d66000640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6d260 0x56345cf7ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d66a00640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf7d870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2d58004770 tx=0x7f2d580096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2ff95a9279bf00 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d50c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d50c00640 1 -- 172.21.15.37:0/2881398993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5801c030 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d50c00640 1 -- 172.21.15.37:0/2881398993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2d5809ab20 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d580a44d0 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d50c00640 1 -- 172.21.15.37:0/2881398993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5809b420 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d7067f740 10 client.17622 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56345cfe1f20 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d50c00640 1 -- 172.21.15.37:0/2881398993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d580a44d0 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.890+0000 7f2d50c00640 1 -- 172.21.15.37:0/2881398993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2d580a0da0 con 0x56345cf6c8b0 2024-03-21T09:22:00.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d7067f740 10 client.17622 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d7067f740 10 client.17622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d7067f740 20 client.17622 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d7067f740 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cfe22b0 0x56345cfe46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d7067f740 4 client.17622 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56345ced7b50 con 0x56345cfe22b0 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d66000640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cfe22b0 0x56345cfe46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d66000640 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cfe22b0 0x56345cfe46a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea81da043c1c0ef server_cookie=c21c6a97a6c5ade4 in_seq=0 out_seq=0 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d50c00640 10 client.17622 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.891+0000 7f2d50c00640 1 -- 172.21.15.37:0/2881398993 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56345ced7b50 con 0x56345cfe22b0 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d50c00640 10 client.17622 handle_command_reply: tid=0 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 client.17622 shutdown 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 msgr2=0x56345cf7d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf7d870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2d58004770 tx=0x7f2d580096e0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 shutdown_connections 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x56345cfe22b0 0x56345cfe46a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56345cf75de0 0x56345cf7bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56345cf6d260 0x56345cf7ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 --2- 172.21.15.37:0/2881398993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345cf6c8b0 0x56345cf7d870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d50c00640 0 client.17622 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 >> 172.21.15.37:0/2881398993 conn(0x56345cf66f80 msgr2=0x56345cf6d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 shutdown_connections 2024-03-21T09:22:00.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 1 -- 172.21.15.37:0/2881398993 wait complete. 2024-03-21T09:22:00.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.892+0000 7f2d7067f740 20 client.17622 trim_cache size 0 max 16384 2024-03-21T09:22:00.899 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2d4c0a3020 msgr2=0x7f2d4c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2d4c0a3020 0x7f2d4c0a54e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d6001a000 tx=0x7f2d60019a60 comp rx=0 tx=0).stop 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 msgr2=0x7f2d68100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 0x7f2d68100280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9f0 tx=0x7f2d5800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 shutdown_connections 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2d4c0a3020 0x7f2d4c0a54e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d6810baa0 0x7f2d68100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102f50 0x7f2d681007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 --2- 172.21.15.37:0/3418604095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d68102630 0x7f2d68100280 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 >> 172.21.15.37:0/3418604095 conn(0x7f2d680fbe90 msgr2=0x7f2d680fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 shutdown_connections 2024-03-21T09:22:00.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:00.893+0000 7f2d67e00640 1 -- 172.21.15.37:0/3418604095 wait complete. 2024-03-21T09:22:00.909 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:22:00.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:22:01.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 -- 172.21.15.37:0/2530086972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd54102650 msgr2=0x7fcd54102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 --2- 172.21.15.37:0/2530086972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd54102650 0x7fcd54102a30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009900 tx=0x7fcd4402bcd0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 -- 172.21.15.37:0/2530086972 shutdown_connections 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 --2- 172.21.15.37:0/2530086972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd5410ba70 0x7fcd5410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 --2- 172.21.15.37:0/2530086972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102f70 0x7fcd5410b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 --2- 172.21.15.37:0/2530086972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd54102650 0x7fcd54102a30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 -- 172.21.15.37:0/2530086972 >> 172.21.15.37:0/2530086972 conn(0x7fcd540fbe30 msgr2=0x7fcd540fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.160+0000 7fcd53e00640 1 -- 172.21.15.37:0/2530086972 shutdown_connections 2024-03-21T09:22:01.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.161+0000 7fcd53e00640 1 -- 172.21.15.37:0/2530086972 wait complete. 2024-03-21T09:22:01.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.162+0000 7fcd53e00640 1 Processor -- start 2024-03-21T09:22:01.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.162+0000 7fcd53e00640 1 -- start start 2024-03-21T09:22:01.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 0x7fcd54199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd54102f70 0x7fcd5419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd5410ba70 0x7fcd5419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54111850 con 0x7fcd54102650 2024-03-21T09:22:01.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541116d0 con 0x7fcd54102f70 2024-03-21T09:22:01.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541119d0 con 0x7fcd5410ba70 2024-03-21T09:22:01.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 0x7fcd54199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 0x7fcd54199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56512/0 (socket says 172.21.15.37:56512) 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd52000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd54102f70 0x7fcd5419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd52a00640 1 -- 172.21.15.37:0/843316939 learned_addr learned my addr 172.21.15.37:0/843316939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd53400640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd5410ba70 0x7fcd5419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd52000640 1 -- 172.21.15.37:0/843316939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd5410ba70 msgr2=0x7fcd5419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.163+0000 7fcd52000640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd5410ba70 0x7fcd5419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd52000640 1 -- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 msgr2=0x7fcd54199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd52000640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 0x7fcd54199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd52000640 1 -- 172.21.15.37:0/843316939 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5419ef80 con 0x7fcd54102f70 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd53400640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd5410ba70 0x7fcd5419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd52a00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 0x7fcd54199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd52000640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd54102f70 0x7fcd5419a4d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcd40005fd0 tx=0x7fcd4000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd4b400640 1 -- 172.21.15.37:0/843316939 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004230 con 0x7fcd54102f70 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd4b400640 1 -- 172.21.15.37:0/843316939 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd400043d0 con 0x7fcd54102f70 2024-03-21T09:22:01.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5419f270 con 0x7fcd54102f70 2024-03-21T09:22:01.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd4b400640 1 -- 172.21.15.37:0/843316939 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40010460 con 0x7fcd54102f70 2024-03-21T09:22:01.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.164+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd541a6b00 con 0x7fcd54102f70 2024-03-21T09:22:01.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.167+0000 7fcd5c3c9740 1 Processor -- start 2024-03-21T09:22:01.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 -- start start 2024-03-21T09:22:01.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd4b400640 1 -- 172.21.15.37:0/843316939 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcd40009820 con 0x7fcd54102f70 2024-03-21T09:22:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c31bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c324810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c327230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3c27f2c0 con 0x55ea3c31c260 2024-03-21T09:22:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3c227c90 con 0x55ea3c324de0 2024-03-21T09:22:01.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3c27b360 con 0x55ea3c31b8b0 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c31bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.168+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c324810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c31bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40242/0 (socket says 172.21.15.37:40242) 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 -- 172.21.15.37:0/2946994757 learned_addr learned my addr 172.21.15.37:0/2946994757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c324810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56518/0 (socket says 172.21.15.37:56518) 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd52000640 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c327230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 -- 172.21.15.37:0/2946994757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 msgr2=0x55ea3c327230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c327230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 -- 172.21.15.37:0/2946994757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 msgr2=0x55ea3c324810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c324810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 -- 172.21.15.37:0/2946994757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3c24c080 con 0x55ea3c31b8b0 2024-03-21T09:22:01.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd53400640 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c31bc90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c011790 tx=0x7fcd4c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469a09023b23ef98 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:01.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.169+0000 7fcd3f400640 1 -- 172.21.15.37:0/2946994757 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c003c30 con 0x55ea3c31b8b0 2024-03-21T09:22:01.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd3f400640 1 -- 172.21.15.37:0/2946994757 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd4c00fe80 con 0x55ea3c31b8b0 2024-03-21T09:22:01.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd3f400640 1 -- 172.21.15.37:0/2946994757 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0193e0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd4b400640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcd380a3110 0x7fcd380a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd4b400640 1 -- 172.21.15.37:0/843316939 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcd400cf580 con 0x7fcd54102f70 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd52a00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcd380a3110 0x7fcd380a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2946994757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 msgr2=0x55ea3c31bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c31bc90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c011790 tx=0x7fcd4c011c50 comp rx=0 tx=0).stop 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2946994757 shutdown_connections 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c327230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c324810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2946994757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c31bc90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.170+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2946994757 >> 172.21.15.37:0/2946994757 conn(0x55ea3c315f80 msgr2=0x55ea3c316360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.171+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2946994757 shutdown_connections 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.171+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2946994757 wait complete. 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.171+0000 7fcd5c3c9740 1 -- start start 2024-03-21T09:22:01.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.171+0000 7fcd52a00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcd380a3110 0x7fcd380a55d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4402cf90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.171+0000 7fcd5c3c9740 1 Processor -- start 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31b8b0 0x55ea3c3246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 0x55ea3c321a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c321f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3c327d50 con 0x55ea3c31b8b0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3c23b0d0 con 0x55ea3c324de0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3c327ed0 con 0x55ea3c31c260 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd5c3c9740 10 client.0 _ll_register_callbacks cb 0x55ea3c23fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 0x55ea3c321a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31b8b0 0x55ea3c3246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd53400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c321f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 0x55ea3c321a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40254/0 (socket says 172.21.15.37:40254) 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 -- 172.21.15.37:0/3919621684 learned_addr learned my addr 172.21.15.37:0/3919621684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:01.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 -- 172.21.15.37:0/3919621684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 msgr2=0x55ea3c321f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c321f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 -- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31b8b0 msgr2=0x55ea3c3246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31b8b0 0x55ea3c3246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.172+0000 7fcd52000640 1 -- 172.21.15.37:0/3919621684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea3bf933f0 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd52000640 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 0x55ea3c321a30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcd40097f90 tx=0x7fcd400d2f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccdc518e756ee151 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd33e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd33e00640 1 -- 172.21.15.37:0/3919621684 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40020050 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd33e00640 1 -- 172.21.15.37:0/3919621684 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcd400cf120 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea3c28c800 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd33e00640 1 -- 172.21.15.37:0/3919621684 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4009af20 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd5c3c9740 10 client.27898 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ea3c05d7d0 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd33e00640 1 -- 172.21.15.37:0/3919621684 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ea3c28c800 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.173+0000 7fcd33e00640 1 -- 172.21.15.37:0/3919621684 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcd400a3a90 con 0x55ea3c31c260 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd5c3c9740 10 client.27898 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd5c3c9740 10 client.27898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd5c3c9740 20 client.27898 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c334c70 0x55ea3c337060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd5c3c9740 4 client.27898 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea3c286b50 con 0x55ea3c334c70 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.174+0000 7fcd52a00640 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c334c70 0x55ea3c337060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.175+0000 7fcd52a00640 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c334c70 0x55ea3c337060 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8c62678af0bf96 server_cookie=915ec03881c2da7d in_seq=0 out_seq=0 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.175+0000 7fcd33e00640 10 client.27898 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.175+0000 7fcd33e00640 1 -- 172.21.15.37:0/3919621684 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ea3c286b50 con 0x55ea3c334c70 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.175+0000 7fcd33e00640 10 client.27898 handle_command_reply: tid=0 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.175+0000 7fcd5c3c9740 1 client.27898 shutdown 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 msgr2=0x55ea3c321a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 0x55ea3c321a30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcd40097f90 tx=0x7fcd400d2f70 comp rx=0 tx=0).stop 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 shutdown_connections 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c334c70 0x55ea3c337060 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c324de0 0x55ea3c321f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31c260 0x55ea3c321a30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd33e00640 0 client.27898 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3919621684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31b8b0 0x55ea3c3246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 >> 172.21.15.37:0/3919621684 conn(0x55ea3c315f80 msgr2=0x55ea3c317cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 shutdown_connections 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3919621684 wait complete. 2024-03-21T09:22:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.176+0000 7fcd5c3c9740 20 client.27898 trim_cache size 0 max 16384 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 Processor -- start 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 -- start start 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 0x55ea3c324160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c3246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c324de0 0x55ea3c31ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541b30f0 con 0x55ea3c31c260 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541b0bf0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541aecb0 con 0x55ea3c324de0 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd52000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c3246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 0x55ea3c324160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 0x55ea3c324160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58402/0 (socket says 172.21.15.37:58402) 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c324de0 0x55ea3c31ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd52a00640 1 -- 172.21.15.37:0/2717044468 learned_addr learned my addr 172.21.15.37:0/2717044468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.190+0000 7fcd52000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c3246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56548/0 (socket says 172.21.15.37:56548) 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd52a00640 1 -- 172.21.15.37:0/2717044468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c324de0 msgr2=0x55ea3c31ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd52a00640 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c324de0 0x55ea3c31ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd52a00640 1 -- 172.21.15.37:0/2717044468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 msgr2=0x55ea3c3246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd52a00640 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c3246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd52a00640 1 -- 172.21.15.37:0/2717044468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3c31aad0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd52a00640 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 0x55ea3c324160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006010 tx=0x7fcd44041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1326a3c8cefb8866 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd3ea00640 1 -- 172.21.15.37:0/2717044468 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004240 con 0x55ea3c31b8b0 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd3ea00640 1 -- 172.21.15.37:0/2717044468 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd440043e0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.191+0000 7fcd3ea00640 1 -- 172.21.15.37:0/2717044468 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44054870 con 0x55ea3c31b8b0 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2717044468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 msgr2=0x55ea3c324160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 0x55ea3c324160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006010 tx=0x7fcd44041040 comp rx=0 tx=0).stop 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2717044468 shutdown_connections 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c324de0 0x55ea3c31ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c31c260 0x55ea3c3246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/2717044468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31b8b0 0x55ea3c324160 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2717044468 >> 172.21.15.37:0/2717044468 conn(0x55ea3c315f80 msgr2=0x55ea3c315160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:01.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2717044468 shutdown_connections 2024-03-21T09:22:01.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.192+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/2717044468 wait complete. 2024-03-21T09:22:01.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 -- start start 2024-03-21T09:22:01.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 Processor -- start 2024-03-21T09:22:01.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c328c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 0x55ea3c329140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c324de0 0x55ea3c38f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541af2b0 con 0x55ea3c324de0 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541b0590 con 0x55ea3c31c260 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541af8d0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.193+0000 7fcd5c3c9740 10 client.0 _ll_register_callbacks cb 0x7fcd400971a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 0x55ea3c329140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c328c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd53400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c324de0 0x55ea3c38f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 0x55ea3c329140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58414/0 (socket says 172.21.15.37:58414) 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52000640 1 -- 172.21.15.37:0/3423322540 learned_addr learned my addr 172.21.15.37:0/3423322540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 -- 172.21.15.37:0/3423322540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 msgr2=0x55ea3c329140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 0x55ea3c329140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 -- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c324de0 msgr2=0x55ea3c38f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:01.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c324de0 0x55ea3c38f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 -- 172.21.15.37:0/3423322540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea3c38f970 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52000640 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 0x55ea3c329140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd52a00640 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c328c00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcd44056040 tx=0x7fcd44004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29406cafe441bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.194+0000 7fcd33e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd33e00640 1 -- 172.21.15.37:0/3423322540 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44061060 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd33e00640 1 -- 172.21.15.37:0/3423322540 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcd44054050 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd440043e0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd33e00640 1 -- 172.21.15.37:0/3423322540 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd44004240 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd5c3c9740 10 client.27904 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ea3c390be0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd33e00640 1 -- 172.21.15.37:0/3423322540 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd440043e0 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.195+0000 7fcd33e00640 1 -- 172.21.15.37:0/3423322540 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcd44046040 con 0x55ea3c31b8b0 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd5c3c9740 10 client.27904 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd5c3c9740 10 client.27904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd5c3c9740 20 client.27904 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:01.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c391250 0x55ea3c393640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd5c3c9740 4 client.27904 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea3c204500 con 0x55ea3c391250 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd52000640 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c391250 0x55ea3c393640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd52000640 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c391250 0x55ea3c393640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffd84e03a4a1c85 server_cookie=7b34eccab27a1a4e in_seq=0 out_seq=0 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.196+0000 7fcd33e00640 10 client.27904 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd33e00640 1 -- 172.21.15.37:0/3423322540 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea3c204500 con 0x55ea3c391250 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd33e00640 10 client.27904 handle_command_reply: tid=0 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 client.27904 shutdown 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 msgr2=0x55ea3c328c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c328c00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcd44056040 tx=0x7fcd44004590 comp rx=0 tx=0).stop 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 shutdown_connections 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea3c324de0 0x55ea3c38f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55ea3c391250 0x55ea3c393640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3c31c260 0x55ea3c329140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 --2- 172.21.15.37:0/3423322540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea3c31b8b0 0x55ea3c328c00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd33e00640 0 client.27904 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 >> 172.21.15.37:0/3423322540 conn(0x55ea3c315f80 msgr2=0x55ea3c31c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 shutdown_connections 2024-03-21T09:22:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 1 -- 172.21.15.37:0/3423322540 wait complete. 2024-03-21T09:22:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.197+0000 7fcd5c3c9740 20 client.27904 trim_cache size 0 max 16384 2024-03-21T09:22:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:22:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcd380a3110 msgr2=0x7fcd380a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcd380a3110 0x7fcd380a55d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd44000c00 tx=0x7fcd4402cf90 comp rx=0 tx=0).stop 2024-03-21T09:22:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd54102f70 msgr2=0x7fcd5419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd54102f70 0x7fcd5419a4d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcd40005fd0 tx=0x7fcd4000db60 comp rx=0 tx=0).stop 2024-03-21T09:22:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 shutdown_connections 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd5410ba70 0x7fcd5419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcd380a3110 0x7fcd380a55d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd54102f70 0x7fcd5419a4d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 --2- 172.21.15.37:0/843316939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd54102650 0x7fcd54199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.198+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 >> 172.21.15.37:0/843316939 conn(0x7fcd540fbe30 msgr2=0x7fcd540fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.199+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 shutdown_connections 2024-03-21T09:22:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:01.199+0000 7fcd53e00640 1 -- 172.21.15.37:0/843316939 wait complete. 2024-03-21T09:22:01.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:01 smithi037 ceph-mon[31830]: pgmap v304: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:01.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:01 smithi053 ceph-mon[36169]: pgmap v304: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:01.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:01 smithi071 ceph-mon[36597]: pgmap v304: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:02.216 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T09:22:02.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:22:02.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 -- 172.21.15.37:0/1835974861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c109670 msgr2=0x7fdc2c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 --2- 172.21.15.37:0/1835974861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c109670 0x7fdc2c109a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009930 tx=0x7fdc1c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 -- 172.21.15.37:0/1835974861 shutdown_connections 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 --2- 172.21.15.37:0/1835974861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c104020 0x7fdc2c110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 --2- 172.21.15.37:0/1835974861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 --2- 172.21.15.37:0/1835974861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c109670 0x7fdc2c109a50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 -- 172.21.15.37:0/1835974861 >> 172.21.15.37:0/1835974861 conn(0x7fdc2c0ff150 msgr2=0x7fdc2c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 -- 172.21.15.37:0/1835974861 shutdown_connections 2024-03-21T09:22:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.464+0000 7fdc2b200640 1 -- 172.21.15.37:0/1835974861 wait complete. 2024-03-21T09:22:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 Processor -- start 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 -- start start 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c104020 0x7fdc2c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c109670 0x7fdc2c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c113a90 con 0x7fdc2c103660 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c113910 con 0x7fdc2c109670 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.465+0000 7fdc2b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c113c10 con 0x7fdc2c104020 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc2a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c109670 0x7fdc2c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c104020 0x7fdc2c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56564/0 (socket says 172.21.15.37:56564) 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 -- 172.21.15.37:0/4131309133 learned_addr learned my addr 172.21.15.37:0/4131309133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 -- 172.21.15.37:0/4131309133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c104020 msgr2=0x7fdc2c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c104020 0x7fdc2c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 -- 172.21.15.37:0/4131309133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c109670 msgr2=0x7fdc2c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c109670 0x7fdc2c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 -- 172.21.15.37:0/4131309133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c19efe0 con 0x7fdc2c103660 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc2a800640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c109670 0x7fdc2c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.466+0000 7fdc29e00640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c199ff0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009900 tx=0x7fdc1c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.467+0000 7fdc22a00640 1 -- 172.21.15.37:0/4131309133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c043020 con 0x7fdc2c103660 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.467+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c19f270 con 0x7fdc2c103660 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.467+0000 7fdc22a00640 1 -- 172.21.15.37:0/4131309133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc1c0054e0 con 0x7fdc2c103660 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.467+0000 7fdc22a00640 1 -- 172.21.15.37:0/4131309133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c003bb0 con 0x7fdc2c103660 2024-03-21T09:22:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.467+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c1a6b60 con 0x7fdc2c103660 2024-03-21T09:22:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.470+0000 7fdc34032740 1 Processor -- start 2024-03-21T09:22:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.470+0000 7fdc34032740 1 -- start start 2024-03-21T09:22:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.470+0000 7fdc22a00640 1 -- 172.21.15.37:0/4131309133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fdc1c046020 con 0x7fdc2c103660 2024-03-21T09:22:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.470+0000 7fdc34032740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 0x55645c570670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc34032740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c579250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc34032740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 0x55645c57bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc34032740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c4d4eb0 con 0x55645c570290 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc34032740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c47da60 con 0x55645c579790 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc34032740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c4d0f50 con 0x55645c570bb0 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc2a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 0x55645c570670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 0x55645c57bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc2a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 0x55645c570670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56580/0 (socket says 172.21.15.37:56580) 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc2a800640 1 -- 172.21.15.37:0/3106392748 learned_addr learned my addr 172.21.15.37:0/3106392748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29e00640 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c579250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 0x55645c57bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58426/0 (socket says 172.21.15.37:58426) 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 -- 172.21.15.37:0/3106392748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 msgr2=0x55645c579250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c579250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 -- 172.21.15.37:0/3106392748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 msgr2=0x55645c570670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 0x55645c570670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc29400640 1 -- 172.21.15.37:0/3106392748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645c2b17e0 con 0x55645c579790 2024-03-21T09:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.471+0000 7fdc2a800640 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 0x55645c570670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.472+0000 7fdc29400640 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 0x55645c57bb80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009bf0 tx=0x7fdc18011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16a3fdde76b080 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.472+0000 7fdc17400640 1 -- 172.21.15.37:0/3106392748 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18004050 con 0x55645c579790 2024-03-21T09:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.472+0000 7fdc17400640 1 -- 172.21.15.37:0/3106392748 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc18005500 con 0x55645c579790 2024-03-21T09:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.472+0000 7fdc17400640 1 -- 172.21.15.37:0/3106392748 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18004e00 con 0x55645c579790 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.472+0000 7fdc22a00640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdc100a3110 0x7fdc100a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.472+0000 7fdc22a00640 1 -- 172.21.15.37:0/4131309133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdc1c0f1a60 con 0x7fdc2c103660 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc2a800640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdc100a3110 0x7fdc100a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 -- 172.21.15.37:0/3106392748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 msgr2=0x55645c57bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 0x55645c57bb80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009bf0 tx=0x7fdc18011810 comp rx=0 tx=0).stop 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 -- 172.21.15.37:0/3106392748 shutdown_connections 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c579790 0x55645c57bb80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c579250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 --2- 172.21.15.37:0/3106392748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570290 0x55645c570670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 -- 172.21.15.37:0/3106392748 >> 172.21.15.37:0/3106392748 conn(0x55645c56a9c0 msgr2=0x55645c56ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 -- 172.21.15.37:0/3106392748 shutdown_connections 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc34032740 1 -- 172.21.15.37:0/3106392748 wait complete. 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.473+0000 7fdc2a800640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdc100a3110 0x7fdc100a55d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc240115a0 tx=0x7fdc24009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.474+0000 7fdc34032740 1 -- start start 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.474+0000 7fdc34032740 1 Processor -- start 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 0x55645c579040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 0x55645c5763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c576910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c57c580 con 0x55645c579790 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c490eb0 con 0x55645c570bb0 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c57c700 con 0x55645c570290 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc34032740 10 client.0 _ll_register_callbacks cb 0x55645c485150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:02.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 0x55645c5763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 0x55645c579040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 0x55645c579040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40312/0 (socket says 172.21.15.37:40312) 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29e00640 1 -- 172.21.15.37:0/1419060285 learned_addr learned my addr 172.21.15.37:0/1419060285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 0x55645c5763d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58436/0 (socket says 172.21.15.37:58436) 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc2a800640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c576910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 -- 172.21.15.37:0/1419060285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 msgr2=0x55645c579040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 0x55645c579040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 -- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 msgr2=0x55645c576910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c576910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29400640 1 -- 172.21.15.37:0/1419060285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55645c311f70 con 0x55645c570bb0 2024-03-21T09:22:02.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.475+0000 7fdc29e00640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 0x55645c579040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc29400640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 0x55645c5763d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009350 tx=0x7fdc18005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5cab883e94754 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc0be00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc0be00640 1 -- 172.21.15.37:0/1419060285 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18004d80 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc18005500 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc0be00640 1 -- 172.21.15.37:0/1419060285 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdc18004f20 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc0be00640 1 -- 172.21.15.37:0/1419060285 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc18033050 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc34032740 10 client.27896 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55645c1a9170 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.476+0000 7fdc0be00640 1 -- 172.21.15.37:0/1419060285 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc18005500 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc0be00640 1 -- 172.21.15.37:0/1419060285 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdc1802e440 con 0x55645c570bb0 2024-03-21T09:22:02.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc34032740 10 client.27896 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc34032740 10 client.27896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc34032740 20 client.27896 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc34032740 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c589610 0x55645c58ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc34032740 4 client.27896 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55645c4e23f0 con 0x55645c589610 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.477+0000 7fdc29e00640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c589610 0x55645c58ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.478+0000 7fdc29e00640 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c589610 0x55645c58ba00 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e96102b01a804a3 server_cookie=72f5c9d381554fcd in_seq=0 out_seq=0 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.478+0000 7fdc0be00640 10 client.27896 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc0be00640 1 -- 172.21.15.37:0/1419060285 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55645c4e23f0 con 0x55645c589610 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc0be00640 10 client.27896 handle_command_reply: tid=0 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 client.27896 shutdown 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 msgr2=0x55645c5763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 0x55645c5763d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009350 tx=0x7fdc18005730 comp rx=0 tx=0).stop 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 shutdown_connections 2024-03-21T09:22:02.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c576910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c589610 0x55645c58ba00 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570bb0 0x55645c5763d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 --2- 172.21.15.37:0/1419060285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570290 0x55645c579040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc0be00640 0 client.27896 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 >> 172.21.15.37:0/1419060285 conn(0x55645c56a9c0 msgr2=0x55645c56c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 shutdown_connections 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 1 -- 172.21.15.37:0/1419060285 wait complete. 2024-03-21T09:22:02.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.479+0000 7fdc34032740 20 client.27896 trim_cache size 0 max 16384 2024-03-21T09:22:02.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.497+0000 7fdc34032740 1 Processor -- start 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 -- start start 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c578b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570bb0 0x55645c579040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 0x55645c5716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1b30a0 con 0x55645c570bb0 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1b0ba0 con 0x55645c570290 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc34032740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1aec60 con 0x55645c579790 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc2a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 0x55645c5716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc2a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 0x55645c5716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40322/0 (socket says 172.21.15.37:40322) 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc2a800640 1 -- 172.21.15.37:0/2851791164 learned_addr learned my addr 172.21.15.37:0/2851791164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc29e00640 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c578b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.498+0000 7fdc29400640 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570bb0 0x55645c579040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc2a800640 1 -- 172.21.15.37:0/2851791164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 msgr2=0x55645c578b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc2a800640 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c578b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc2a800640 1 -- 172.21.15.37:0/2851791164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570bb0 msgr2=0x55645c579040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc2a800640 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570bb0 0x55645c579040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc2a800640 1 -- 172.21.15.37:0/2851791164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645c57c460 con 0x55645c579790 2024-03-21T09:22:02.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc2a800640 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 0x55645c5716a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400ffc0 tx=0x7fdc2401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15772026a9e3564c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc16a00640 1 -- 172.21.15.37:0/2851791164 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24012640 con 0x55645c579790 2024-03-21T09:22:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc16a00640 1 -- 172.21.15.37:0/2851791164 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc24011b50 con 0x55645c579790 2024-03-21T09:22:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.499+0000 7fdc16a00640 1 -- 172.21.15.37:0/2851791164 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24005970 con 0x55645c579790 2024-03-21T09:22:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 -- 172.21.15.37:0/2851791164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 msgr2=0x55645c5716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 0x55645c5716a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400ffc0 tx=0x7fdc2401b000 comp rx=0 tx=0).stop 2024-03-21T09:22:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 -- 172.21.15.37:0/2851791164 shutdown_connections 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c579790 0x55645c5716a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c570bb0 0x55645c579040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 --2- 172.21.15.37:0/2851791164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c578b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 -- 172.21.15.37:0/2851791164 >> 172.21.15.37:0/2851791164 conn(0x55645c56a9c0 msgr2=0x55645c570f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 -- 172.21.15.37:0/2851791164 shutdown_connections 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 -- 172.21.15.37:0/2851791164 wait complete. 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.500+0000 7fdc34032740 1 -- start start 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 Processor -- start 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c5814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c5735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c573b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1af260 con 0x55645c579790 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1b0540 con 0x55645c570290 2024-03-21T09:22:02.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c1af880 con 0x55645c570bb0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc34032740 10 client.0 _ll_register_callbacks cb 0x7fdc24009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc29400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c5735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc29e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c5814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc29400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c5735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40336/0 (socket says 172.21.15.37:40336) 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.501+0000 7fdc29400640 1 -- 172.21.15.37:0/2234140802 learned_addr learned my addr 172.21.15.37:0/2234140802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc2a800640 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c573b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc29400640 1 -- 172.21.15.37:0/2234140802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 msgr2=0x55645c5814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc29400640 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c5814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc29400640 1 -- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 msgr2=0x55645c573b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc29400640 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c573b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc29400640 1 -- 172.21.15.37:0/2234140802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55645c574150 con 0x55645c570bb0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc29400640 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c5735d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc18011850 tx=0x7fdc18005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73be7ed2eff860f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc0be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc0be00640 1 -- 172.21.15.37:0/2234140802 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1802ec20 con 0x55645c570bb0 2024-03-21T09:22:02.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc24011b50 con 0x55645c570bb0 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.502+0000 7fdc0be00640 1 -- 172.21.15.37:0/2234140802 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdc180035f0 con 0x55645c570bb0 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc0be00640 1 -- 172.21.15.37:0/2234140802 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc18011d10 con 0x55645c570bb0 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 10 client.27922 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55645c5ee3c0 con 0x55645c570bb0 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc0be00640 1 -- 172.21.15.37:0/2234140802 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc24011b50 con 0x55645c570bb0 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc0be00640 1 -- 172.21.15.37:0/2234140802 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdc18004cb0 con 0x55645c570bb0 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 10 client.27922 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 10 client.27922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 20 client.27922 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c5eea30 0x55645c5f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 4 client.27922 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55645c4dc740 con 0x55645c5eea30 2024-03-21T09:22:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.503+0000 7fdc29e00640 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c5eea30 0x55645c5f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.504+0000 7fdc29e00640 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c5eea30 0x55645c5f0e50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c579f8be7f910c8 server_cookie=74cb2c673a758c77 in_seq=0 out_seq=0 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.504+0000 7fdc0be00640 10 client.27922 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.505+0000 7fdc0be00640 1 -- 172.21.15.37:0/2234140802 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55645c4dc740 con 0x55645c5eea30 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.505+0000 7fdc0be00640 10 client.27922 handle_command_reply: tid=0 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.505+0000 7fdc34032740 1 client.27922 shutdown 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.505+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 msgr2=0x55645c5735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.505+0000 7fdc34032740 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c5735d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc18011850 tx=0x7fdc18005f80 comp rx=0 tx=0).stop 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 shutdown_connections 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55645c579790 0x55645c573b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c570bb0 0x55645c5735d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x55645c5eea30 0x55645c5f0e50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 --2- 172.21.15.37:0/2234140802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55645c570290 0x55645c5814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 >> 172.21.15.37:0/2234140802 conn(0x55645c56a9c0 msgr2=0x55645c571820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc0be00640 0 client.27922 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 shutdown_connections 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 1 -- 172.21.15.37:0/2234140802 wait complete. 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.506+0000 7fdc34032740 20 client.27922 trim_cache size 0 max 16384 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "d3ede0b5-5f45-4054-a313-8e79adca17ce", 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-03-21T09:22:02.506 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:02.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdc100a3110 msgr2=0x7fdc100a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdc100a3110 0x7fdc100a55d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc240115a0 tx=0x7fdc24009500 comp rx=0 tx=0).stop 2024-03-21T09:22:02.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 msgr2=0x7fdc2c199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c199ff0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009900 tx=0x7fdc1c004360 comp rx=0 tx=0).stop 2024-03-21T09:22:02.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 shutdown_connections 2024-03-21T09:22:02.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdc100a3110 0x7fdc100a55d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc2c109670 0x7fdc2c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc2c104020 0x7fdc2c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 --2- 172.21.15.37:0/4131309133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc2c103660 0x7fdc2c199ff0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 >> 172.21.15.37:0/4131309133 conn(0x7fdc2c0ff150 msgr2=0x7fdc2c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.507+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 shutdown_connections 2024-03-21T09:22:02.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.508+0000 7fdc2b200640 1 -- 172.21.15.37:0/4131309133 wait complete. 2024-03-21T09:22:02.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:22:02.525 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.743+0000 7ffa0de00640 1 -- 172.21.15.37:0/1569641259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 msgr2=0x7ffa08075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.743+0000 7ffa0de00640 1 --2- 172.21.15.37:0/1569641259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa08075fb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009930 tx=0x7ff9f802bc30 comp rx=0 tx=0).stop 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 -- 172.21.15.37:0/1569641259 shutdown_connections 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 --2- 172.21.15.37:0/1569641259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 0x7ffa081148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 --2- 172.21.15.37:0/1569641259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 0x7ffa08111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 --2- 172.21.15.37:0/1569641259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa08075fb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 -- 172.21.15.37:0/1569641259 >> 172.21.15.37:0/1569641259 conn(0x7ffa080ff310 msgr2=0x7ffa08101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 -- 172.21.15.37:0/1569641259 shutdown_connections 2024-03-21T09:22:02.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.744+0000 7ffa0de00640 1 -- 172.21.15.37:0/1569641259 wait complete. 2024-03-21T09:22:02.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 Processor -- start 2024-03-21T09:22:02.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 -- start start 2024-03-21T09:22:02.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa0819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 0x7ffa0819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 0x7ffa081a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081180a0 con 0x7ffa080764f0 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08117f20 con 0x7ffa08075bd0 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.745+0000 7ffa0de00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08118220 con 0x7ffa08112500 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa0819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 0x7ffa081a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa0819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58460/0 (socket says 172.21.15.37:58460) 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa06a00640 1 -- 172.21.15.37:0/2084125660 learned_addr learned my addr 172.21.15.37:0/2084125660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 0x7ffa0819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 -- 172.21.15.37:0/2084125660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 msgr2=0x7ffa0819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa0819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 -- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 msgr2=0x7ffa0819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 0x7ffa0819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 -- 172.21.15.37:0/2084125660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa081a35f0 con 0x7ffa08112500 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 0x7ffa0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.746+0000 7ffa07400640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 0x7ffa081a2ec0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005fd0 tx=0x7ffa0000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.747+0000 7ffa0ca00640 1 -- 172.21.15.37:0/2084125660 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000cdf0 con 0x7ffa08112500 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.747+0000 7ffa0ca00640 1 -- 172.21.15.37:0/2084125660 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa00005020 con 0x7ffa08112500 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.747+0000 7ffa0ca00640 1 -- 172.21.15.37:0/2084125660 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000106d0 con 0x7ffa08112500 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.747+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa081a38e0 con 0x7ffa08112500 2024-03-21T09:22:02.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.747+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa081ab170 con 0x7ffa08112500 2024-03-21T09:22:02.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.750+0000 7ffa0ca00640 1 -- 172.21.15.37:0/2084125660 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ffa000054d0 con 0x7ffa08112500 2024-03-21T09:22:02.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.750+0000 7ffa10b81740 1 Processor -- start 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.750+0000 7ffa10b81740 1 -- start start 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.750+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.750+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa10b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9a730250 con 0x557e9a7ccc00 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa10b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9a6d8a30 con 0x557e9a7cbf20 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa10b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9a72c2f0 con 0x557e9a7d5290 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58470/0 (socket says 172.21.15.37:58470) 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 -- 172.21.15.37:0/3253874058 learned_addr learned my addr 172.21.15.37:0/3253874058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 -- 172.21.15.37:0/3253874058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 msgr2=0x557e9a7d7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 -- 172.21.15.37:0/3253874058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 msgr2=0x557e9a7cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ffa06a00640 1 -- 172.21.15.37:0/3253874058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e9a6c3690 con 0x557e9a7cbf20 2024-03-21T09:22:02.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.751+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:02.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cc3a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005e90 tx=0x7ff9f8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19762aad53ff7a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ffa0ca00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff9ec0a3090 0x7ff9ec0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ffa0ca00640 1 -- 172.21.15.37:0/2084125660 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ffa000d03a0 con 0x7ffa08112500 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ff9f7e00640 1 -- 172.21.15.37:0/3253874058 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8045070 con 0x557e9a7cbf20 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ff9f7e00640 1 -- 172.21.15.37:0/3253874058 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9f802c650 con 0x557e9a7cbf20 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ff9f7e00640 1 -- 172.21.15.37:0/3253874058 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8004c20 con 0x557e9a7cbf20 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.752+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff9ec0a3090 0x7ff9ec0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 -- 172.21.15.37:0/3253874058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 msgr2=0x557e9a7cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cc3a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005e90 tx=0x7ff9f8009560 comp rx=0 tx=0).stop 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 -- 172.21.15.37:0/3253874058 shutdown_connections 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cc3a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3253874058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 -- 172.21.15.37:0/3253874058 >> 172.21.15.37:0/3253874058 conn(0x557e9a793200 msgr2=0x557e9a7c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff9ec0a3090 0x7ff9ec0a5550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009fd0 tx=0x7ff9f0015040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 -- 172.21.15.37:0/3253874058 shutdown_connections 2024-03-21T09:22:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 -- 172.21.15.37:0/3253874058 wait complete. 2024-03-21T09:22:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.753+0000 7ffa10b81740 1 -- start start 2024-03-21T09:22:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 Processor -- start 2024-03-21T09:22:02.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 0x557e9a7d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9a7d80a0 con 0x557e9a7cbf20 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9a6ebe80 con 0x557e9a7ccc00 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9a7d8220 con 0x557e9a7d5290 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.754+0000 7ffa10b81740 10 client.0 _ll_register_callbacks cb 0x557e9a6fe130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 0x557e9a7d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 0x557e9a7d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58478/0 (socket says 172.21.15.37:58478) 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ff9ffe00640 1 -- 172.21.15.37:0/3514819845 learned_addr learned my addr 172.21.15.37:0/3514819845 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 -- 172.21.15.37:0/3514819845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 msgr2=0x557e9a7d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 0x557e9a7d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 -- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7cbf20 msgr2=0x557e9a7d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 -- 172.21.15.37:0/3514819845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e9a50e7d0 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 0x557e9a7d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ffa07400640 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d2230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa000cd660 tx=0x7ffa0009c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf6fc242c6e0e0a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ff9f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.755+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3514819845 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000a4d40 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3514819845 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ffa000989a0 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9f802c650 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3514819845 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa000b5200 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ffa10b81740 10 client.27934 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557e9a4443f0 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3514819845 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9f802c650 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3514819845 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ffa0009b780 con 0x557e9a7d5290 2024-03-21T09:22:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ffa10b81740 10 client.27934 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.756+0000 7ffa10b81740 10 client.27934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ffa10b81740 20 client.27934 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a7e50e0 0x557e9a7e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ffa10b81740 4 client.27934 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e9a73d790 con 0x557e9a7e50e0 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a7e50e0 0x557e9a7e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a7e50e0 0x557e9a7e74d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5250ba529aaf6e61 server_cookie=1a15d759b783aa6d in_seq=0 out_seq=0 2024-03-21T09:22:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.757+0000 7ff9f4c00640 10 client.27934 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.758+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3514819845 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557e9a73d790 con 0x557e9a7e50e0 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.758+0000 7ff9f4c00640 10 client.27934 handle_command_reply: tid=0 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 client.27934 shutdown 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 msgr2=0x557e9a7d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d2230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa000cd660 tx=0x7ffa0009c330 comp rx=0 tx=0).stop 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 shutdown_connections 2024-03-21T09:22:02.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d2230 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a7e50e0 0x557e9a7e74d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7ccc00 0x557e9a7d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3514819845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 >> 172.21.15.37:0/3514819845 conn(0x557e9a793200 msgr2=0x557e9a7c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ff9f4c00640 0 client.27934 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 shutdown_connections 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 1 -- 172.21.15.37:0/3514819845 wait complete. 2024-03-21T09:22:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.759+0000 7ffa10b81740 20 client.27934 trim_cache size 0 max 16384 2024-03-21T09:22:02.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 Processor -- start 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 -- start start 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081b7600 con 0x557e9a7ccc00 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081b5100 con 0x557e9a7cbf20 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.777+0000 7ffa10b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081b31c0 con 0x557e9a7d5290 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40390/0 (socket says 172.21.15.37:40390) 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 -- 172.21.15.37:0/3350033531 learned_addr learned my addr 172.21.15.37:0/3350033531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 -- 172.21.15.37:0/3350033531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 msgr2=0x557e9a7d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 -- 172.21.15.37:0/3350033531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 msgr2=0x557e9a7d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 -- 172.21.15.37:0/3350033531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e9a4040f0 con 0x557e9a7d5290 2024-03-21T09:22:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ffa07400640 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d6680 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffa0009c2c0 tx=0x7ffa0009a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab667502f9a896c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ff9f7400640 1 -- 172.21.15.37:0/3350033531 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0009b550 con 0x557e9a7d5290 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ff9f7400640 1 -- 172.21.15.37:0/3350033531 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa00066070 con 0x557e9a7d5290 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.778+0000 7ff9f7400640 1 -- 172.21.15.37:0/3350033531 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000989a0 con 0x557e9a7d5290 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 -- 172.21.15.37:0/3350033531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 msgr2=0x557e9a7d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d6680 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffa0009c2c0 tx=0x7ffa0009a840 comp rx=0 tx=0).stop 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 -- 172.21.15.37:0/3350033531 shutdown_connections 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7d5290 0x557e9a7d6680 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7ccc00 0x557e9a7d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3350033531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 -- 172.21.15.37:0/3350033531 >> 172.21.15.37:0/3350033531 conn(0x557e9a793200 msgr2=0x557e9a7c64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 -- 172.21.15.37:0/3350033531 shutdown_connections 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.779+0000 7ffa10b81740 1 -- 172.21.15.37:0/3350033531 wait complete. 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 -- start start 2024-03-21T09:22:02.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 Processor -- start 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7ccc00 0x557e9a7cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7d5290 0x557e9a7ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081b37c0 con 0x557e9a7d5290 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081b4aa0 con 0x557e9a7cbf20 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081b3de0 con 0x557e9a7ccc00 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.780+0000 7ffa10b81740 10 client.0 _ll_register_callbacks cb 0x7ffa000985d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ff9ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7ccc00 0x557e9a7cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58486/0 (socket says 172.21.15.37:58486) 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 -- 172.21.15.37:0/3195790613 learned_addr learned my addr 172.21.15.37:0/3195790613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa07400640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7d5290 0x557e9a7ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 -- 172.21.15.37:0/3195790613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7ccc00 msgr2=0x557e9a7cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7ccc00 0x557e9a7cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 -- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7d5290 msgr2=0x557e9a7ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7d5290 0x557e9a7ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 -- 172.21.15.37:0/3195790613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e9a7cec30 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa07400640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7d5290 0x557e9a7ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ffa06a00640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cf960 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004730 tx=0x7ff9f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1676687495ae27de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ff9f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.781+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3195790613 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8059060 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa00066070 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3195790613 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff9f8004040 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3195790613 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f8060050 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 10 client.27920 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557e9a840910 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3195790613 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa00066070 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3195790613 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff9f804a450 con 0x557e9a7cbf20 2024-03-21T09:22:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 10 client.27920 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 10 client.27920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 20 client.27920 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a840cf0 0x557e9a8430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 4 client.27920 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.782+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e9a737ae0 con 0x557e9a840cf0 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.783+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a840cf0 0x557e9a8430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.783+0000 7ff9ffe00640 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a840cf0 0x557e9a8430e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443530c96f58834f server_cookie=fe9ba9a640d314ee in_seq=0 out_seq=0 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.783+0000 7ff9f4c00640 10 client.27920 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ff9f4c00640 1 -- 172.21.15.37:0/3195790613 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557e9a737ae0 con 0x557e9a840cf0 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ff9f4c00640 10 client.27920 handle_command_reply: tid=0 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 client.27920 shutdown 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 msgr2=0x557e9a7cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cf960 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004730 tx=0x7ff9f80046c0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 shutdown_connections 2024-03-21T09:22:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e9a7d5290 0x557e9a7ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e9a7ccc00 0x557e9a7cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x557e9a840cf0 0x557e9a8430e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 --2- 172.21.15.37:0/3195790613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9a7cbf20 0x557e9a7cf960 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 >> 172.21.15.37:0/3195790613 conn(0x557e9a793200 msgr2=0x557e9a7c5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ff9f4c00640 0 client.27920 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 shutdown_connections 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 1 -- 172.21.15.37:0/3195790613 wait complete. 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.784+0000 7ffa10b81740 20 client.27920 trim_cache size 0 max 16384 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T09:22:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "d3ede0b5-5f45-4054-a313-8e79adca17ce": { 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "d3ede0b5-5f45-4054-a313-8e79adca17ce", 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:02.789 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff9ec0a3090 msgr2=0x7ff9ec0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff9ec0a3090 0x7ff9ec0a5550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009fd0 tx=0x7ff9f0015040 comp rx=0 tx=0).stop 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 msgr2=0x7ffa081a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 0x7ffa081a2ec0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005fd0 tx=0x7ffa0000ef10 comp rx=0 tx=0).stop 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 shutdown_connections 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa08112500 0x7ffa081a2ec0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa080764f0 0x7ffa0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff9ec0a3090 0x7ff9ec0a5550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 --2- 172.21.15.37:0/2084125660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa08075bd0 0x7ffa0819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 >> 172.21.15.37:0/2084125660 conn(0x7ffa080ff310 msgr2=0x7ffa08113bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.785+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 shutdown_connections 2024-03-21T09:22:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:02.786+0000 7ffa0de00640 1 -- 172.21.15.37:0/2084125660 wait complete. 2024-03-21T09:22:02.808 INFO:tasks.cephfs.filesystem:scrub status for tag:d3ede0b5-5f45-4054-a313-8e79adca17ce - {'path': '/', 'tag': 'd3ede0b5-5f45-4054-a313-8e79adca17ce', 'options': 'recursive,force'} 2024-03-21T09:22:02.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:02 smithi037 ceph-mon[31830]: scrub queued for path: / 2024-03-21T09:22:02.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:02 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:22:02.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:02 smithi037 ceph-mon[31830]: scrub summary: active paths [/] 2024-03-21T09:22:02.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:02 smithi037 ceph-mon[31830]: scrub summary: idle+waiting paths [/] 2024-03-21T09:22:02.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:02 smithi053 ceph-mon[36169]: scrub queued for path: / 2024-03-21T09:22:02.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:02 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:22:02.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:02 smithi053 ceph-mon[36169]: scrub summary: active paths [/] 2024-03-21T09:22:02.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:02 smithi053 ceph-mon[36169]: scrub summary: idle+waiting paths [/] 2024-03-21T09:22:03.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:02 smithi071 ceph-mon[36597]: scrub queued for path: / 2024-03-21T09:22:03.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:02 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:22:03.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:02 smithi071 ceph-mon[36597]: scrub summary: active paths [/] 2024-03-21T09:22:03.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:02 smithi071 ceph-mon[36597]: scrub summary: idle+waiting paths [/] 2024-03-21T09:22:03.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:03 smithi037 ceph-mon[31830]: pgmap v305: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:22:03.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:03 smithi053 ceph-mon[36169]: pgmap v305: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:22:04.105 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:03 smithi071 ceph-mon[36597]: pgmap v305: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:22:05.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:05 smithi037 ceph-mon[31830]: pgmap v306: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:22:05.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:05 smithi053 ceph-mon[36169]: pgmap v306: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:22:05.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:05 smithi071 ceph-mon[36597]: pgmap v306: 129 pgs: 129 active+clean; 17 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:22:07.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:07 smithi071 ceph-mon[36597]: scrub summary: idle 2024-03-21T09:22:07.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:07 smithi071 ceph-mon[36597]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T09:22:07.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:07 smithi071 ceph-mon[36597]: pgmap v308: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:07.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:07 smithi037 ceph-mon[31830]: scrub summary: idle 2024-03-21T09:22:07.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:07 smithi037 ceph-mon[31830]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T09:22:07.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:07 smithi037 ceph-mon[31830]: pgmap v308: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:07.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:07 smithi053 ceph-mon[36169]: scrub summary: idle 2024-03-21T09:22:07.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:07 smithi053 ceph-mon[36169]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T09:22:07.944 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:07 smithi053 ceph-mon[36169]: pgmap v308: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:09.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:09 smithi037 ceph-mon[31830]: pgmap v309: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:09.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:09 smithi053 ceph-mon[36169]: pgmap v309: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:09.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:09 smithi071 ceph-mon[36597]: pgmap v309: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:11.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:11 smithi037 ceph-mon[31830]: pgmap v310: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:11.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:11 smithi053 ceph-mon[36169]: pgmap v310: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:11.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:11 smithi071 ceph-mon[36597]: pgmap v310: 129 pgs: 129 active+clean; 17 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-03-21T09:22:13.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:13 smithi037 ceph-mon[31830]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-03-21T09:22:13.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:13 smithi053 ceph-mon[36169]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-03-21T09:22:13.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:13 smithi071 ceph-mon[36597]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-03-21T09:22:14.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-21T09:22:14.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.313+0000 7f38f1c00640 1 -- 172.21.15.37:0/1373901946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec069cb0 msgr2=0x7f38ec06a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.313+0000 7f38f1c00640 1 --2- 172.21.15.37:0/1373901946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec069cb0 0x7f38ec06a130 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009900 tx=0x7f38d802bcd0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.313+0000 7f38f1c00640 1 -- 172.21.15.37:0/1373901946 shutdown_connections 2024-03-21T09:22:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.313+0000 7f38f1c00640 1 --2- 172.21.15.37:0/1373901946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec06a670 0x7f38ec10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.313+0000 7f38f1c00640 1 --2- 172.21.15.37:0/1373901946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec069cb0 0x7f38ec06a130 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.313+0000 7f38f1c00640 1 --2- 172.21.15.37:0/1373901946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38ec107710 0x7f38ec107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.314+0000 7f38f1c00640 1 -- 172.21.15.37:0/1373901946 >> 172.21.15.37:0/1373901946 conn(0x7f38ec077360 msgr2=0x7f38ec077770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.314+0000 7f38f1c00640 1 -- 172.21.15.37:0/1373901946 shutdown_connections 2024-03-21T09:22:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.314+0000 7f38f1c00640 1 -- 172.21.15.37:0/1373901946 wait complete. 2024-03-21T09:22:14.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 Processor -- start 2024-03-21T09:22:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 -- start start 2024-03-21T09:22:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec069cb0 0x7f38ec1008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38ec06a670 0x7f38ec100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 0x7f38ec101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec111ab0 con 0x7f38ec06a670 2024-03-21T09:22:14.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec111930 con 0x7f38ec069cb0 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38f1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec111c30 con 0x7f38ec107710 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38eb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 0x7f38ec101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38eaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec069cb0 0x7f38ec1008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38ea000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38ec06a670 0x7f38ec100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38eb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 0x7f38ec101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33228/0 (socket says 172.21.15.37:33228) 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38eaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec069cb0 0x7f38ec1008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:43048/0 (socket says 172.21.15.37:43048) 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.315+0000 7f38eb400640 1 -- 172.21.15.37:0/2802245362 learned_addr learned my addr 172.21.15.37:0/2802245362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38eb400640 1 -- 172.21.15.37:0/2802245362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec069cb0 msgr2=0x7f38ec1008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38eb400640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec069cb0 0x7f38ec1008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38eb400640 1 -- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38ec06a670 msgr2=0x7f38ec100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38eb400640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38ec06a670 0x7f38ec100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38eb400640 1 -- 172.21.15.37:0/2802245362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec1a6410 con 0x7f38ec107710 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38eb400640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 0x7f38ec101340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f38e400eb10 tx=0x7f38e400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4004b20 con 0x7f38ec107710 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.316+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38e4004cc0 con 0x7f38ec107710 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.317+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec1a6700 con 0x7f38ec107710 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.317+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4010760 con 0x7f38ec107710 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.317+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec1a6c70 con 0x7f38ec107710 2024-03-21T09:22:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.318+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec102da0 con 0x7f38ec107710 2024-03-21T09:22:14.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.320+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f38e4005020 con 0x7f38ec107710 2024-03-21T09:22:14.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.321+0000 7f38e3400640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f38d00a3100 0x7f38d00a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.321+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f38e40d02d0 con 0x7f38ec107710 2024-03-21T09:22:14.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.321+0000 7f38eaa00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f38d00a3100 0x7f38d00a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.322+0000 7f38eaa00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f38d00a3100 0x7f38d00a55c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009b50 tx=0x7f38dc005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:14.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.325+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38e409c910 con 0x7f38ec107710 2024-03-21T09:22:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.556+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 --> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f38ec061ae0 con 0x7f38d00a3100 2024-03-21T09:22:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.568+0000 7f38e3400640 1 -- 172.21.15.37:0/2802245362 <== mgr.25553 v2:172.21.15.37:6800/1212038665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f38ec061ae0 con 0x7f38d00a3100 2024-03-21T09:22:14.565 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-21T00:00:00", "created": "2024-03-21T09:12:12", "first": "2024-03-21T09:13:00", "last": "2024-03-21T09:22:00", "last_pruned": "2024-03-21T09:22:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-21T09:22:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f38d00a3100 msgr2=0x7f38d00a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f38d00a3100 0x7f38d00a55c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009b50 tx=0x7f38dc005f90 comp rx=0 tx=0).stop 2024-03-21T09:22:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 msgr2=0x7f38ec101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 0x7f38ec101340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f38e400eb10 tx=0x7f38e400c2d0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 shutdown_connections 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38ec107710 0x7f38ec101340 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38ec06a670 0x7f38ec100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f38d00a3100 0x7f38d00a55c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 --2- 172.21.15.37:0/2802245362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38ec069cb0 0x7f38ec1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.572+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 >> 172.21.15.37:0/2802245362 conn(0x7f38ec077360 msgr2=0x7f38ec0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.573+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 shutdown_connections 2024-03-21T09:22:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.573+0000 7f38f1c00640 1 -- 172.21.15.37:0/2802245362 wait complete. 2024-03-21T09:22:14.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 -- 172.21.15.37:0/576951578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 msgr2=0x7f6f80101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 --2- 172.21.15.37:0/576951578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f80101ad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009900 tx=0x7f6f6c02bc40 comp rx=0 tx=0).stop 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 -- 172.21.15.37:0/576951578 shutdown_connections 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 --2- 172.21.15.37:0/576951578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80102010 0x7f6f8010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 --2- 172.21.15.37:0/576951578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f80101ad0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 --2- 172.21.15.37:0/576951578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80107660 0x7f6f80107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 -- 172.21.15.37:0/576951578 >> 172.21.15.37:0/576951578 conn(0x7f6f800fd120 msgr2=0x7f6f800ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 -- 172.21.15.37:0/576951578 shutdown_connections 2024-03-21T09:22:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.793+0000 7f6f87600640 1 -- 172.21.15.37:0/576951578 wait complete. 2024-03-21T09:22:14.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.794+0000 7f6f87600640 1 Processor -- start 2024-03-21T09:22:14.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.794+0000 7f6f87600640 1 -- start start 2024-03-21T09:22:14.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f87600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f800739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f87600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80102010 0x7f6f80073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f87600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80107660 0x7f6f80074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f87600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80111a70 con 0x7f6f80107660 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f87600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f801118f0 con 0x7f6f80101650 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f87600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80111bf0 con 0x7f6f80102010 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f800739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f800739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:43058/0 (socket says 172.21.15.37:43058) 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 -- 172.21.15.37:0/2545143814 learned_addr learned my addr 172.21.15.37:0/2545143814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f7fe00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80102010 0x7f6f80073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f85600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80107660 0x7f6f80074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 -- 172.21.15.37:0/2545143814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80102010 msgr2=0x7f6f80073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80102010 0x7f6f80073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 -- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80107660 msgr2=0x7f6f80074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:14.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80107660 0x7f6f80074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f84c00640 1 -- 172.21.15.37:0/2545143814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f801a6450 con 0x7f6f80101650 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f7fe00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80102010 0x7f6f80073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.795+0000 7f6f85600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80107660 0x7f6f80074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.796+0000 7f6f84c00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f800739a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400e9f0 tx=0x7f6f7400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.796+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7400cd80 con 0x7f6f80101650 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.796+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f74005500 con 0x7f6f80101650 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.796+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f74010690 con 0x7f6f80101650 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.796+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f801a6740 con 0x7f6f80101650 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.796+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f801a6c40 con 0x7f6f80101650 2024-03-21T09:22:14.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.797+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8004fc50 con 0x7f6f80101650 2024-03-21T09:22:14.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.799+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6f740046c0 con 0x7f6f80101650 2024-03-21T09:22:14.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.801+0000 7f6f7d600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6f680a3130 0x7f6f680a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:14.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.801+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f6f740d00d0 con 0x7f6f80101650 2024-03-21T09:22:14.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.801+0000 7f6f7fe00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6f680a3130 0x7f6f680a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:14.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.802+0000 7f6f7fe00640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6f680a3130 0x7f6f680a55f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6f800753e0 tx=0x7f6f6c005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:14.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:14.805+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6f7409c710 con 0x7f6f80101650 2024-03-21T09:22:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.133+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 --> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6f80061ae0 con 0x7f6f680a3130 2024-03-21T09:22:15.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.145+0000 7f6f7d600640 1 -- 172.21.15.37:0/2545143814 <== mgr.25553 v2:172.21.15.37:6800/1212038665 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6f80061ae0 con 0x7f6f680a3130 2024-03-21T09:22:15.142 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-03-21T09:22:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.148+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6f680a3130 msgr2=0x7f6f680a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6f680a3130 0x7f6f680a55f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6f800753e0 tx=0x7f6f6c005a20 comp rx=0 tx=0).stop 2024-03-21T09:22:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 msgr2=0x7f6f800739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f800739a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400e9f0 tx=0x7f6f7400edf0 comp rx=0 tx=0).stop 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 shutdown_connections 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f80107660 0x7f6f80074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f80102010 0x7f6f80073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f6f680a3130 0x7f6f680a55f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 --2- 172.21.15.37:0/2545143814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f80101650 0x7f6f800739a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 >> 172.21.15.37:0/2545143814 conn(0x7f6f800fd120 msgr2=0x7f6f800fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 shutdown_connections 2024-03-21T09:22:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:15.149+0000 7f6f87600640 1 -- 172.21.15.37:0/2545143814 wait complete. 2024-03-21T09:22:15.176 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-21T09:22:15.191 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-21T09:22:15.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:15 smithi037 ceph-mon[31830]: from='client.27946 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:22:15.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:15 smithi037 ceph-mon[31830]: from='client.27932 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:22:15.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:15 smithi037 ceph-mon[31830]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-03-21T09:22:15.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:15 smithi053 ceph-mon[36169]: from='client.27946 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:22:15.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:15 smithi053 ceph-mon[36169]: from='client.27932 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:22:15.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:15 smithi053 ceph-mon[36169]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-03-21T09:22:15.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:15 smithi071 ceph-mon[36597]: from='client.27946 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:22:15.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:15 smithi071 ceph-mon[36597]: from='client.27932 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:22:15.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:15 smithi071 ceph-mon[36597]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-03-21T09:22:17.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:17 smithi037 ceph-mon[31830]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T09:22:17.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:17 smithi053 ceph-mon[36169]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T09:22:17.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:17 smithi071 ceph-mon[36597]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T09:22:18.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:18 smithi037 ceph-mon[31830]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:18.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:18 smithi053 ceph-mon[36169]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:18.605 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:18 smithi071 ceph-mon[36597]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:19.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:19 smithi037 ceph-mon[31830]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:19.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:19 smithi053 ceph-mon[36169]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:19.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:19 smithi071 ceph-mon[36597]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:21.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:21 smithi037 ceph-mon[31830]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:21.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:21 smithi053 ceph-mon[36169]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:21.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:21 smithi071 ceph-mon[36597]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T09:22:23.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:23 smithi037 ceph-mon[31830]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-21T09:22:23.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:23 smithi053 ceph-mon[36169]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-21T09:22:23.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:23 smithi071 ceph-mon[36597]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-21T09:22:25.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:25 smithi037 ceph-mon[31830]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-21T09:22:25.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:25 smithi053 ceph-mon[36169]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-21T09:22:25.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:25 smithi071 ceph-mon[36597]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-21T09:22:27.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:27 smithi037 ceph-mon[31830]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T09:22:27.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:27 smithi053 ceph-mon[36169]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T09:22:27.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:27 smithi071 ceph-mon[36597]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T09:22:29.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:29 smithi037 ceph-mon[31830]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:29.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:29 smithi053 ceph-mon[36169]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:29.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:29 smithi071 ceph-mon[36597]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:31.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:31 smithi037 ceph-mon[31830]: pgmap v321: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:31.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:31 smithi053 ceph-mon[36169]: pgmap v321: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:31.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:31 smithi071 ceph-mon[36597]: pgmap v321: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:32.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T09:22:32.809 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T09:22:33.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 -- 172.21.15.37:0/414143037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c103830 msgr2=0x7f3b8c103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 --2- 172.21.15.37:0/414143037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c103830 0x7f3b8c103cb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009900 tx=0x7f3b7802bc30 comp rx=0 tx=0).stop 2024-03-21T09:22:33.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 -- 172.21.15.37:0/414143037 shutdown_connections 2024-03-21T09:22:33.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 --2- 172.21.15.37:0/414143037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 0x7f3b8c110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 --2- 172.21.15.37:0/414143037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c103830 0x7f3b8c103cb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 --2- 172.21.15.37:0/414143037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c109840 0x7f3b8c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 -- 172.21.15.37:0/414143037 >> 172.21.15.37:0/414143037 conn(0x7f3b8c0ff320 msgr2=0x7f3b8c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 -- 172.21.15.37:0/414143037 shutdown_connections 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.048+0000 7f3b90e00640 1 -- 172.21.15.37:0/414143037 wait complete. 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.049+0000 7f3b90e00640 1 Processor -- start 2024-03-21T09:22:33.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 -- start start 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c103830 0x7f3b8c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 0x7f3b8c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c109840 0x7f3b8c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c113cc0 con 0x7f3b8c1041f0 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c113b40 con 0x7f3b8c109840 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b90e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c113e40 con 0x7f3b8c103830 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b89400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 0x7f3b8c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c109840 0x7f3b8c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c103830 0x7f3b8c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c109840 0x7f3b8c19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41026/0 (socket says 172.21.15.37:41026) 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b89400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 0x7f3b8c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49376/0 (socket says 172.21.15.37:49376) 2024-03-21T09:22:33.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.050+0000 7f3b8a800640 1 -- 172.21.15.37:0/802262611 learned_addr learned my addr 172.21.15.37:0/802262611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b89e00640 1 -- 172.21.15.37:0/802262611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c109840 msgr2=0x7f3b8c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b89e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c109840 0x7f3b8c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b89e00640 1 -- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 msgr2=0x7f3b8c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b89e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 0x7f3b8c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b89e00640 1 -- 172.21.15.37:0/802262611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c19f230 con 0x7f3b8c103830 2024-03-21T09:22:33.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b89e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c103830 0x7f3b8c19a240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009fd0 tx=0x7f3b7c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.051+0000 7f3b82a00640 1 -- 172.21.15.37:0/802262611 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c004b20 con 0x7f3b8c103830 2024-03-21T09:22:33.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.052+0000 7f3b82a00640 1 -- 172.21.15.37:0/802262611 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c004cc0 con 0x7f3b8c103830 2024-03-21T09:22:33.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.052+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c19f520 con 0x7f3b8c103830 2024-03-21T09:22:33.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.052+0000 7f3b82a00640 1 -- 172.21.15.37:0/802262611 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0107b0 con 0x7f3b8c103830 2024-03-21T09:22:33.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.052+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1a6db0 con 0x7f3b8c103830 2024-03-21T09:22:33.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.055+0000 7f3b93b9e740 1 Processor -- start 2024-03-21T09:22:33.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.055+0000 7f3b82a00640 1 -- 172.21.15.37:0/802262611 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3b7c005020 con 0x7f3b8c103830 2024-03-21T09:22:33.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.055+0000 7f3b93b9e740 1 -- start start 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.055+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef7e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef7f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef8a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eeee3250 con 0x5588eef7fc00 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eee8ba30 con 0x5588eef7ef20 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eeedf2f0 con 0x5588eef88290 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef7e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef7f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef8a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef7e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49378/0 (socket says 172.21.15.37:49378) 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b8a800640 1 -- 172.21.15.37:0/1369567287 learned_addr learned my addr 172.21.15.37:0/1369567287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89400640 1 -- 172.21.15.37:0/1369567287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 msgr2=0x5588eef7f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89400640 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef7f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89400640 1 -- 172.21.15.37:0/1369567287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 msgr2=0x5588eef7e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89400640 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef7e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.056+0000 7f3b89400640 1 -- 172.21.15.37:0/1369567287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588eee76690 con 0x5588eef88290 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.057+0000 7f3b89400640 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef8a680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3b7802c170 tx=0x7f3b780095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2520acce77cbbb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.057+0000 7f3b77400640 1 -- 172.21.15.37:0/1369567287 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78042070 con 0x5588eef88290 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.057+0000 7f3b77400640 1 -- 172.21.15.37:0/1369567287 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b78004240 con 0x5588eef88290 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.057+0000 7f3b77400640 1 -- 172.21.15.37:0/1369567287 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78004f80 con 0x5588eef88290 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.057+0000 7f3b82a00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3b700a3110 0x7f3b700a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1369567287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 msgr2=0x5588eef8a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef8a680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3b7802c170 tx=0x7f3b780095e0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b82a00640 1 -- 172.21.15.37:0/802262611 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3b7c0d0330 con 0x7f3b8c103830 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b8a800640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3b700a3110 0x7f3b700a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1369567287 shutdown_connections 2024-03-21T09:22:33.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef8a680 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef7f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1369567287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef7e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1369567287 >> 172.21.15.37:0/1369567287 conn(0x5588eef46200 msgr2=0x5588eef79970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1369567287 shutdown_connections 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.058+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1369567287 wait complete. 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b8a800640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3b700a3110 0x7f3b700a55d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009300 tx=0x7f3b84009190 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b93b9e740 1 -- start start 2024-03-21T09:22:33.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b93b9e740 1 Processor -- start 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7ef20 0x5588eef87920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 0x5588eef84cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.059+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eef8b0a0 con 0x5588eef7ef20 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eee9ee80 con 0x5588eef88290 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eef8b220 con 0x5588eef7fc00 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b93b9e740 10 client.0 _ll_register_callbacks cb 0x5588eeeb1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 0x5588eef84cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7ef20 0x5588eef87920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef851f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41048/0 (socket says 172.21.15.37:41048) 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 0x5588eef84cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56206/0 (socket says 172.21.15.37:56206) 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b8a800640 1 -- 172.21.15.37:0/2970798678 learned_addr learned my addr 172.21.15.37:0/2970798678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 -- 172.21.15.37:0/2970798678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 msgr2=0x5588eef851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 -- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7ef20 msgr2=0x5588eef87920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7ef20 0x5588eef87920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 -- 172.21.15.37:0/2970798678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588eecc17d0 con 0x5588eef7fc00 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b8a800640 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b89400640 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 0x5588eef84cb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3b7802c240 tx=0x7f3b7802c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce10e65c3723e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.060+0000 7f3b6be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b6be00640 1 -- 172.21.15.37:0/2970798678 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78042040 con 0x5588eef7fc00 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b6be00640 1 -- 172.21.15.37:0/2970798678 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3b7802cab0 con 0x5588eef7fc00 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b78004240 con 0x5588eef7fc00 2024-03-21T09:22:33.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b6be00640 1 -- 172.21.15.37:0/2970798678 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b7802ccb0 con 0x5588eef7fc00 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b93b9e740 10 client.27964 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5588eebf73f0 con 0x5588eef7fc00 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b6be00640 1 -- 172.21.15.37:0/2970798678 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b78004240 con 0x5588eef7fc00 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.061+0000 7f3b6be00640 1 -- 172.21.15.37:0/2970798678 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3b78047bf0 con 0x5588eef7fc00 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b93b9e740 10 client.27964 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b93b9e740 10 client.27964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b93b9e740 20 client.27964 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eef97f80 0x5588eef9a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b93b9e740 4 client.27964 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588eeef0790 con 0x5588eef97f80 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.062+0000 7f3b89e00640 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eef97f80 0x5588eef9a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.063+0000 7f3b89e00640 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eef97f80 0x5588eef9a370 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28468601bb2b42d server_cookie=f2487c64a09f2d35 in_seq=0 out_seq=0 2024-03-21T09:22:33.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.063+0000 7f3b6be00640 10 client.27964 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:33.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.063+0000 7f3b6be00640 1 -- 172.21.15.37:0/2970798678 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5588eeef0790 con 0x5588eef97f80 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.063+0000 7f3b6be00640 10 client.27964 handle_command_reply: tid=0 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.063+0000 7f3b93b9e740 1 client.27964 shutdown 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 msgr2=0x5588eef84cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 0x5588eef84cb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3b7802c240 tx=0x7f3b7802c800 comp rx=0 tx=0).stop 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 shutdown_connections 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eef97f80 0x5588eef9a370 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7fc00 0x5588eef84cb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b6be00640 0 client.27964 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2970798678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7ef20 0x5588eef87920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 >> 172.21.15.37:0/2970798678 conn(0x5588eef46200 msgr2=0x5588eef7b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 shutdown_connections 2024-03-21T09:22:33.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2970798678 wait complete. 2024-03-21T09:22:33.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.064+0000 7f3b93b9e740 20 client.27964 trim_cache size 0 max 16384 2024-03-21T09:22:33.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 Processor -- start 2024-03-21T09:22:33.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 -- start start 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef87920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1b30e0 con 0x5588eef7fc00 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1b0be0 con 0x5588eef7ef20 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.086+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1aeca0 con 0x5588eef88290 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef87920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef944c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41052/0 (socket says 172.21.15.37:41052) 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 -- 172.21.15.37:0/1478100672 learned_addr learned my addr 172.21.15.37:0/1478100672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b8a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef87920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56220/0 (socket says 172.21.15.37:56220) 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 -- 172.21.15.37:0/1478100672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 msgr2=0x5588eef87920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef87920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 -- 172.21.15.37:0/1478100672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 msgr2=0x5588eef873e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef873e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 -- 172.21.15.37:0/1478100672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588eebb70f0 con 0x5588eef7ef20 2024-03-21T09:22:33.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.087+0000 7f3b89e00640 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef944c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00c760 tx=0x7f3b7c0d3de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64cdcdc273c9b15e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b76a00640 1 -- 172.21.15.37:0/1478100672 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c09be00 con 0x5588eef7ef20 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b76a00640 1 -- 172.21.15.37:0/1478100672 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c09c9c0 con 0x5588eef7ef20 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b76a00640 1 -- 172.21.15.37:0/1478100672 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c09bc20 con 0x5588eef7ef20 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1478100672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 msgr2=0x5588eef944c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef944c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00c760 tx=0x7f3b7c0d3de0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1478100672 shutdown_connections 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef88290 0x5588eef87920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef873e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/1478100672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef7ef20 0x5588eef944c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.088+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1478100672 >> 172.21.15.37:0/1478100672 conn(0x5588eef46200 msgr2=0x5588eef794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.089+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1478100672 shutdown_connections 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.089+0000 7f3b93b9e740 1 -- 172.21.15.37:0/1478100672 wait complete. 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.089+0000 7f3b93b9e740 1 -- start start 2024-03-21T09:22:33.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.089+0000 7f3b93b9e740 1 Processor -- start 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 0x5588eef828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1af2a0 con 0x5588eef7fc00 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1b0580 con 0x5588eef88290 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1af8c0 con 0x5588eef7ef20 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b93b9e740 10 client.0 _ll_register_callbacks cb 0x7f3b840094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 0x5588eef828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 0x5588eef828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56230/0 (socket says 172.21.15.37:56230) 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b89e00640 1 -- 172.21.15.37:0/2607083536 learned_addr learned my addr 172.21.15.37:0/2607083536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b8a800640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.090+0000 7f3b89400640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89e00640 1 -- 172.21.15.37:0/2607083536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 msgr2=0x5588eef814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89e00640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89e00640 1 -- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 msgr2=0x5588eef80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89e00640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89e00640 1 -- 172.21.15.37:0/2607083536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588eef81ba0 con 0x5588eef7ef20 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89400640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b89e00640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 0x5588eef828f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c09bfc0 tx=0x7f3b7c0d3ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8415bcc34de2ac65 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b6be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b6be00640 1 -- 172.21.15.37:0/2607083536 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c09bc20 con 0x5588eef7ef20 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b6be00640 1 -- 172.21.15.37:0/2607083536 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3b7c09a5c0 con 0x5588eef7ef20 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b7c09c9c0 con 0x5588eef7ef20 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b6be00640 1 -- 172.21.15.37:0/2607083536 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b7c09be00 con 0x5588eef7ef20 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b93b9e740 10 client.27970 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.091+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5588eeff33c0 con 0x5588eef7ef20 2024-03-21T09:22:33.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b6be00640 1 -- 172.21.15.37:0/2607083536 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c09c9c0 con 0x5588eef7ef20 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b6be00640 1 -- 172.21.15.37:0/2607083536 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3b7c0cd8a0 con 0x5588eef7ef20 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b93b9e740 10 client.27970 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b93b9e740 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b93b9e740 20 client.27970 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eeff3a40 0x5588eeff5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b93b9e740 4 client.27970 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588eeeeaae0 con 0x5588eeff3a40 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.092+0000 7f3b89400640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eeff3a40 0x5588eeff5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b89400640 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eeff3a40 0x5588eeff5e60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df619ff98718a1e4 server_cookie=39e0b5f73812feb6 in_seq=0 out_seq=0 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b6be00640 10 client.27970 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b6be00640 1 -- 172.21.15.37:0/2607083536 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5588eeeeaae0 con 0x5588eeff3a40 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b6be00640 10 client.27970 handle_command_reply: tid=0 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 client.27970 shutdown 2024-03-21T09:22:33.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 msgr2=0x5588eef828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 0x5588eef828f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c09bfc0 tx=0x7f3b7c0d3ed0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 shutdown_connections 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x5588eeff3a40 0x5588eeff5e60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588eef88290 0x5588eef814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588eef7fc00 0x5588eef80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 --2- 172.21.15.37:0/2607083536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eef7ef20 0x5588eef828f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 >> 172.21.15.37:0/2607083536 conn(0x5588eef46200 msgr2=0x5588eef84aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b6be00640 0 client.27970 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 shutdown_connections 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.093+0000 7f3b93b9e740 1 -- 172.21.15.37:0/2607083536 wait complete. 2024-03-21T09:22:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b93b9e740 20 client.27970 trim_cache size 0 max 16384 2024-03-21T09:22:33.096 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:33.096 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-03-21T09:22:33.096 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-03-21T09:22:33.096 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3b700a3110 msgr2=0x7f3b700a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3b700a3110 0x7f3b700a55d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009300 tx=0x7f3b84009190 comp rx=0 tx=0).stop 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c103830 msgr2=0x7f3b8c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c103830 0x7f3b8c19a240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009fd0 tx=0x7f3b7c00ef90 comp rx=0 tx=0).stop 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 shutdown_connections 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f3b700a3110 0x7f3b700a55d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b8c109840 0x7f3b8c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b8c1041f0 0x7f3b8c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 --2- 172.21.15.37:0/802262611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b8c103830 0x7f3b8c19a240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.094+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 >> 172.21.15.37:0/802262611 conn(0x7f3b8c0ff320 msgr2=0x7f3b8c0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.095+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 shutdown_connections 2024-03-21T09:22:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.095+0000 7f3b90e00640 1 -- 172.21.15.37:0/802262611 wait complete. 2024-03-21T09:22:33.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T09:22:33.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 -- 172.21.15.37:0/1041788991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 msgr2=0x7f7128109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 --2- 172.21.15.37:0/1041788991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 0x7f7128109c50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7110009930 tx=0x7f711002bc30 comp rx=0 tx=0).stop 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 -- 172.21.15.37:0/1041788991 shutdown_connections 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 --2- 172.21.15.37:0/1041788991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7128104220 0x7f7128110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 --2- 172.21.15.37:0/1041788991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128103860 0x7f7128103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 --2- 172.21.15.37:0/1041788991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 0x7f7128109c50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.325+0000 7f712ea00640 1 -- 172.21.15.37:0/1041788991 >> 172.21.15.37:0/1041788991 conn(0x7f71280ff2f0 msgr2=0x7f7128101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.326+0000 7f712ea00640 1 -- 172.21.15.37:0/1041788991 shutdown_connections 2024-03-21T09:22:33.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.326+0000 7f712ea00640 1 -- 172.21.15.37:0/1041788991 wait complete. 2024-03-21T09:22:33.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.327+0000 7f712ea00640 1 Processor -- start 2024-03-21T09:22:33.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.327+0000 7f712ea00640 1 -- start start 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7128103860 0x7f712819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 0x7f712819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 0x7f712819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128113c90 con 0x7f7128103860 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128113b10 con 0x7f7128104220 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128113e10 con 0x7f7128109870 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f7127e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7128103860 0x7f712819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f7127400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 0x7f712819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f7127400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 0x7f712819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:41080/0 (socket says 172.21.15.37:41080) 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f7127400640 1 -- 172.21.15.37:0/3333758981 learned_addr learned my addr 172.21.15.37:0/3333758981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.328+0000 7f712ca00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 0x7f712819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7127400640 1 -- 172.21.15.37:0/3333758981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 msgr2=0x7f712819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7127400640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 0x7f712819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7127400640 1 -- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7128103860 msgr2=0x7f712819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7127400640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7128103860 0x7f712819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7127400640 1 -- 172.21.15.37:0/3333758981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712819f160 con 0x7f7128104220 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7127400640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 0x7f712819a6b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f711800ea60 tx=0x7f711800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800cde0 con 0x7f7128104220 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.329+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71180035f0 con 0x7f7128104220 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.330+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118010670 con 0x7f7128104220 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.330+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712819f450 con 0x7f7128104220 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.330+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71281a6ce0 con 0x7f7128104220 2024-03-21T09:22:33.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.331+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128068970 con 0x7f7128104220 2024-03-21T09:22:33.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.333+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f71180037a0 con 0x7f7128104220 2024-03-21T09:22:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.334+0000 7f7124c00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f710c0a3110 0x7f710c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.334+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f71180cfeb0 con 0x7f7128104220 2024-03-21T09:22:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.334+0000 7f7127e00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f710c0a3110 0x7f710c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.335+0000 7f7127e00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f710c0a3110 0x7f710c0a55d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7110009900 tx=0x7f711002df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.338+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f711809c540 con 0x7f7128104220 2024-03-21T09:22:33.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:33 smithi037 ceph-mon[31830]: pgmap v322: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.693+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f712810b840 con 0x7f7128104220 2024-03-21T09:22:33.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.694+0000 7f7124c00640 1 -- 172.21.15.37:0/3333758981 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7f711809be40 con 0x7f7128104220 2024-03-21T09:22:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:22:33.693 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:22:33.693 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:22:33.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:33 smithi053 ceph-mon[36169]: pgmap v322: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f710c0a3110 msgr2=0x7f710c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f710c0a3110 0x7f710c0a55d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7110009900 tx=0x7f711002df90 comp rx=0 tx=0).stop 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 msgr2=0x7f712819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 0x7f712819a6b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f711800ea60 tx=0x7f711800ef20 comp rx=0 tx=0).stop 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 shutdown_connections 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7128109870 0x7f712819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f710c0a3110 0x7f710c0a55d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7128104220 0x7f712819a6b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 --2- 172.21.15.37:0/3333758981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7128103860 0x7f712819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 >> 172.21.15.37:0/3333758981 conn(0x7f71280ff2f0 msgr2=0x7f71280ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 shutdown_connections 2024-03-21T09:22:33.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.697+0000 7f712ea00640 1 -- 172.21.15.37:0/3333758981 wait complete. 2024-03-21T09:22:33.713 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:22:33.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T09:22:33.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:33 smithi071 ceph-mon[36597]: pgmap v322: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 -- 172.21.15.37:0/2288347546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6934101e80 msgr2=0x7f693410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 --2- 172.21.15.37:0/2288347546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6934101e80 0x7f693410e340 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f692c009cd0 tx=0x7f692c02c0e0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 -- 172.21.15.37:0/2288347546 shutdown_connections 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 --2- 172.21.15.37:0/2288347546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6934101e80 0x7f693410e340 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 --2- 172.21.15.37:0/2288347546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69341014c0 0x7f6934101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 --2- 172.21.15.37:0/2288347546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 -- 172.21.15.37:0/2288347546 >> 172.21.15.37:0/2288347546 conn(0x7f69340fcfb0 msgr2=0x7f69340ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 -- 172.21.15.37:0/2288347546 shutdown_connections 2024-03-21T09:22:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.924+0000 7f6938a00640 1 -- 172.21.15.37:0/2288347546 wait complete. 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.925+0000 7f6938a00640 1 Processor -- start 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.925+0000 7f6938a00640 1 -- start start 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6938a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 0x7f69340739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6938a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934101e80 0x7f6934073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6938a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6938a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934111860 con 0x7f6934101e80 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6938a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341116e0 con 0x7f6934107580 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6938a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341119e0 con 0x7f69341014c0 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 0x7f69340739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6933200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f692be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934101e80 0x7f6934073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 0x7f69340739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54898/0 (socket says 172.21.15.37:54898) 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 -- 172.21.15.37:0/1026250385 learned_addr learned my addr 172.21.15.37:0/1026250385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6933200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58490/0 (socket says 172.21.15.37:58490) 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 -- 172.21.15.37:0/1026250385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 msgr2=0x7f6934074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 -- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934101e80 msgr2=0x7f6934073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934101e80 0x7f6934073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6932800640 1 -- 172.21.15.37:0/1026250385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69341a61c0 con 0x7f69341014c0 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.926+0000 7f6933200640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.927+0000 7f6932800640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 0x7f69340739a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f692400e9f0 tx=0x7f692400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.927+0000 7f692b400640 1 -- 172.21.15.37:0/1026250385 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692400cd80 con 0x7f69341014c0 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.927+0000 7f692b400640 1 -- 172.21.15.37:0/1026250385 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6924005500 con 0x7f69341014c0 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.927+0000 7f692b400640 1 -- 172.21.15.37:0/1026250385 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924010690 con 0x7f69341014c0 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.927+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69341a64b0 con 0x7f69341014c0 2024-03-21T09:22:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.927+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69341a69b0 con 0x7f69341014c0 2024-03-21T09:22:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.930+0000 7f692b400640 1 -- 172.21.15.37:0/1026250385 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f69240046c0 con 0x7f69341014c0 2024-03-21T09:22:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.930+0000 7f693c1ad740 1 Processor -- start 2024-03-21T09:22:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.930+0000 7f693c1ad740 1 -- start start 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f693c1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6dd7d2c0 con 0x557d6de22de0 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f693c1ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6dd25c90 con 0x557d6de1a260 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f693c1ad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6dd79360 con 0x557d6de198b0 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f6932800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f6932800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58498/0 (socket says 172.21.15.37:58498) 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f6932800640 1 -- 172.21.15.37:0/3720772908 learned_addr learned my addr 172.21.15.37:0/3720772908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.931+0000 7f6933200640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f692be00640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6932800640 1 -- 172.21.15.37:0/3720772908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 msgr2=0x557d6de19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6932800640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6932800640 1 -- 172.21.15.37:0/3720772908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 msgr2=0x557d6de25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6932800640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de25230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6932800640 1 -- 172.21.15.37:0/3720772908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d6dd4a080 con 0x557d6de1a260 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6933200640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f692be00640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:22:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6932800640 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f69240cd190 tx=0x7f69240d3f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332229c3c3401668 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6923400640 1 -- 172.21.15.37:0/3720772908 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240e3070 con 0x557d6de1a260 2024-03-21T09:22:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6923400640 1 -- 172.21.15.37:0/3720772908 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69240d8cb0 con 0x557d6de1a260 2024-03-21T09:22:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.932+0000 7f6923400640 1 -- 172.21.15.37:0/3720772908 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240dd7a0 con 0x557d6de1a260 2024-03-21T09:22:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f692b400640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69180a3110 0x7f69180a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 -- 172.21.15.37:0/3720772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 msgr2=0x557d6de22810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f69240cd190 tx=0x7f69240d3f70 comp rx=0 tx=0).stop 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f692b400640 1 -- 172.21.15.37:0/1026250385 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f69240cff60 con 0x7f69341014c0 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f6933200640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69180a3110 0x7f69180a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 -- 172.21.15.37:0/3720772908 shutdown_connections 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22810 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 --2- 172.21.15.37:0/3720772908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 -- 172.21.15.37:0/3720772908 >> 172.21.15.37:0/3720772908 conn(0x557d6de13f80 msgr2=0x557d6de14360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 -- 172.21.15.37:0/3720772908 shutdown_connections 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.933+0000 7f693c1ad740 1 -- 172.21.15.37:0/3720772908 wait complete. 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.934+0000 7f693c1ad740 1 -- start start 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.934+0000 7f6933200640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69180a3110 0x7f69180a55d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f692c02c620 tx=0x7f692c02e040 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.934+0000 7f693c1ad740 1 Processor -- start 2024-03-21T09:22:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de1fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de20250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6de25d50 con 0x557d6de22de0 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6dd390d0 con 0x557d6de1a260 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6de25ed0 con 0x557d6de198b0 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f693c1ad740 10 client.0 _ll_register_callbacks cb 0x557d6dd3de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f692be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de1fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de22780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54910/0 (socket says 172.21.15.37:54910) 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 -- 172.21.15.37:0/1496956653 learned_addr learned my addr 172.21.15.37:0/1496956653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6933200640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de20250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 -- 172.21.15.37:0/1496956653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 msgr2=0x557d6de1fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de1fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 -- 172.21.15.37:0/1496956653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 msgr2=0x557d6de20250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de20250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6932800640 1 -- 172.21.15.37:0/1496956653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d6da913f0 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.935+0000 7f6933200640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f6932800640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de22780 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f692409bbf0 tx=0x7f69240dee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c129a0822344500 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f6917e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f6917e00640 1 -- 172.21.15.37:0/1496956653 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924098190 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f6917e00640 1 -- 172.21.15.37:0/1496956653 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f692409a1f0 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69240d8cb0 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f6917e00640 1 -- 172.21.15.37:0/1496956653 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692409b850 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f693c1ad740 10 client.27988 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557d6db5b7d0 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.936+0000 7f6917e00640 1 -- 172.21.15.37:0/1496956653 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69240d8cb0 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f6917e00640 1 -- 172.21.15.37:0/1496956653 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f69240cd4d0 con 0x557d6de198b0 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f693c1ad740 10 client.27988 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f693c1ad740 10 client.27988 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f693c1ad740 20 client.27988 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de32ad0 0x557d6de34ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f693c1ad740 4 client.27988 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d6dd8a800 con 0x557d6de32ad0 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f692be00640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de32ad0 0x557d6de34ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f692be00640 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de32ad0 0x557d6de34ec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f676c7d006bc22f server_cookie=f9957c2923c2f2a5 in_seq=0 out_seq=0 2024-03-21T09:22:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.937+0000 7f6917e00640 10 client.27988 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f6917e00640 1 -- 172.21.15.37:0/1496956653 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557d6dd8a800 con 0x557d6de32ad0 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f6917e00640 10 client.27988 handle_command_reply: tid=0 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 client.27988 shutdown 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 msgr2=0x557d6de22780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de22780 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f692409bbf0 tx=0x7f69240dee70 comp rx=0 tx=0).stop 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 shutdown_connections 2024-03-21T09:22:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de32ad0 0x557d6de34ec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de1fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1496956653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de22780 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.938+0000 7f6917e00640 0 client.27988 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.939+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 >> 172.21.15.37:0/1496956653 conn(0x557d6de13f80 msgr2=0x557d6de15cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.939+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 shutdown_connections 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.939+0000 7f693c1ad740 1 -- 172.21.15.37:0/1496956653 wait complete. 2024-03-21T09:22:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.939+0000 7f693c1ad740 20 client.27988 trim_cache size 0 max 16384 2024-03-21T09:22:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.952+0000 7f693c1ad740 1 Processor -- start 2024-03-21T09:22:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 -- start start 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de198b0 0x557d6de133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de22de0 0x557d6de22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341b2fe0 con 0x557d6de198b0 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341b0ae0 con 0x557d6de1a260 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f693c1ad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341aeba0 con 0x557d6de22de0 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f692be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f6933200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de22de0 0x557d6de22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f692be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58516/0 (socket says 172.21.15.37:58516) 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f692be00640 1 -- 172.21.15.37:0/1469174336 learned_addr learned my addr 172.21.15.37:0/1469174336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.953+0000 7f6932800640 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de198b0 0x557d6de133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f692be00640 1 -- 172.21.15.37:0/1469174336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de22de0 msgr2=0x557d6de22780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f692be00640 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de22de0 0x557d6de22780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f692be00640 1 -- 172.21.15.37:0/1469174336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de198b0 msgr2=0x557d6de133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f692be00640 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de198b0 0x557d6de133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f692be00640 1 -- 172.21.15.37:0/1469174336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d6de2faf0 con 0x557d6de1a260 2024-03-21T09:22:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f692be00640 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22240 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f691c005940 tx=0x7f691c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=971c5a69b078c730 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f6922a00640 1 -- 172.21.15.37:0/1469174336 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c01e250 con 0x557d6de1a260 2024-03-21T09:22:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f6922a00640 1 -- 172.21.15.37:0/1469174336 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f691c01eba0 con 0x557d6de1a260 2024-03-21T09:22:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.954+0000 7f6922a00640 1 -- 172.21.15.37:0/1469174336 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c0126b0 con 0x557d6de1a260 2024-03-21T09:22:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 -- 172.21.15.37:0/1469174336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 msgr2=0x557d6de22240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22240 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f691c005940 tx=0x7f691c004610 comp rx=0 tx=0).stop 2024-03-21T09:22:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 -- 172.21.15.37:0/1469174336 shutdown_connections 2024-03-21T09:22:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de22de0 0x557d6de22780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de22240 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 --2- 172.21.15.37:0/1469174336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de198b0 0x557d6de133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 -- 172.21.15.37:0/1469174336 >> 172.21.15.37:0/1469174336 conn(0x557d6de13f80 msgr2=0x557d6de1aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 -- 172.21.15.37:0/1469174336 shutdown_connections 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 -- 172.21.15.37:0/1469174336 wait complete. 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.955+0000 7f693c1ad740 1 -- start start 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 Processor -- start 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de35e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341af1a0 con 0x557d6de22de0 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341b0480 con 0x557d6de1a260 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69341af7c0 con 0x557d6de198b0 2024-03-21T09:22:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f693c1ad740 10 client.0 _ll_register_callbacks cb 0x7f69240e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f6932800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f6932800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de37a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54928/0 (socket says 172.21.15.37:54928) 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f6932800640 1 -- 172.21.15.37:0/4156410565 learned_addr learned my addr 172.21.15.37:0/4156410565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f692be00640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de35e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.956+0000 7f6933200640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6932800640 1 -- 172.21.15.37:0/4156410565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 msgr2=0x557d6de35e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6932800640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de35e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6932800640 1 -- 172.21.15.37:0/4156410565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 msgr2=0x557d6de36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6932800640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6932800640 1 -- 172.21.15.37:0/4156410565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d6de36910 con 0x557d6de198b0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6932800640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de37a70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f692409be80 tx=0x7f6924005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a92b0c41eede64 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6917e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6917e00640 1 -- 172.21.15.37:0/4156410565 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692409ba30 con 0x557d6de198b0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f6917e00640 1 -- 172.21.15.37:0/4156410565 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f69240a11d0 con 0x557d6de198b0 2024-03-21T09:22:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.957+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f691c01eba0 con 0x557d6de198b0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f6917e00640 1 -- 172.21.15.37:0/4156410565 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692409bcc0 con 0x557d6de198b0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 10 client.27994 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557d6de8ede0 con 0x557d6de198b0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f6917e00640 1 -- 172.21.15.37:0/4156410565 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f691c01eba0 con 0x557d6de198b0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f6917e00640 1 -- 172.21.15.37:0/4156410565 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f69240cd4d0 con 0x557d6de198b0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 10 client.27994 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 10 client.27994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 20 client.27994 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de8f1a0 0x557d6de915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 4 client.27994 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 --> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d6dd84b50 con 0x557d6de8f1a0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.958+0000 7f692be00640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de8f1a0 0x557d6de915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f692be00640 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de8f1a0 0x557d6de915c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40d8836a89d6c566 server_cookie=9fa603dd99fa277a in_seq=0 out_seq=0 2024-03-21T09:22:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f6917e00640 10 client.27994 ms_handle_connect on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f6917e00640 1 -- 172.21.15.37:0/4156410565 <== mds.2 v2:172.21.15.71:6838/543433986 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d6dd84b50 con 0x557d6de8f1a0 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f6917e00640 10 client.27994 handle_command_reply: tid=0 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 client.27994 shutdown 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 msgr2=0x557d6de37a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de37a70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f692409be80 tx=0x7f6924005b40 comp rx=0 tx=0).stop 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 shutdown_connections 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:6838/543433986,v1:172.21.15.71:6839/543433986] conn(0x557d6de8f1a0 0x557d6de915c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d6de22de0 0x557d6de36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d6de1a260 0x557d6de35e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f6917e00640 0 client.27994 ms_handle_reset on v2:172.21.15.71:6838/543433986 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 --2- 172.21.15.37:0/4156410565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6de198b0 0x557d6de37a70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 >> 172.21.15.37:0/4156410565 conn(0x557d6de13f80 msgr2=0x557d6de14fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 shutdown_connections 2024-03-21T09:22:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.959+0000 7f693c1ad740 1 -- 172.21.15.37:0/4156410565 wait complete. 2024-03-21T09:22:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f693c1ad740 20 client.27994 trim_cache size 0 max 16384 2024-03-21T09:22:33.963 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69180a3110 msgr2=0x7f69180a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69180a3110 0x7f69180a55d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f692c02c620 tx=0x7f692c02e040 comp rx=0 tx=0).stop 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 msgr2=0x7f69340739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 0x7f69340739a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f692400e9f0 tx=0x7f692400edf0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 shutdown_connections 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f69180a3110 0x7f69180a55d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6934107580 0x7f6934074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934101e80 0x7f6934073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 --2- 172.21.15.37:0/1026250385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69341014c0 0x7f69340739a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 >> 172.21.15.37:0/1026250385 conn(0x7f69340fcfb0 msgr2=0x7f69340fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.960+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 shutdown_connections 2024-03-21T09:22:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:33.961+0000 7f6938a00640 1 -- 172.21.15.37:0/1026250385 wait complete. 2024-03-21T09:22:33.976 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:22:33.976 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T09:22:34.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 -- 172.21.15.37:0/4020383311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318101e20 msgr2=0x7f331810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 --2- 172.21.15.37:0/4020383311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318101e20 0x7f331810e330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3310009ca0 tx=0x7f331002c100 comp rx=0 tx=0).stop 2024-03-21T09:22:34.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 -- 172.21.15.37:0/4020383311 shutdown_connections 2024-03-21T09:22:34.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 --2- 172.21.15.37:0/4020383311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318101e20 0x7f331810e330 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 --2- 172.21.15.37:0/4020383311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3318101460 0x7f33181018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 --2- 172.21.15.37:0/4020383311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318107500 0x7f33181078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 -- 172.21.15.37:0/4020383311 >> 172.21.15.37:0/4020383311 conn(0x7f33180fcff0 msgr2=0x7f33180ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 -- 172.21.15.37:0/4020383311 shutdown_connections 2024-03-21T09:22:34.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.195+0000 7f331d600640 1 -- 172.21.15.37:0/4020383311 wait complete. 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.196+0000 7f331d600640 1 Processor -- start 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.196+0000 7f331d600640 1 -- start start 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318101460 0x7f3318199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3318101e20 0x7f331819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 0x7f331819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331d600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318111810 con 0x7f3318101e20 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331d600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318111690 con 0x7f3318107500 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318111990 con 0x7f3318101460 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f3317e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318101460 0x7f3318199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 0x7f331819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f3317e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318101460 0x7f3318199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54940/0 (socket says 172.21.15.37:54940) 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f3317e00640 1 -- 172.21.15.37:0/71442888 learned_addr learned my addr 172.21.15.37:0/71442888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f331cc00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 0x7f331819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58530/0 (socket says 172.21.15.37:58530) 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.197+0000 7f330fe00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3318101e20 0x7f331819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331cc00640 1 -- 172.21.15.37:0/71442888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318101460 msgr2=0x7f3318199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331cc00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318101460 0x7f3318199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331cc00640 1 -- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3318101e20 msgr2=0x7f331819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331cc00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3318101e20 0x7f331819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331cc00640 1 -- 172.21.15.37:0/71442888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331819eef0 con 0x7f3318107500 2024-03-21T09:22:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331cc00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 0x7f331819e7c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3310009bf0 tx=0x7f3310034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f3315600640 1 -- 172.21.15.37:0/71442888 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310039050 con 0x7f3318107500 2024-03-21T09:22:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f3315600640 1 -- 172.21.15.37:0/71442888 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3310045070 con 0x7f3318107500 2024-03-21T09:22:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331819f180 con 0x7f3318107500 2024-03-21T09:22:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f3315600640 1 -- 172.21.15.37:0/71442888 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100045e0 con 0x7f3318107500 2024-03-21T09:22:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.198+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33181a6a70 con 0x7f3318107500 2024-03-21T09:22:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f3315600640 1 -- 172.21.15.37:0/71442888 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3310003970 con 0x7f3318107500 2024-03-21T09:22:34.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 Processor -- start 2024-03-21T09:22:34.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 -- start start 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff27810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff2a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862fe822c0 con 0x55862ff1e8b0 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862fe2ac90 con 0x55862ff27de0 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f332199e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862fe7e360 con 0x55862ff1f260 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f3317e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff27810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f330fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.202+0000 7f3317e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff27810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54948/0 (socket says 172.21.15.37:54948) 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f330fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff1ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57700/0 (socket says 172.21.15.37:57700) 2024-03-21T09:22:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 -- 172.21.15.37:0/2880121350 learned_addr learned my addr 172.21.15.37:0/2880121350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 -- 172.21.15.37:0/2880121350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 msgr2=0x55862ff2a230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff2a230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 -- 172.21.15.37:0/2880121350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 msgr2=0x55862ff1ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff1ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 -- 172.21.15.37:0/2880121350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862fe4f080 con 0x55862ff1f260 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3315600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f32fc0a3110 0x7f32fc0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3317e00640 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff27810 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3308009d40 tx=0x7f3308011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de74fc382ae7956 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.203+0000 7f3315600640 1 -- 172.21.15.37:0/71442888 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f33100f2210 con 0x7f3318107500 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f330fe00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f32fc0a3110 0x7f32fc0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f330d200640 1 -- 172.21.15.37:0/2880121350 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308004220 con 0x55862ff1f260 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f330d200640 1 -- 172.21.15.37:0/2880121350 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33080043c0 con 0x55862ff1f260 2024-03-21T09:22:34.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f330d200640 1 -- 172.21.15.37:0/2880121350 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33080035f0 con 0x55862ff1f260 2024-03-21T09:22:34.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f332199e740 1 -- 172.21.15.37:0/2880121350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 msgr2=0x55862ff27810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f332199e740 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff27810 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3308009d40 tx=0x7f3308011810 comp rx=0 tx=0).stop 2024-03-21T09:22:34.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.204+0000 7f330fe00640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f32fc0a3110 0x7f32fc0a55d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3300005fd0 tx=0x7f330000b550 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 -- 172.21.15.37:0/2880121350 shutdown_connections 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff2a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff27810 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 --2- 172.21.15.37:0/2880121350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 -- 172.21.15.37:0/2880121350 >> 172.21.15.37:0/2880121350 conn(0x55862ff18f80 msgr2=0x55862ff19360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 -- 172.21.15.37:0/2880121350 shutdown_connections 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 -- 172.21.15.37:0/2880121350 wait complete. 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 -- start start 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.205+0000 7f332199e740 1 Processor -- start 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff27640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff24f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ff2ada0 con 0x55862ff1e8b0 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862fe3e0d0 con 0x55862ff27de0 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ff2af20 con 0x55862ff1f260 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f332199e740 10 client.0 _ll_register_callbacks cb 0x55862fe42e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f330fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f331cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff24f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f331cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff24f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58546/0 (socket says 172.21.15.37:58546) 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f331cc00640 1 -- 172.21.15.37:0/2873991983 learned_addr learned my addr 172.21.15.37:0/2873991983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f330fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54962/0 (socket says 172.21.15.37:54962) 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.206+0000 7f3317e00640 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff27640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f331cc00640 1 -- 172.21.15.37:0/2873991983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 msgr2=0x55862ff249d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f331cc00640 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff249d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f331cc00640 1 -- 172.21.15.37:0/2873991983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 msgr2=0x55862ff27640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f331cc00640 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff27640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f331cc00640 1 -- 172.21.15.37:0/2873991983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55862fb963f0 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f331cc00640 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff24f10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f33100bdfb0 tx=0x7f33100bdcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f26e4690855070 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f3305600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f3305600640 1 -- 172.21.15.37:0/2873991983 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100c2400 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33080043c0 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f3305600640 1 -- 172.21.15.37:0/2873991983 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3310004780 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f3305600640 1 -- 172.21.15.37:0/2873991983 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33100c6cc0 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f332199e740 10 client.27983 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.207+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55862fc607d0 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f3305600640 1 -- 172.21.15.37:0/2873991983 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33080043c0 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f3305600640 1 -- 172.21.15.37:0/2873991983 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f33100c6830 con 0x55862ff27de0 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f332199e740 10 client.27983 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f332199e740 10 client.27983 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:22:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f332199e740 20 client.27983 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f332199e740 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff37bf0 0x55862ff3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f332199e740 4 client.27983 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55862fe8f800 con 0x55862ff37bf0 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.208+0000 7f3317e00640 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff37bf0 0x55862ff3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.209+0000 7f3317e00640 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff37bf0 0x55862ff3a000 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e5eb20bbc7bfc46 server_cookie=da42df57cfa207cf in_seq=0 out_seq=0 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.209+0000 7f3305600640 10 client.27983 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.209+0000 7f3305600640 1 -- 172.21.15.37:0/2873991983 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55862fe8f800 con 0x55862ff37bf0 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.209+0000 7f3305600640 10 client.27983 handle_command_reply: tid=0 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.209+0000 7f332199e740 1 client.27983 shutdown 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 msgr2=0x55862ff24f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff24f10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f33100bdfb0 tx=0x7f33100bdcd0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 shutdown_connections 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff37bf0 0x55862ff3a000 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff27de0 0x55862ff24f10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff249d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f3305600640 0 client.27983 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 --2- 172.21.15.37:0/2873991983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff1e8b0 0x55862ff27640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 >> 172.21.15.37:0/2873991983 conn(0x55862ff18f80 msgr2=0x55862ff1acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 shutdown_connections 2024-03-21T09:22:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 1 -- 172.21.15.37:0/2873991983 wait complete. 2024-03-21T09:22:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.210+0000 7f332199e740 20 client.27983 trim_cache size 0 max 16384 2024-03-21T09:22:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 Processor -- start 2024-03-21T09:22:34.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 -- start start 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 0x55862ff1aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1f260 0x55862ff27100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff27640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33181b2fe0 con 0x55862ff27de0 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33181b0ae0 con 0x55862ff1f260 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.225+0000 7f332199e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33181aeba0 con 0x55862ff1e8b0 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 0x55862ff1aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 0x55862ff1aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54972/0 (socket says 172.21.15.37:54972) 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 -- 172.21.15.37:0/1695324131 learned_addr learned my addr 172.21.15.37:0/1695324131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f330fe00640 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1f260 0x55862ff27100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f331cc00640 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff27640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 -- 172.21.15.37:0/1695324131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1f260 msgr2=0x55862ff27100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1f260 0x55862ff27100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 -- 172.21.15.37:0/1695324131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 msgr2=0x55862ff27640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff27640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 -- 172.21.15.37:0/1695324131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862ff1e480 con 0x55862ff1e8b0 2024-03-21T09:22:34.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f331cc00640 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff27640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3317e00640 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 0x55862ff1aa70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3308004a50 tx=0x7f3308004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8b160ca07e7ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3307e00640 1 -- 172.21.15.37:0/1695324131 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330801d040 con 0x55862ff1e8b0 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3307e00640 1 -- 172.21.15.37:0/1695324131 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33080035f0 con 0x55862ff1e8b0 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.226+0000 7f3307e00640 1 -- 172.21.15.37:0/1695324131 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308004520 con 0x55862ff1e8b0 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 -- 172.21.15.37:0/1695324131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 msgr2=0x55862ff1aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 0x55862ff1aa70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3308004a50 tx=0x7f3308004970 comp rx=0 tx=0).stop 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 -- 172.21.15.37:0/1695324131 shutdown_connections 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff27640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1f260 0x55862ff27100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 --2- 172.21.15.37:0/1695324131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1e8b0 0x55862ff1aa70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 -- 172.21.15.37:0/1695324131 >> 172.21.15.37:0/1695324131 conn(0x55862ff18f80 msgr2=0x55862ff1a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 -- 172.21.15.37:0/1695324131 shutdown_connections 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.227+0000 7f332199e740 1 -- 172.21.15.37:0/1695324131 wait complete. 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.228+0000 7f332199e740 1 -- start start 2024-03-21T09:22:34.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.228+0000 7f332199e740 1 Processor -- start 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 0x55862ff2bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff2c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33181af1a0 con 0x55862ff27de0 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33181b0480 con 0x55862ff1e8b0 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33181af7c0 con 0x55862ff1f260 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f332199e740 10 client.0 _ll_register_callbacks cb 0x7f3310003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f3317e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 0x55862ff2bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f330fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff2c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f3317e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 0x55862ff2bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58556/0 (socket says 172.21.15.37:58556) 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f3317e00640 1 -- 172.21.15.37:0/3593205075 learned_addr learned my addr 172.21.15.37:0/3593205075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.229+0000 7f331cc00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3317e00640 1 -- 172.21.15.37:0/3593205075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 msgr2=0x55862ff2c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3317e00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff2c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3317e00640 1 -- 172.21.15.37:0/3593205075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 msgr2=0x55862ff924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3317e00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3317e00640 1 -- 172.21.15.37:0/3593205075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55862ff92a10 con 0x55862ff1e8b0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f330fe00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f331cc00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3317e00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 0x55862ff2bce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3308004940 tx=0x7f3308009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9408d92e51b3f655 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3305600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3305600640 1 -- 172.21.15.37:0/3593205075 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330803a060 con 0x55862ff1e8b0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33080035f0 con 0x55862ff1e8b0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.230+0000 7f3305600640 1 -- 172.21.15.37:0/3593205075 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f33080037a0 con 0x55862ff1e8b0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f3305600640 1 -- 172.21.15.37:0/3593205075 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3308041050 con 0x55862ff1e8b0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 10 client.27989 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55862ff93d10 con 0x55862ff1e8b0 2024-03-21T09:22:34.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f3305600640 1 -- 172.21.15.37:0/3593205075 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33080035f0 con 0x55862ff1e8b0 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f3305600640 1 -- 172.21.15.37:0/3593205075 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f33080266f0 con 0x55862ff1e8b0 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 10 client.27989 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 10 client.27989 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 20 client.27989 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff94190 0x55862ff96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 4 client.27989 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 --> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55862fe89b50 con 0x55862ff94190 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.231+0000 7f330fe00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff94190 0x55862ff96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f3305600640 10 client.27989 ms_handle_connect on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f330fe00640 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff94190 0x55862ff96580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11da2d9f4e6a2292 server_cookie=21282bb223f95cc3 in_seq=0 out_seq=0 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f3305600640 1 -- 172.21.15.37:0/3593205075 <== mds.1 v2:172.21.15.53:6838/380096256 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55862fe89b50 con 0x55862ff94190 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f3305600640 10 client.27989 handle_command_reply: tid=0 2024-03-21T09:22:34.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 client.27989 shutdown 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 msgr2=0x55862ff2bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 0x55862ff2bce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3308004940 tx=0x7f3308009fa0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 shutdown_connections 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:6838/380096256,v1:172.21.15.53:6839/380096256] conn(0x55862ff94190 0x55862ff96580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55862ff27de0 0x55862ff924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55862ff1f260 0x55862ff2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f3305600640 0 client.27989 ms_handle_reset on v2:172.21.15.53:6838/380096256 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 --2- 172.21.15.37:0/3593205075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55862ff1e8b0 0x55862ff2bce0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.232+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 >> 172.21.15.37:0/3593205075 conn(0x55862ff18f80 msgr2=0x55862ff1fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 shutdown_connections 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f332199e740 1 -- 172.21.15.37:0/3593205075 wait complete. 2024-03-21T09:22:34.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f332199e740 20 client.27989 trim_cache size 0 max 16384 2024-03-21T09:22:34.236 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:22:34.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f32fc0a3110 msgr2=0x7f32fc0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f331d600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f32fc0a3110 0x7f32fc0a55d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3300005fd0 tx=0x7f330000b550 comp rx=0 tx=0).stop 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 msgr2=0x7f331819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.233+0000 7f331d600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 0x7f331819e7c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3310009bf0 tx=0x7f3310034670 comp rx=0 tx=0).stop 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 shutdown_connections 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f32fc0a3110 0x7f32fc0a55d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3318107500 0x7f331819e7c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3318101e20 0x7f331819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 --2- 172.21.15.37:0/71442888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3318101460 0x7f3318199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 >> 172.21.15.37:0/71442888 conn(0x7f33180fcff0 msgr2=0x7f331810e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 shutdown_connections 2024-03-21T09:22:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.234+0000 7f331d600640 1 -- 172.21.15.37:0/71442888 wait complete. 2024-03-21T09:22:34.251 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T09:22:34.251 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T09:22:34.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 -- 172.21.15.37:0/3884681894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2814112500 msgr2=0x7f28141148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 --2- 172.21.15.37:0/3884681894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2814112500 0x7f28141148f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2810009c10 tx=0x7f281002bef0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 -- 172.21.15.37:0/3884681894 shutdown_connections 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 --2- 172.21.15.37:0/3884681894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2814112500 0x7f28141148f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 --2- 172.21.15.37:0/3884681894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28140764f0 0x7f2814111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 --2- 172.21.15.37:0/3884681894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814075bd0 0x7f2814075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 -- 172.21.15.37:0/3884681894 >> 172.21.15.37:0/3884681894 conn(0x7f28140ff310 msgr2=0x7f2814101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 -- 172.21.15.37:0/3884681894 shutdown_connections 2024-03-21T09:22:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.477+0000 7f281c800640 1 -- 172.21.15.37:0/3884681894 wait complete. 2024-03-21T09:22:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.478+0000 7f281c800640 1 Processor -- start 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.478+0000 7f281c800640 1 -- start start 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281c800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2814075bd0 0x7f281419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281c800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28140764f0 0x7f281419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 0x7f28141a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281c800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814118070 con 0x7f2814075bd0 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281c800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814117ef0 con 0x7f28140764f0 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141181f0 con 0x7f2814112500 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 0x7f28141a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f2819e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2814075bd0 0x7f281419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f2819400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28140764f0 0x7f281419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 0x7f28141a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54998/0 (socket says 172.21.15.37:54998) 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 -- 172.21.15.37:0/1781173130 learned_addr learned my addr 172.21.15.37:0/1781173130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 -- 172.21.15.37:0/1781173130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28140764f0 msgr2=0x7f281419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28140764f0 0x7f281419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 -- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2814075bd0 msgr2=0x7f281419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2814075bd0 0x7f281419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.479+0000 7f281a800640 1 -- 172.21.15.37:0/1781173130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28141a3590 con 0x7f2814112500 2024-03-21T09:22:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.480+0000 7f281a800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 0x7f28141a2e60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2810004af0 tx=0x7f281002cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.480+0000 7f280ea00640 1 -- 172.21.15.37:0/1781173130 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810043020 con 0x7f2814112500 2024-03-21T09:22:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.480+0000 7f280ea00640 1 -- 172.21.15.37:0/1781173130 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2810003e50 con 0x7f2814112500 2024-03-21T09:22:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.480+0000 7f280ea00640 1 -- 172.21.15.37:0/1781173130 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281003e6b0 con 0x7f2814112500 2024-03-21T09:22:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.480+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28141a3820 con 0x7f2814112500 2024-03-21T09:22:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.480+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28141ab110 con 0x7f2814112500 2024-03-21T09:22:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.484+0000 7f281f5b6740 1 Processor -- start 2024-03-21T09:22:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f280ea00640 1 -- 172.21.15.37:0/1781173130 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2810046020 con 0x7f2814112500 2024-03-21T09:22:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 -- start start 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ed9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76edc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c76e342c0 con 0x555c76ed9c00 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c76ddcc90 con 0x555c76ed1080 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.485+0000 7f281f5b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c76e30360 con 0x555c76ed06d0 2024-03-21T09:22:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ed9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ed9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58578/0 (socket says 172.21.15.37:58578) 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 -- 172.21.15.37:0/1050308518 learned_addr learned my addr 172.21.15.37:0/1050308518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819400640 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819400640 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76edc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 -- 172.21.15.37:0/1050308518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 msgr2=0x555c76ed0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 -- 172.21.15.37:0/1050308518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 msgr2=0x555c76edc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76edc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 -- 172.21.15.37:0/1050308518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c76e01080 con 0x555c76ed1080 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819400640 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76edc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.486+0000 7f2819e00640 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ed9630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2808009cc0 tx=0x7f2808011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983ca4d689e3d134 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.487+0000 7f280ea00640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27fc0a3110 0x7f27fc0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.487+0000 7f2803000640 1 -- 172.21.15.37:0/1050308518 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808005220 con 0x555c76ed1080 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.487+0000 7f2803000640 1 -- 172.21.15.37:0/1050308518 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2808004c80 con 0x555c76ed1080 2024-03-21T09:22:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.487+0000 7f2803000640 1 -- 172.21.15.37:0/1050308518 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808019970 con 0x555c76ed1080 2024-03-21T09:22:34.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.487+0000 7f280ea00640 1 -- 172.21.15.37:0/1781173130 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28100f1ff0 con 0x7f2814112500 2024-03-21T09:22:34.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.487+0000 7f2819400640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27fc0a3110 0x7f27fc0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 -- 172.21.15.37:0/1050308518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 msgr2=0x555c76ed9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ed9630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2808009cc0 tx=0x7f2808011810 comp rx=0 tx=0).stop 2024-03-21T09:22:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 -- 172.21.15.37:0/1050308518 shutdown_connections 2024-03-21T09:22:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76edc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ed9630 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1050308518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 -- 172.21.15.37:0/1050308518 >> 172.21.15.37:0/1050308518 conn(0x555c76ecada0 msgr2=0x555c76ecb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f2819400640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27fc0a3110 0x7f27fc0a55d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28141017e0 tx=0x7f280400b410 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 -- 172.21.15.37:0/1050308518 shutdown_connections 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 -- 172.21.15.37:0/1050308518 wait complete. 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.488+0000 7f281f5b6740 1 -- start start 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 Processor -- start 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed1080 0x555c76ed6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 0x555c76ed6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c76edcb70 con 0x555c76ed1080 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c76df00d0 con 0x555c76ed9c00 2024-03-21T09:22:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c76edccf0 con 0x555c76ed06d0 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281f5b6740 10 client.0 _ll_register_callbacks cb 0x555c76df4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f2819e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.489+0000 7f281a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 0x555c76ed6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 0x555c76ed6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58588/0 (socket says 172.21.15.37:58588) 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 -- 172.21.15.37:0/1654473958 learned_addr learned my addr 172.21.15.37:0/1654473958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f2819e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55020/0 (socket says 172.21.15.37:55020) 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f2819400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed1080 0x555c76ed6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 -- 172.21.15.37:0/1654473958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 msgr2=0x555c76ed9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 -- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed1080 msgr2=0x555c76ed6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed1080 0x555c76ed6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 -- 172.21.15.37:0/1654473958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c76b483f0 con 0x555c76ed9c00 2024-03-21T09:22:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281a800640 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 0x555c76ed6f30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28100baa80 tx=0x7f28100bda10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefac891892487e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f27f7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f27f7400640 1 -- 172.21.15.37:0/1654473958 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100c2200 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f27f7400640 1 -- 172.21.15.37:0/1654473958 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28100042d0 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.490+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2808004c80 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f27f7400640 1 -- 172.21.15.37:0/1654473958 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28100c6ac0 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 10 client.28001 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555c76c127d0 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f27f7400640 1 -- 172.21.15.37:0/1654473958 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2808004c80 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f27f7400640 1 -- 172.21.15.37:0/1654473958 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f28100c6630 con 0x555c76ed9c00 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 10 client.28001 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 10 client.28001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 20 client.28001 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76ee9a10 0x555c76eebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 4 client.28001 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T09:22:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.491+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c76e41800 con 0x555c76ee9a10 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.492+0000 7f2819e00640 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76ee9a10 0x555c76eebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.492+0000 7f2819e00640 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76ee9a10 0x555c76eebe50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea33d965bb3f8a35 server_cookie=7a1c4b87c6d632d0 in_seq=0 out_seq=0 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.492+0000 7f27f7400640 10 client.28001 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f27f7400640 1 -- 172.21.15.37:0/1654473958 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555c76e41800 con 0x555c76ee9a10 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f27f7400640 10 client.28001 handle_command_reply: tid=0 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 client.28001 shutdown 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 msgr2=0x555c76ed6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 0x555c76ed6f30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28100baa80 tx=0x7f28100bda10 comp rx=0 tx=0).stop 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 shutdown_connections 2024-03-21T09:22:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76ee9a10 0x555c76eebe50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed9c00 0x555c76ed6f30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed1080 0x555c76ed6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1654473958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76ed9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.493+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 >> 172.21.15.37:0/1654473958 conn(0x555c76ecada0 msgr2=0x555c76eccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.494+0000 7f27f7400640 0 client.28001 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.494+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 shutdown_connections 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.494+0000 7f281f5b6740 1 -- 172.21.15.37:0/1654473958 wait complete. 2024-03-21T09:22:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.494+0000 7f281f5b6740 20 client.28001 trim_cache size 0 max 16384 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 Processor -- start 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 -- start start 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 0x555c76ed9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed1080 0x555c76ed9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76ed3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141b7730 con 0x555c76ed9c00 2024-03-21T09:22:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141b5230 con 0x555c76ed06d0 2024-03-21T09:22:34.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f281f5b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141b32f0 con 0x555c76ed1080 2024-03-21T09:22:34.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f2819400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed1080 0x555c76ed9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f2819e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 0x555c76ed9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f2819e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 0x555c76ed9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58594/0 (socket says 172.21.15.37:58594) 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f2819e00640 1 -- 172.21.15.37:0/1259866871 learned_addr learned my addr 172.21.15.37:0/1259866871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.508+0000 7f2819400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed1080 0x555c76ed9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55034/0 (socket says 172.21.15.37:55034) 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f281a800640 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76ed3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2819e00640 1 -- 172.21.15.37:0/1259866871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed1080 msgr2=0x555c76ed9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2819e00640 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed1080 0x555c76ed9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2819e00640 1 -- 172.21.15.37:0/1259866871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 msgr2=0x555c76ed3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2819e00640 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76ed3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2819e00640 1 -- 172.21.15.37:0/1259866871 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c76ed0e20 con 0x555c76ed06d0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2819e00640 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 0x555c76ed9050 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2808005790 tx=0x7f2808009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f70dbe357255c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2802600640 1 -- 172.21.15.37:0/1259866871 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808034070 con 0x555c76ed06d0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2802600640 1 -- 172.21.15.37:0/1259866871 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2808005cb0 con 0x555c76ed06d0 2024-03-21T09:22:34.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.509+0000 7f2802600640 1 -- 172.21.15.37:0/1259866871 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280802e8b0 con 0x555c76ed06d0 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 -- 172.21.15.37:0/1259866871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 msgr2=0x555c76ed9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 0x555c76ed9050 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2808005790 tx=0x7f2808009260 comp rx=0 tx=0).stop 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 -- 172.21.15.37:0/1259866871 shutdown_connections 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76ed3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed1080 0x555c76ed9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1259866871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed06d0 0x555c76ed9050 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 -- 172.21.15.37:0/1259866871 >> 172.21.15.37:0/1259866871 conn(0x555c76ecada0 msgr2=0x555c76ec9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 -- 172.21.15.37:0/1259866871 shutdown_connections 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.510+0000 7f281f5b6740 1 -- 172.21.15.37:0/1259866871 wait complete. 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 -- start start 2024-03-21T09:22:34.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 Processor -- start 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76edfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ee0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76f3e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141b38f0 con 0x555c76ed9c00 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141b4bd0 con 0x555c76ed1080 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141b3f10 con 0x555c76ed06d0 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f281f5b6740 10 client.0 _ll_register_callbacks cb 0x7f2810005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f2819400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ee0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.511+0000 7f2819e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76edfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ee0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58596/0 (socket says 172.21.15.37:58596) 2024-03-21T09:22:34.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 -- 172.21.15.37:0/1516010902 learned_addr learned my addr 172.21.15.37:0/1516010902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76edfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55042/0 (socket says 172.21.15.37:55042) 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f281a800640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76f3e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 -- 172.21.15.37:0/1516010902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 msgr2=0x555c76edfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76edfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 -- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 msgr2=0x555c76f3e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76f3e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 -- 172.21.15.37:0/1516010902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c76f3e8f0 con 0x555c76ed1080 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f281a800640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76f3e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819e00640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76edfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f2819400640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ee0110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2804005d00 tx=0x7f280400b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9df0c0cbda7a324 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f27f7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.512+0000 7f27f7400640 1 -- 172.21.15.37:0/1516010902 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804004220 con 0x555c76ed1080 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2808005cb0 con 0x555c76ed1080 2024-03-21T09:22:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f27f7400640 1 -- 172.21.15.37:0/1516010902 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28040043c0 con 0x555c76ed1080 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f27f7400640 1 -- 172.21.15.37:0/1516010902 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2804034050 con 0x555c76ed1080 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 10 client.28007 fetch_fsmap learned FSMap version 30 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555c76f45cf0 con 0x555c76ed1080 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f27f7400640 1 -- 172.21.15.37:0/1516010902 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2808005cb0 con 0x555c76ed1080 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f27f7400640 1 -- 172.21.15.37:0/1516010902 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2804028070 con 0x555c76ed1080 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 10 client.28007 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 10 client.28007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 20 client.28007 populate_metadata read hostname 'smithi037' 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76f464a0 0x555c76f488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 4 client.28007 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.513+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 --> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c76e3bb50 con 0x555c76f464a0 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.514+0000 7f2819e00640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76f464a0 0x555c76f488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.514+0000 7f2819e00640 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76f464a0 0x555c76f488c0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0146528b5ad743 server_cookie=bcef53d695181024 in_seq=0 out_seq=0 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.514+0000 7f27f7400640 10 client.28007 ms_handle_connect on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f27f7400640 1 -- 172.21.15.37:0/1516010902 <== mds.0 v2:172.21.15.37:6840/600888190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c76e3bb50 con 0x555c76f464a0 2024-03-21T09:22:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f27f7400640 10 client.28007 handle_command_reply: tid=0 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 client.28007 shutdown 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 msgr2=0x555c76ee0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ee0110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2804005d00 tx=0x7f280400b650 comp rx=0 tx=0).stop 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 shutdown_connections 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c76ed9c00 0x555c76f3e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.37:6840/600888190,v1:172.21.15.37:6841/600888190] conn(0x555c76f464a0 0x555c76f488c0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c76ed1080 0x555c76ee0110 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 --2- 172.21.15.37:0/1516010902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555c76ed06d0 0x555c76edfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f27f7400640 0 client.28007 ms_handle_reset on v2:172.21.15.37:6840/600888190 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 >> 172.21.15.37:0/1516010902 conn(0x555c76ecada0 msgr2=0x555c76ed1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 shutdown_connections 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 1 -- 172.21.15.37:0/1516010902 wait complete. 2024-03-21T09:22:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.515+0000 7f281f5b6740 20 client.28007 trim_cache size 0 max 16384 2024-03-21T09:22:34.518 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-03-21T09:22:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27fc0a3110 msgr2=0x7f27fc0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27fc0a3110 0x7f27fc0a55d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28141017e0 tx=0x7f280400b410 comp rx=0 tx=0).stop 2024-03-21T09:22:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 msgr2=0x7f28141a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 0x7f28141a2e60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2810004af0 tx=0x7f281002cbc0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 shutdown_connections 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2814112500 0x7f28141a2e60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f27fc0a3110 0x7f27fc0a55d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28140764f0 0x7f281419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 --2- 172.21.15.37:0/1781173130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2814075bd0 0x7f281419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 >> 172.21.15.37:0/1781173130 conn(0x7f28140ff310 msgr2=0x7f2814113bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 shutdown_connections 2024-03-21T09:22:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:34.516+0000 7f281c800640 1 -- 172.21.15.37:0/1781173130 wait complete. 2024-03-21T09:22:34.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:34 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3333758981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:22:34.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:34 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3333758981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:22:34.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:34 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3333758981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:22:35.534 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-21T09:22:35.534 INFO:tasks.fwd_scrub:done joining 2024-03-21T09:22:35.534 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-21T09:22:35.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:22:35.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:35 smithi037 ceph-mon[31830]: pgmap v323: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:35.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:35 smithi053 ceph-mon[36169]: pgmap v323: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.771+0000 7f10af200640 1 -- 172.21.15.37:0/3818852465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 msgr2=0x7f10b01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.771+0000 7f10af200640 1 --2- 172.21.15.37:0/3818852465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 0x7f10b01079e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009930 tx=0x7f10a002bc20 comp rx=0 tx=0).stop 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 -- 172.21.15.37:0/3818852465 shutdown_connections 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 --2- 172.21.15.37:0/3818852465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101f00 0x7f10b010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 --2- 172.21.15.37:0/3818852465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101540 0x7f10b01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 --2- 172.21.15.37:0/3818852465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 0x7f10b01079e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 -- 172.21.15.37:0/3818852465 >> 172.21.15.37:0/3818852465 conn(0x7f10b00fd050 msgr2=0x7f10b00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:35.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 -- 172.21.15.37:0/3818852465 shutdown_connections 2024-03-21T09:22:35.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.772+0000 7f10af200640 1 -- 172.21.15.37:0/3818852465 wait complete. 2024-03-21T09:22:35.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.773+0000 7f10af200640 1 Processor -- start 2024-03-21T09:22:35.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.773+0000 7f10af200640 1 -- start start 2024-03-21T09:22:35.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10af200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101540 0x7f10b00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:35.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10af200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 0x7f10b0073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:35.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10af200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 0x7f10b0074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:35.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10af200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0111910 con 0x7f10b0101540 2024-03-21T09:22:35.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10af200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0111790 con 0x7f10b0101f00 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10af200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0111a90 con 0x7f10b0107600 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 0x7f10b0073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10ade00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101540 0x7f10b00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 0x7f10b0073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58612/0 (socket says 172.21.15.37:58612) 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 -- 172.21.15.37:0/1008234567 learned_addr learned my addr 172.21.15.37:0/1008234567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10ade00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101540 0x7f10b00739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57812/0 (socket says 172.21.15.37:57812) 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10ae800640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 0x7f10b0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 -- 172.21.15.37:0/1008234567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 msgr2=0x7f10b0074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 0x7f10b0074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 -- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101540 msgr2=0x7f10b00739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101540 0x7f10b00739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.774+0000 7f10a7e00640 1 -- 172.21.15.37:0/1008234567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b01a6310 con 0x7f10b0101f00 2024-03-21T09:22:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.775+0000 7f10a7e00640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 0x7f10b0073ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f109800e9f0 tx=0x7f109800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.775+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109800cd80 con 0x7f10b0101f00 2024-03-21T09:22:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.775+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1098010910 con 0x7f10b0101f00 2024-03-21T09:22:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.775+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b01a6600 con 0x7f10b0101f00 2024-03-21T09:22:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.775+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b01a6b50 con 0x7f10b0101f00 2024-03-21T09:22:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.775+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098018990 con 0x7f10b0101f00 2024-03-21T09:22:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.776+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c005290 con 0x7f10b0101f00 2024-03-21T09:22:35.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.778+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1098018b30 con 0x7f10b0101f00 2024-03-21T09:22:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.780+0000 7f10a6a00640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f10940a3080 0x7f10940a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.780+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f10980cfe60 con 0x7f10b0101f00 2024-03-21T09:22:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.780+0000 7f10ade00640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f10940a3080 0x7f10940a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.781+0000 7f10ade00640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f10940a3080 0x7f10940a5540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009900 tx=0x7f10a0005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.784+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f109809bea0 con 0x7f10b0101f00 2024-03-21T09:22:35.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:35 smithi071 ceph-mon[36597]: pgmap v323: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:35.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.988+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f108c005850 con 0x7f10b0101f00 2024-03-21T09:22:35.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.990+0000 7f10a6a00640 1 -- 172.21.15.37:0/1008234567 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25290 (secure 0 0 0) 0x7f109809ff40 con 0x7f10b0101f00 2024-03-21T09:22:35.989 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:22:35.989 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":103,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:22:16.105869+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2204787263":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1173483074":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2805488546":"2024-03-22T09:12:10.987337+0000","172.21.15.37:6800/1300592626":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6801/1300592626":"2024-03-22T09:12:10.987337+0000","172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/1005590151":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/1759352225":"2024-03-22T09:12:10.987337+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.992+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f10940a3080 msgr2=0x7f10940a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.992+0000 7f10af200640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f10940a3080 0x7f10940a5540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009900 tx=0x7f10a0005a20 comp rx=0 tx=0).stop 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 msgr2=0x7f10b0073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 0x7f10b0073ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f109800e9f0 tx=0x7f109800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 shutdown_connections 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b0107600 0x7f10b0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f10940a3080 0x7f10940a5540 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10b0101f00 0x7f10b0073ee0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 --2- 172.21.15.37:0/1008234567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b0101540 0x7f10b00739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:35.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 >> 172.21.15.37:0/1008234567 conn(0x7f10b00fd050 msgr2=0x7f10b00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:35.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 shutdown_connections 2024-03-21T09:22:35.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:35.993+0000 7f10af200640 1 -- 172.21.15.37:0/1008234567 wait complete. 2024-03-21T09:22:36.011 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:22:36.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 -- 172.21.15.37:0/2437719566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1014c0 msgr2=0x7fe59c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/2437719566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1014c0 0x7fe59c101940 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe588009930 tx=0x7fe58802bc30 comp rx=0 tx=0).stop 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 -- 172.21.15.37:0/2437719566 shutdown_connections 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/2437719566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c101e80 0x7fe59c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/2437719566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1014c0 0x7fe59c101940 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/2437719566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c107580 0x7fe59c107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 -- 172.21.15.37:0/2437719566 >> 172.21.15.37:0/2437719566 conn(0x7fe59c0fcfb0 msgr2=0x7fe59c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.233+0000 7fe5a2a00640 1 -- 172.21.15.37:0/2437719566 shutdown_connections 2024-03-21T09:22:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.234+0000 7fe5a2a00640 1 -- 172.21.15.37:0/2437719566 wait complete. 2024-03-21T09:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.235+0000 7fe5a2a00640 1 Processor -- start 2024-03-21T09:22:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.235+0000 7fe5a2a00640 1 -- start start 2024-03-21T09:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 0x7fe59c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c101e80 0x7fe59c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c107580 0x7fe59c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1118b0 con 0x7fe59c1014c0 2024-03-21T09:22:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c111730 con 0x7fe59c107580 2024-03-21T09:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c111a30 con 0x7fe59c101e80 2024-03-21T09:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 0x7fe59c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 0x7fe59c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57820/0 (socket says 172.21.15.37:57820) 2024-03-21T09:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a1600640 1 -- 172.21.15.37:0/3291047353 learned_addr learned my addr 172.21.15.37:0/3291047353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a2000640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c107580 0x7fe59c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.236+0000 7fe5a0c00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c101e80 0x7fe59c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2000640 1 -- 172.21.15.37:0/3291047353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c101e80 msgr2=0x7fe59c19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2000640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c101e80 0x7fe59c19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2000640 1 -- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 msgr2=0x7fe59c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2000640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 0x7fe59c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2000640 1 -- 172.21.15.37:0/3291047353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c1983f0 con 0x7fe59c107580 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a1600640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 0x7fe59c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a0c00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c101e80 0x7fe59c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:22:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2000640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c107580 0x7fe59c197b30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe594009fd0 tx=0x7fe59400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:36.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59400ce20 con 0x7fe59c107580 2024-03-21T09:22:36.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe594005020 con 0x7fe59c107580 2024-03-21T09:22:36.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594010720 con 0x7fe59c107580 2024-03-21T09:22:36.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c1986e0 con 0x7fe59c107580 2024-03-21T09:22:36.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.237+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c1a49f0 con 0x7fe59c107580 2024-03-21T09:22:36.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.239+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c198a20 con 0x7fe59c107580 2024-03-21T09:22:36.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.241+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe594005540 con 0x7fe59c107580 2024-03-21T09:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.243+0000 7fe59a000640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe5840a3090 0x7fe5840a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.243+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe5940d0350 con 0x7fe59c107580 2024-03-21T09:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.243+0000 7fe5a1600640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe5840a3090 0x7fe5840a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.244+0000 7fe5a1600640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe5840a3090 0x7fe5840a5550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe590011550 tx=0x7fe590005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:36.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.247+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe59409c9e0 con 0x7fe59c107580 2024-03-21T09:22:36.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.487+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe59c1098d0 con 0x7fe59c107580 2024-03-21T09:22:36.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.487+0000 7fe59a000640 1 -- 172.21.15.37:0/3291047353 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe59409c2e0 con 0x7fe59c107580 2024-03-21T09:22:36.486 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:22:36.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe5840a3090 msgr2=0x7fe5840a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe5840a3090 0x7fe5840a5550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe590011550 tx=0x7fe590005f90 comp rx=0 tx=0).stop 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c107580 msgr2=0x7fe59c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c107580 0x7fe59c197b30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe594009fd0 tx=0x7fe59400ef10 comp rx=0 tx=0).stop 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 shutdown_connections 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fe5840a3090 0x7fe5840a5550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe59c107580 0x7fe59c197b30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c101e80 0x7fe59c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 --2- 172.21.15.37:0/3291047353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe59c1014c0 0x7fe59c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 >> 172.21.15.37:0/3291047353 conn(0x7fe59c0fcfb0 msgr2=0x7fe59c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 shutdown_connections 2024-03-21T09:22:36.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.490+0000 7fe5a2a00640 1 -- 172.21.15.37:0/3291047353 wait complete. 2024-03-21T09:22:36.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:22:36.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:36 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1008234567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:22:36.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:36 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:22:36.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:36 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1008234567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:22:36.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:36 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 -- 172.21.15.37:0/3938148987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 msgr2=0x7f51ac114860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3938148987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac114860 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009ca0 tx=0x7f51a402c180 comp rx=0 tx=0).stop 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 -- 172.21.15.37:0/3938148987 shutdown_connections 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3938148987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac114860 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3938148987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac074360 0x7f51ac111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3938148987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac073a40 0x7f51ac073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 -- 172.21.15.37:0/3938148987 >> 172.21.15.37:0/3938148987 conn(0x7f51ac0fcff0 msgr2=0x7f51ac0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 -- 172.21.15.37:0/3938148987 shutdown_connections 2024-03-21T09:22:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.732+0000 7f51b2c00640 1 -- 172.21.15.37:0/3938148987 wait complete. 2024-03-21T09:22:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.733+0000 7f51b2c00640 1 Processor -- start 2024-03-21T09:22:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 -- start start 2024-03-21T09:22:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac073a40 0x7f51ac19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 0x7f51ac19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac1a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac117e70 con 0x7f51ac073a40 2024-03-21T09:22:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac117cf0 con 0x7f51ac074360 2024-03-21T09:22:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b2c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac117ff0 con 0x7f51ac112470 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.734+0000 7f51b0e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 0x7f51ac19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac073a40 0x7f51ac19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac1a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 0x7f51ac19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58658/0 (socket says 172.21.15.37:58658) 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 -- 172.21.15.37:0/3474898053 learned_addr learned my addr 172.21.15.37:0/3474898053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 -- 172.21.15.37:0/3474898053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 msgr2=0x7f51ac1a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac1a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 -- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac073a40 msgr2=0x7f51ac19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac073a40 0x7f51ac19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 -- 172.21.15.37:0/3474898053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac1a3350 con 0x7f51ac074360 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b2200640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.735+0000 7f51b0e00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 0x7f51ac19e8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f519800e9f0 tx=0x7f519800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.736+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800cd80 con 0x7f51ac074360 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.736+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5198010910 con 0x7f51ac074360 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.736+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198018990 con 0x7f51ac074360 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.736+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac1a3640 con 0x7f51ac074360 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.736+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac1aaed0 con 0x7f51ac074360 2024-03-21T09:22:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.737+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac04f400 con 0x7f51ac074360 2024-03-21T09:22:36.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.740+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5198010ac0 con 0x7f51ac074360 2024-03-21T09:22:36.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.742+0000 7f51aa000640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51940a3090 0x7f51940a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:36.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.742+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f51980cfee0 con 0x7f51ac074360 2024-03-21T09:22:36.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.742+0000 7f51b1800640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51940a3090 0x7f51940a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:36.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.743+0000 7f51b1800640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51940a3090 0x7f51940a5550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009bd0 tx=0x7f51a0005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:36.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.750+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f519809bf20 con 0x7f51ac074360 2024-03-21T09:22:36.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:36 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1008234567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:22:36.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:36 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:22:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.960+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51ac02d070 con 0x7f51ac074360 2024-03-21T09:22:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.961+0000 7f51aa000640 1 -- 172.21.15.37:0/3474898053 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7f519809ffc0 con 0x7f51ac074360 2024-03-21T09:22:36.959 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:22:36.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51940a3090 msgr2=0x7f51940a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51940a3090 0x7f51940a5550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009bd0 tx=0x7f51a0005f90 comp rx=0 tx=0).stop 2024-03-21T09:22:36.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 msgr2=0x7f51ac19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 0x7f51ac19e8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f519800e9f0 tx=0x7f519800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 shutdown_connections 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51ac112470 0x7f51ac1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f51940a3090 0x7f51940a5550 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac074360 0x7f51ac19e8f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 --2- 172.21.15.37:0/3474898053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac073a40 0x7f51ac19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 >> 172.21.15.37:0/3474898053 conn(0x7f51ac0fcff0 msgr2=0x7f51ac0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 shutdown_connections 2024-03-21T09:22:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:36.963+0000 7f51b2c00640 1 -- 172.21.15.37:0/3474898053 wait complete. 2024-03-21T09:22:36.982 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 -- 172.21.15.37:0/55304372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28104220 msgr2=0x7fee28110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 --2- 172.21.15.37:0/55304372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28104220 0x7fee28110730 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee20009c10 tx=0x7fee2002bef0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 -- 172.21.15.37:0/55304372 shutdown_connections 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 --2- 172.21.15.37:0/55304372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28104220 0x7fee28110730 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 --2- 172.21.15.37:0/55304372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28103860 0x7fee28103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 --2- 172.21.15.37:0/55304372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee28109870 0x7fee28109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.189+0000 7fee2ea00640 1 -- 172.21.15.37:0/55304372 >> 172.21.15.37:0/55304372 conn(0x7fee280ff310 msgr2=0x7fee28101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.190+0000 7fee2ea00640 1 -- 172.21.15.37:0/55304372 shutdown_connections 2024-03-21T09:22:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.190+0000 7fee2ea00640 1 -- 172.21.15.37:0/55304372 wait complete. 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.190+0000 7fee2ea00640 1 Processor -- start 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 -- start start 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee28103860 0x7fee2819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28104220 0x7fee2819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28109870 0x7fee2819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28113c40 con 0x7fee28103860 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28113ac0 con 0x7fee28104220 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28113dc0 con 0x7fee28109870 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee27400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28104220 0x7fee2819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee28103860 0x7fee2819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee27400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28104220 0x7fee2819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58688/0 (socket says 172.21.15.37:58688) 2024-03-21T09:22:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee2ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28109870 0x7fee2819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.191+0000 7fee27400640 1 -- 172.21.15.37:0/2294046517 learned_addr learned my addr 172.21.15.37:0/2294046517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ca00640 1 -- 172.21.15.37:0/2294046517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28104220 msgr2=0x7fee2819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ca00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28104220 0x7fee2819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ca00640 1 -- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee28103860 msgr2=0x7fee2819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ca00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee28103860 0x7fee2819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ca00640 1 -- 172.21.15.37:0/2294046517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2819f1b0 con 0x7fee28109870 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ca00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28109870 0x7fee2819ea80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee2002c430 tx=0x7fee200336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20043020 con 0x7fee28109870 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee2002ca80 con 0x7fee28109870 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2003e6b0 con 0x7fee28109870 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.192+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2819f440 con 0x7fee28109870 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.193+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281a6d30 con 0x7fee28109870 2024-03-21T09:22:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.194+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee28104fa0 con 0x7fee28109870 2024-03-21T09:22:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.197+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fee20046020 con 0x7fee28109870 2024-03-21T09:22:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.198+0000 7fee24c00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fee0c0a3190 0x7fee0c0a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.198+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fee200f1f70 con 0x7fee28109870 2024-03-21T09:22:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.198+0000 7fee27e00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fee0c0a3190 0x7fee0c0a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.199+0000 7fee27e00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fee0c0a3190 0x7fee0c0a5650 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fee18009b50 tx=0x7fee18005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.202+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee2004d020 con 0x7fee28109870 2024-03-21T09:22:37.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.405+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee2810bdf0 con 0x7fee28109870 2024-03-21T09:22:37.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.406+0000 7fee24c00640 1 -- 172.21.15.37:0/2294046517 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee200be710 con 0x7fee28109870 2024-03-21T09:22:37.404 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fee0c0a3190 msgr2=0x7fee0c0a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fee0c0a3190 0x7fee0c0a5650 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fee18009b50 tx=0x7fee18005f90 comp rx=0 tx=0).stop 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28109870 msgr2=0x7fee2819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28109870 0x7fee2819ea80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee2002c430 tx=0x7fee200336b0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 shutdown_connections 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee28109870 0x7fee2819ea80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fee0c0a3190 0x7fee0c0a5650 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee28104220 0x7fee2819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 --2- 172.21.15.37:0/2294046517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee28103860 0x7fee2819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.408+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 >> 172.21.15.37:0/2294046517 conn(0x7fee280ff310 msgr2=0x7fee280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:37.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.409+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 shutdown_connections 2024-03-21T09:22:37.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.409+0000 7fee2ea00640 1 -- 172.21.15.37:0/2294046517 wait complete. 2024-03-21T09:22:37.426 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 -- 172.21.15.37:0/2628903190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 msgr2=0x7f63b4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 --2- 172.21.15.37:0/2628903190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b4107a20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009930 tx=0x7f63a402bc20 comp rx=0 tx=0).stop 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 -- 172.21.15.37:0/2628903190 shutdown_connections 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 --2- 172.21.15.37:0/2628903190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101ff0 0x7f63b410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 --2- 172.21.15.37:0/2628903190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101630 0x7f63b4101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 --2- 172.21.15.37:0/2628903190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b4107a20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 -- 172.21.15.37:0/2628903190 >> 172.21.15.37:0/2628903190 conn(0x7f63b40fd140 msgr2=0x7f63b40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.637+0000 7f63baa00640 1 -- 172.21.15.37:0/2628903190 shutdown_connections 2024-03-21T09:22:37.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.638+0000 7f63baa00640 1 -- 172.21.15.37:0/2628903190 wait complete. 2024-03-21T09:22:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.638+0000 7f63baa00640 1 Processor -- start 2024-03-21T09:22:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 -- start start 2024-03-21T09:22:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 0x7f63b419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101ff0 0x7f63b419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4111aa0 con 0x7f63b4101ff0 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4111920 con 0x7f63b4101630 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63baa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4111c20 con 0x7f63b4107640 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 0x7f63b419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63b3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101ff0 0x7f63b419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 0x7f63b419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:58704/0 (socket says 172.21.15.37:58704) 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.639+0000 7f63b3e00640 1 -- 172.21.15.37:0/737547752 learned_addr learned my addr 172.21.15.37:0/737547752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b8a00640 1 -- 172.21.15.37:0/737547752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 msgr2=0x7f63b419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b8a00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 0x7f63b419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b8a00640 1 -- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101ff0 msgr2=0x7f63b419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b8a00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101ff0 0x7f63b419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b8a00640 1 -- 172.21.15.37:0/737547752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b419f170 con 0x7f63b4107640 2024-03-21T09:22:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b3e00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 0x7f63b419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b3400640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101ff0 0x7f63b419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b8a00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b419ea40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009fd0 tx=0x7f63ac00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00ce20 con 0x7f63b4107640 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63ac005020 con 0x7f63b4107640 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac010720 con 0x7f63b4107640 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b419f460 con 0x7f63b4107640 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.640+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b41a6cf0 con 0x7f63b4107640 2024-03-21T09:22:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.641+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4068970 con 0x7f63b4107640 2024-03-21T09:22:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.643+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f63ac005540 con 0x7f63b4107640 2024-03-21T09:22:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.645+0000 7f63b0c00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63980a3110 0x7f63980a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.645+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f63ac0d0160 con 0x7f63b4107640 2024-03-21T09:22:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.645+0000 7f63b3e00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63980a3110 0x7f63980a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.646+0000 7f63b3e00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63980a3110 0x7f63980a55d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63a40096c0 tx=0x7f63a4005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:37.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.650+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f63ac09c7f0 con 0x7f63b4107640 2024-03-21T09:22:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3291047353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/3474898053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:37 smithi037 ceph-mon[31830]: pgmap v324: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:37.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:37 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2294046517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3291047353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/3474898053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:37 smithi053 ceph-mon[36169]: pgmap v324: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:37.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:37 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2294046517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3291047353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/3474898053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:37 smithi071 ceph-mon[36597]: pgmap v324: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:37.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:37 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2294046517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:22:37.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.870+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f63b41a7020 con 0x7f63b4107640 2024-03-21T09:22:37.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.871+0000 7f63b0c00640 1 -- 172.21.15.37:0/737547752 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) v1 ==== 103+0+11 (secure 0 0 0) 0x7f63ac09c0f0 con 0x7f63b4107640 2024-03-21T09:22:37.869 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.874+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63980a3110 msgr2=0x7f63980a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.874+0000 7f63baa00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63980a3110 0x7f63980a55d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63a40096c0 tx=0x7f63a4005a20 comp rx=0 tx=0).stop 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.874+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 msgr2=0x7f63b419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.874+0000 7f63baa00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b419ea40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009fd0 tx=0x7f63ac00ef10 comp rx=0 tx=0).stop 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 shutdown_connections 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63b4107640 0x7f63b419ea40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63b4101ff0 0x7f63b419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f63980a3110 0x7f63980a55d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 --2- 172.21.15.37:0/737547752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4101630 0x7f63b419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:37.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 >> 172.21.15.37:0/737547752 conn(0x7f63b40fd140 msgr2=0x7f63b40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:37.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 shutdown_connections 2024-03-21T09:22:37.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:37.875+0000 7f63baa00640 1 -- 172.21.15.37:0/737547752 wait complete. 2024-03-21T09:22:37.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-21T09:22:38.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 -- 172.21.15.37:0/3092898199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c102800 msgr2=0x7fc50c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:38.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 --2- 172.21.15.37:0/3092898199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c102800 0x7fc50c102be0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc500009900 tx=0x7fc50002bcd0 comp rx=0 tx=0).stop 2024-03-21T09:22:38.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 -- 172.21.15.37:0/3092898199 shutdown_connections 2024-03-21T09:22:38.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 --2- 172.21.15.37:0/3092898199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c10bc20 0x7fc50c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:38.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 --2- 172.21.15.37:0/3092898199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c103120 0x7fc50c10b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:38.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 --2- 172.21.15.37:0/3092898199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c102800 0x7fc50c102be0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 -- 172.21.15.37:0/3092898199 >> 172.21.15.37:0/3092898199 conn(0x7fc50c0fc020 msgr2=0x7fc50c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 -- 172.21.15.37:0/3092898199 shutdown_connections 2024-03-21T09:22:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.113+0000 7fc514600640 1 -- 172.21.15.37:0/3092898199 wait complete. 2024-03-21T09:22:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.114+0000 7fc514600640 1 Processor -- start 2024-03-21T09:22:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 -- start start 2024-03-21T09:22:38.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 0x7fc50c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:38.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c103120 0x7fc50c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:38.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c10bc20 0x7fc50c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c111a70 con 0x7fc50c10bc20 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c1118f0 con 0x7fc50c103120 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc514600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c111bf0 con 0x7fc50c102800 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc511c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 0x7fc50c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc511c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 0x7fc50c19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:55128/0 (socket says 172.21.15.37:55128) 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc511c00640 1 -- 172.21.15.37:0/1370526346 learned_addr learned my addr 172.21.15.37:0/1370526346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc511200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c103120 0x7fc50c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.115+0000 7fc512600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c10bc20 0x7fc50c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511200640 1 -- 172.21.15.37:0/1370526346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 msgr2=0x7fc50c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511200640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 0x7fc50c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511200640 1 -- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c10bc20 msgr2=0x7fc50c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511200640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c10bc20 0x7fc50c19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511200640 1 -- 172.21.15.37:0/1370526346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c19f160 con 0x7fc50c103120 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511c00640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 0x7fc50c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc512600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c10bc20 0x7fc50c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc511200640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c103120 0x7fc50c19a6b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800e7a0 tx=0x7fc4f800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800ccc0 con 0x7fc50c103120 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8009530 con 0x7fc50c103120 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.116+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c19f450 con 0x7fc50c103120 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.117+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8018880 con 0x7fc50c103120 2024-03-21T09:22:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.117+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c1a6ce0 con 0x7fc50c103120 2024-03-21T09:22:38.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.118+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c06a660 con 0x7fc50c103120 2024-03-21T09:22:38.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.120+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc4f8018a20 con 0x7fc50c103120 2024-03-21T09:22:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.122+0000 7fc50a000640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc4f40a3110 0x7fc4f40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:22:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.122+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc4f80cfd50 con 0x7fc50c103120 2024-03-21T09:22:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.122+0000 7fc511c00640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc4f40a3110 0x7fc4f40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:22:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.123+0000 7fc511c00640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc4f40a3110 0x7fc4f40a55d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5000096c0 tx=0x7fc500005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:22:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.125+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4f809bd90 con 0x7fc50c103120 2024-03-21T09:22:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.425+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fc50c19b4c0 con 0x7fc50c103120 2024-03-21T09:22:38.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.436+0000 7fc50a000640 1 -- 172.21.15.37:0/1370526346 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) v1 ==== 81+0+742375 (secure 0 0 0) 0x7fc4f809fe30 con 0x7fc50c103120 2024-03-21T09:22:38.445 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:22:38.445 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:38.445 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 28, 2024-03-21T09:22:38.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 25553, 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1212038665 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-03-21T09:22:38.446 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1212038665 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/1212038665", 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-03-21T09:12:10.987922+0000", 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 25550, 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-03-21T09:22:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T09:22:38.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.449 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.450 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.455 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-03-21T09:22:38.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-03-21T09:22:38.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:22:38.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.468 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.470 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-03-21T09:22:38.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-21T09:22:38.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T09:22:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T09:22:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-21T09:22:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-03-21T09:22:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.509 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.510 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-03-21T09:22:38.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-03-21T09:22:38.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-03-21T09:22:38.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.517 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-03-21T09:22:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T09:22:38.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.540 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.542 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.544 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T09:22:38.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.558 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T09:22:38.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T09:22:38.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T09:22:38.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:38.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.576 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T09:22:38.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T09:22:38.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.591 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.592 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-03-21T09:22:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-03-21T09:22:38.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.599 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.600 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.615 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T09:22:38.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-03-21T09:22:38.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.629 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T09:22:38.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-03-21T09:22:38.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-03-21T09:22:38.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-21T09:22:38.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-03-21T09:22:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.647 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.649 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.652 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-03-21T09:22:38.656 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.659 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.662 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.665 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T09:22:38.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T09:22:38.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.678 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.683 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.685 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.686 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.692 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.696 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.701 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.706 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.708 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.709 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T09:22:38.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.718 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-03-21T09:22:38.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:38.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.727 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.739 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.740 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-03-21T09:22:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:38.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.756 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.761 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.763 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.764 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.769 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.770 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-03-21T09:22:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.786 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-03-21T09:22:38.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.791 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-03-21T09:22:38.793 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.794 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.799 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.809 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.824 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-03-21T09:22:38.831 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.832 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.838 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.845 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-03-21T09:22:38.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.849 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.853 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.854 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-03-21T09:22:38.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 25579, 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.859 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.861 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-03-21T09:22:38.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.867 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.868 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.869 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T09:22:38.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T09:22:38.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T09:22:38.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-03-21T09:22:38.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.886 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.887 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-03-21T09:22:38.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T09:22:38.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T09:22:38.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T09:22:38.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-03-21T09:22:38.896 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-03-21T09:22:38.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-03-21T09:22:38.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-03-21T09:22:38.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.913 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-03-21T09:22:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.915 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-03-21T09:22:38.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-03-21T09:22:38.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.925 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-03-21T09:22:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T09:22:38.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T09:22:38.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-03-21T09:22:38.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-03-21T09:22:38.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-03-21T09:22:38.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-03-21T09:22:38.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-03-21T09:22:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-03-21T09:22:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-03-21T09:22:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-03-21T09:22:38.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-03-21T09:22:38.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-03-21T09:22:38.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-03-21T09:22:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-03-21T09:22:38.950 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.951 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.955 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-03-21T09:22:38.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-03-21T09:22:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.959 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.960 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.961 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.962 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.963 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.964 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T09:22:38.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T09:22:38.966 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T09:22:38.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T09:22:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.977 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.987 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T09:22:38.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T09:22:38.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T09:22:38.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:38.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:38.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-03-21T09:22:39.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.006 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-03-21T09:22:39.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-03-21T09:22:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.019 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.021 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.024 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-03-21T09:22:39.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-03-21T09:22:39.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.032 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.037 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.038 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-03-21T09:22:39.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.045 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-03-21T09:22:39.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-21T09:22:39.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-03-21T09:22:39.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-03-21T09:22:39.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-03-21T09:22:39.054 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.056 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.063 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-03-21T09:22:39.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-03-21T09:22:39.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.071 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.073 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T09:22:39.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.075 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-03-21T09:22:39.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.081 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T09:22:39.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.088 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.093 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.095 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.096 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.103 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.107 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.108 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.109 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.114 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.122 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-03-21T09:22:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-03-21T09:22:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.140 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.153 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.163 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.170 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.172 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.173 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.176 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-03-21T09:22:39.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.179 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-03-21T09:22:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-03-21T09:22:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-03-21T09:22:39.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.200 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.203 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.208 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T09:22:39.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-03-21T09:22:39.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.233 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.238 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.240 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.246 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.249 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T09:22:39.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T09:22:39.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T09:22:39.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.260 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.261 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-03-21T09:22:39.266 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.269 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.271 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-03-21T09:22:39.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-03-21T09:22:39.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-03-21T09:22:39.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.279 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T09:22:39.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T09:22:39.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.287 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.291 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:39.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-03-21T09:22:39.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-03-21T09:22:39.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.300 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.301 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.302 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.303 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-03-21T09:22:39.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-03-21T09:22:39.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T09:22:39.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:39.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-03-21T09:22:39.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-21T09:22:39.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.329 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.330 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-03-21T09:22:39.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-03-21T09:22:39.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-03-21T09:22:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-03-21T09:22:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-03-21T09:22:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T09:22:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T09:22:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T09:22:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-03-21T09:22:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-03-21T09:22:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-03-21T09:22:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-03-21T09:22:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T09:22:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T09:22:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T09:22:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T09:22:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T09:22:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T09:22:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T09:22:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-03-21T09:22:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-03-21T09:22:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-03-21T09:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.444 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.446 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.448 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-21T09:22:39.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-03-21T09:22:39.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-03-21T09:22:39.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-03-21T09:22:39.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.460 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-03-21T09:22:39.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:39.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.469 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-03-21T09:22:39.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.484 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.486 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.488 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-03-21T09:22:39.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-03-21T09:22:39.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T09:22:39.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.499 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-03-21T09:22:39.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-03-21T09:22:39.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.501 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.505 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.514 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T09:22:39.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.518 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.527 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-03-21T09:22:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.535 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.538 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-03-21T09:22:39.541 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.544 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.547 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-21T09:22:39.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.553 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.568 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.569 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.578 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.586 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.588 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.589 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-03-21T09:22:39.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.595 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.598 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.613 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.621 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.626 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.628 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.631 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.633 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.641 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-03-21T09:22:39.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-21T09:22:39.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.649 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.650 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.651 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.659 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.664 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-03-21T09:22:39.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-03-21T09:22:39.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-21T09:22:39.676 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-21T09:22:39.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-21T09:22:39.681 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-21T09:22:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-21T09:22:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-03-21T09:22:39.684 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 95, 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3882117194 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-21T09:22:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3910903309 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4098596523 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2642491376 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3457223710 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-21T09:22:39.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3776608566 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-21T09:22:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:39.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/737547752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1370526346' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:38 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/737547752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1370526346' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:38 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc4f40a3110 msgr2=0x7fc4f40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc4f40a3110 0x7fc4f40a55d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5000096c0 tx=0x7fc500005a20 comp rx=0 tx=0).stop 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c103120 msgr2=0x7fc50c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c103120 0x7fc50c19a6b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800e7a0 tx=0x7fc4f800ec60 comp rx=0 tx=0).stop 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 shutdown_connections 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c10bc20 0x7fc50c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:39.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fc4f40a3110 0x7fc4f40a55d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:39.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc50c103120 0x7fc50c19a6b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:39.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 --2- 172.21.15.37:0/1370526346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50c102800 0x7fc50c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:22:39.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.443+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 >> 172.21.15.37:0/1370526346 conn(0x7fc50c0fc020 msgr2=0x7fc50c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:22:39.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.444+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 shutdown_connections 2024-03-21T09:22:39.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:22:38.444+0000 7fc514600640 1 -- 172.21.15.37:0/1370526346 wait complete. 2024-03-21T09:22:39.701 DEBUG:tasks.check_counter:Getting stats from a 2024-03-21T09:22:39.702 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/737547752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1370526346' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T09:22:39.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:39.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:38 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:22:40.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi071 to 174.1M 2024-03-21T09:22:40.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: Unable to set osd_memory_target on smithi071 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-21T09:22:40.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi053 to 175.2M 2024-03-21T09:22:40.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: Unable to set osd_memory_target on smithi053 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-21T09:22:40.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:40.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:40.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:40 smithi037 ceph-mon[31830]: pgmap v325: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi071 to 174.1M 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: Unable to set osd_memory_target on smithi071 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi053 to 175.2M 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: Unable to set osd_memory_target on smithi053 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:40.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:40 smithi071 ceph-mon[36597]: pgmap v325: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:40.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi071 to 174.1M 2024-03-21T09:22:40.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: Unable to set osd_memory_target on smithi071 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-21T09:22:40.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi053 to 175.2M 2024-03-21T09:22:40.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: Unable to set osd_memory_target on smithi053 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-21T09:22:40.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:40.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:40.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:40 smithi053 ceph-mon[36169]: pgmap v325: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: Adjusting osd_memory_target on smithi037 to 177.8M 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: Unable to set osd_memory_target on smithi037 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T09:22:41.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:22:41.273 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:22:41.273 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:41 smithi037 ceph-mon[31830]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.345 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.346 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: Adjusting osd_memory_target on smithi037 to 177.8M 2024-03-21T09:22:41.346 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: Unable to set osd_memory_target on smithi037 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T09:22:41.346 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:22:41.346 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:22:41.346 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:41 smithi053 ceph-mon[36169]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: Adjusting osd_memory_target on smithi037 to 177.8M 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: Unable to set osd_memory_target on smithi037 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:22:41.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:22:41.357 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:41 smithi071 ceph-mon[36597]: from='mgr.25553 ' entity='mgr.x' 2024-03-21T09:22:42.112 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:42 smithi037 ceph-mon[31830]: pgmap v326: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:42.356 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:42 smithi071 ceph-mon[36597]: pgmap v326: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:42.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:42 smithi053 ceph-mon[36169]: pgmap v326: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:42.694 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 1326, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 1114, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 556028, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 2452803, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 173, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 4, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.371761711, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.093279025, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.181173688, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.057834861, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1116, 2024-03-21T09:22:42.695 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.406339783, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000364103 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1339, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000481697, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000359 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 51504, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 2720 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 1747, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 1456, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2506049, 2024-03-21T09:22:42.696 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 829543, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 12, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 6, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.375800847, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.140885147, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.222937386, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.051746954, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1458, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.840097125, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000576198 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:42.697 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1878, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000654757, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000348 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1943424, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 485520 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2061, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 780, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 807600, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 707332, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 35, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 8, 2024-03-21T09:22:42.698 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.566188045, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.243028883, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.170955185, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.145463530, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 771, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.472967468, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000613446 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1284, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000485127, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000377 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.699 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 404864, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 276592 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 579, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.263512699, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000455116 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.700 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 30, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.002215662, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000073855 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 177, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 177, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 177, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.249698608, 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001410726 2024-03-21T09:22:42.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 0, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 2, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 1, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 49, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 0, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 0, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 31, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 8, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 12, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 11, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 20, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 7, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 3, 2024-03-21T09:22:42.702 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 3, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 3, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 254, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 245, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 1, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 2, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 1, 2024-03-21T09:22:42.703 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 2, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 224, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 4, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 234, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 6, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 224, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 144, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 20, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 124, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 80, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 2, 2024-03-21T09:22:42.704 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 19, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 14, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 28, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 4, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 3, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 24 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 8, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 4, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 2, 2024-03-21T09:22:42.705 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 2, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 17, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 1, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 3, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 2, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 20, 2024-03-21T09:22:42.706 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 1, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 157, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 95, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 95, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 62, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 3, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 13, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 12, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 12, 2024-03-21T09:22:42.707 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 1, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 1, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 1, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 4458921, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 4538395, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 0, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 127, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.236198070, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001859827 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 0 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 27, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 55, 2024-03-21T09:22:42.708 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 28, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 22, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 27, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 5, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 31, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 41, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 10, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 3, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 23, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 20, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 54864, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 231804 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 231, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 5, 2024-03-21T09:22:42.709 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 177, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 39, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 14, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 6, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.010535897, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001755982 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 118, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.064084136, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000543085 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.710 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000852299, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000426149 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000280149, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000280149 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.712 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 10, 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.002605210, 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000260521 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-03-21T09:22:42.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 9, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014610995, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001623443 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 10, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.068711953, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.006871195 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.714 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.001501020, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001501020 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.007534061, 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.003767030 2024-03-21T09:22:42.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.021453609, 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.010726804 2024-03-21T09:22:42.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 4, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.024503354, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.006125838 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.005167481, 2024-03-21T09:22:42.717 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002583740 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.718 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 9, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.026771383, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002974598 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.001087061, 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001087061 2024-03-21T09:22:42.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 2, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 7, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 5, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 2, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 27, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 13, 2024-03-21T09:22:42.720 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 617, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:22:42.721 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:22:42.722 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 335192, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 134, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 880, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 10, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-21T09:22:42.723 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 9736, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 203, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 126636, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 895, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-03-21T09:22:42.724 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 1097, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 427078, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 31, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 1097, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1097, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 16.422745350, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.014970597 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1097, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 1249 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 1097, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 269, 2024-03-21T09:22:42.725 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 828, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 30, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 29, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 22, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 129, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 60, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 523, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-03-21T09:22:42.726 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 247, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 54, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-03-21T09:22:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 103, 2024-03-21T09:22:42.728 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 18, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 12, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 12, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 0, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 137, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 6, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 12 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 1, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-03-21T09:22:42.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 0 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 10, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 1, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 11, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 2, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:22:42.730 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 5130, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 3351341, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 5130, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 5130, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 3351341, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 1097, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 427252, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 1097, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 427252, 2024-03-21T09:22:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-21T09:22:42.733 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 1097, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 1097, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 1097, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1097, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-03-21T09:22:42.734 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 2, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 210, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 2, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 2, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 210, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x56084d398460": { 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 157, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 344091, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 157, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 127, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 344091, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:42.736 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:42.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:42.737 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:43.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:43 smithi037 ceph-mon[31830]: pgmap v327: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:43.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:43 smithi053 ceph-mon[36169]: pgmap v327: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds.exported=1 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=8 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=4 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=10 2024-03-21T09:22:43.826 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=4 2024-03-21T09:22:43.826 DEBUG:tasks.check_counter:Getting stats from d 2024-03-21T09:22:43.826 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-21T09:22:43.855 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:43 smithi071 ceph-mon[36597]: pgmap v327: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:22:45.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:45 smithi037 ceph-mon[31830]: pgmap v328: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:45.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:45 smithi053 ceph-mon[36169]: pgmap v328: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:45.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:45 smithi071 ceph-mon[36597]: pgmap v328: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:46.967 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:22:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-03-21T09:22:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-03-21T09:22:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:22:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-03-21T09:22:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-03-21T09:22:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-03-21T09:22:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.003847215, 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000838665, 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 250, 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 235, 2024-03-21T09:22:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1866336, 2024-03-21T09:22:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 140832, 2024-03-21T09:22:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-03-21T09:22:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-03-21T09:22:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.105609175, 2024-03-21T09:22:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.037700201, 2024-03-21T09:22:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.076329241, 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.010729422, 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 235, 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.022969320, 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000097741 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1866336, 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 140832 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 178, 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-03-21T09:22:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 186640, 2024-03-21T09:22:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-03-21T09:22:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-03-21T09:22:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.037362366, 2024-03-21T09:22:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.025924124, 2024-03-21T09:22:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.004573074, 2024-03-21T09:22:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 175, 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.027966674, 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000159809 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:22:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 186640 2024-03-21T09:22:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:22:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:22:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:22:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:22:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:22:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:22:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:22:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:22:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:22:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-21T09:22:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:22:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-21T09:22:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:22:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:22:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:22:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:22:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:22:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-21T09:22:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-21T09:22:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-21T09:22:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:22:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148754, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-03-21T09:22:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-21T09:22:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-21T09:22:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:22:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-21T09:22:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:22:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-21T09:22:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-03-21T09:22:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-03-21T09:22:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:22:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:22:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-21T09:22:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-21T09:22:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 248, 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1828536, 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 248, 2024-03-21T09:22:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-21T09:22:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-21T09:22:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 248, 2024-03-21T09:22:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1828536, 2024-03-21T09:22:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:46.991 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:47.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:47 smithi037 ceph-mon[31830]: pgmap v329: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:47.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:47 smithi053 ceph-mon[36169]: pgmap v329: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:47.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:47 smithi071 ceph-mon[36597]: pgmap v329: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:47.891 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-21T09:22:47.891 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-21T09:22:47.891 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-21T09:22:47.891 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-21T09:22:47.891 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-21T09:22:47.892 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-21T09:22:47.892 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-21T09:22:47.892 DEBUG:tasks.check_counter:Getting stats from g 2024-03-21T09:22:47.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-21T09:22:48.358 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:22:49.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:49 smithi037 ceph-mon[31830]: pgmap v330: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:49.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:49 smithi053 ceph-mon[36169]: pgmap v330: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:49.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:49 smithi071 ceph-mon[36597]: pgmap v330: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-03-21T09:22:50.798 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.003985386, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000833128, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:22:50.799 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 246, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 231, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1865248, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 138336, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.100177165, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.036880332, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.069978093, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.010966394, 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:50.800 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 231, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.028199668, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000122076 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1865248, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 138336 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-03-21T09:22:50.801 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 175, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 186112, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.036831512, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.026571267, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.003781323, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 172, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.024155795, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000140440 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.802 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 186112 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:22:50.803 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-21T09:22:50.804 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:22:50.805 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-21T09:22:50.806 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:50.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 244, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1828016, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 244, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 244, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1828016, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:50.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:50.809 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:51.626 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:51 smithi037 ceph-mon[31830]: pgmap v331: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:51.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:51 smithi053 ceph-mon[36169]: pgmap v331: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-21T09:22:51.700 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-21T09:22:51.700 DEBUG:tasks.check_counter:Getting stats from j 2024-03-21T09:22:51.701 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-21T09:22:51.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:51 smithi071 ceph-mon[36597]: pgmap v331: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:52.179 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:22:53.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:53 smithi037 ceph-mon[31830]: pgmap v332: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:53.698 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:53 smithi053 ceph-mon[36169]: pgmap v332: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:53.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:53 smithi071 ceph-mon[36597]: pgmap v332: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:54.620 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-21T09:22:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:22:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-03-21T09:22:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-03-21T09:22:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.004006197, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000977072, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 183, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3392, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 187616, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-03-21T09:22:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.041486757, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.028770279, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.005957982, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 181, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.027860124, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000153923 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-21T09:22:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 187616 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 249, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 236, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1864080, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 142256, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.108214476, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.035811638, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.086640090, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.011043261, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 235, 2024-03-21T09:22:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.021741024, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000092514 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1864080, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 142256 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-21T09:22:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:22:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:22:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 150776, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 32, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-21T09:22:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-21T09:22:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 251, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1828926, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 251, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 251, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1828926, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-21T09:22:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-21T09:22:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-21T09:22:54.631 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-21T09:22:55.587 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-21T09:22:55.587 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-21T09:22:55.587 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-21T09:22:55.587 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-21T09:22:55.587 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-21T09:22:55.588 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-21T09:22:55.588 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-21T09:22:55.588 DEBUG:tasks.check_counter:Getting stats from b 2024-03-21T09:22:55.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-21T09:22:55.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:55 smithi037 ceph-mon[31830]: pgmap v333: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:55.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:55 smithi053 ceph-mon[36169]: pgmap v333: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:55.853 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:22:55.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:55 smithi071 ceph-mon[36597]: pgmap v333: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:57.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:57 smithi037 ceph-mon[31830]: pgmap v334: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:57.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:57 smithi053 ceph-mon[36169]: pgmap v334: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:57.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:57 smithi071 ceph-mon[36597]: pgmap v334: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 521, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 652, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 164490, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 369361, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 11, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.045004997, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.015558870, 2024-03-21T09:22:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.022369984, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.003404829, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 650, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.040958375, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000063012 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1088, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000204063, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000187 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-03-21T09:22:57.862 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 5152 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 884, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 587, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2098438, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 660217, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 50, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.078519334, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.018525189, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.075986746, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.004509251, 2024-03-21T09:22:57.863 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 586, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.048107825, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000082095 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 828, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000116877, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000141 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1926896, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 146784 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:22:57.864 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 32, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 219, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 10787, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 482088, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 10, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.013674077, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.006716706, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.005160245, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000347694, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 217, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.021924083, 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000101032 2024-03-21T09:22:57.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 37, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000011459, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000309 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 5568, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 468768 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.866 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 48, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002409317, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000050194 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000006772, 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000003386 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 4, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 3, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001036076, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000345358 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 1, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 11, 2024-03-21T09:22:57.868 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 19, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 2, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 17, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 13, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 9, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 4, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 18, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 11, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 1, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 4, 2024-03-21T09:22:57.869 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-03-21T09:22:57.870 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:22:57.871 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 15, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 9, 2024-03-21T09:22:57.872 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 11, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-03-21T09:22:57.873 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 8, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 8, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-03-21T09:22:57.874 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 2, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 4196653, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.009955227, 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001422175 2024-03-21T09:22:57.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 24, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 30, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 6, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 23, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 28, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 5, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 19, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 28, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 9, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 2, 2024-03-21T09:22:57.876 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 2, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 50468, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223612 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 5, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 1, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 4, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 24, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 2, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000310805, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000310805 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000725271, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000362635 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.880 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.882 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-03-21T09:22:57.882 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.882 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.882 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.883 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.884 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-03-21T09:22:57.885 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.886 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-03-21T09:22:57.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:22:57.888 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 4, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 2, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 1, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 564, 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-03-21T09:22:57.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:22:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:22:57.891 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 264197, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 93, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-03-21T09:22:57.892 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 9736, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 203, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 99661, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 381, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-03-21T09:22:57.893 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 47, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 12976, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 33, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 47, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 47, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.286994180, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.006106259 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.894 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 47, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 71 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 47, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 0, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 47, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 0, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 7, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 9, 2024-03-21T09:22:57.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 6, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-03-21T09:22:57.896 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-03-21T09:22:57.897 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 103, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 13, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 9, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 9, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 17, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 2 2024-03-21T09:22:57.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-03-21T09:22:57.899 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 1433, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2125729, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 1433, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:22:57.900 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1433, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2125729, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:22:57.901 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 47, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 12976, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 47, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 12976, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:22:57.902 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 47, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 47, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 47, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 47, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:22:57.904 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x5566d8dbc460": { 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 8, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2349, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 8, 2024-03-21T09:22:57.905 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 7, 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2349, 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:22:57.906 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=15 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-03-21T09:22:58.760 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T09:22:58.761 DEBUG:tasks.check_counter:Getting stats from e 2024-03-21T09:22:58.761 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-21T09:22:59.017 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:22:59.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:22:59 smithi037 ceph-mon[31830]: pgmap v335: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:59.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:22:59 smithi053 ceph-mon[36169]: pgmap v335: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:22:59.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:22:59 smithi071 ceph-mon[36597]: pgmap v335: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4, 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 4, 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3072, 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 448, 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-03-21T09:23:00.752 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001565920, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000033936, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000611305, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000675627, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000225209 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-21T09:23:00.753 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 249, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 236, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1864080, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 142256, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.041136645, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.009380449, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.039562563, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.004250445, 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:00.754 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 235, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.008109615, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000034509 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1864080, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 142256 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 181, 2024-03-21T09:23:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 320, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 265088, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.008292101, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.005165727, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.001542689, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 178, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.017577662, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000098750 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:00.756 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 265088 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:00.757 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:00.758 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:00.759 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 148692, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 30, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:00.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:00.761 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 251, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1828926, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 251, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 251, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1828926, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:00.762 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:00.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:00.763 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-03-21T09:23:01.395 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-03-21T09:23:01.396 DEBUG:tasks.check_counter:Getting stats from h 2024-03-21T09:23:01.396 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-21T09:23:01.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:01 smithi053 ceph-mon[36169]: pgmap v336: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:23:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:23:01.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:01 smithi037 ceph-mon[31830]: pgmap v336: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:23:01.858 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:01 smithi071 ceph-mon[36597]: pgmap v336: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T09:23:03.407 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 819, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 821, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4401281, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 260506, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.055840202, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.016376873, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.015006050, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.018662649, 2024-03-21T09:23:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 819, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020294111, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000024779 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1035, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1013, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 9382841, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 385943, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.089863555, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.023381063, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.056392779, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.019598548, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1013, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.026262801, 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000025925 2024-03-21T09:23:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1919312, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 144576 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 867, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1045, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 4481720, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 818990, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2024-03-21T09:23:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.070551578, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.023607835, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.019008047, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.020593158, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1042, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.047264254, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000045359 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 543920 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4139, 2024-03-21T09:23:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.082428765, 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000019915 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-03-21T09:23:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 1219, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 1219, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 1219, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-03-21T09:23:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 1, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-03-21T09:23:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-03-21T09:23:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:23:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:23:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-03-21T09:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:23:03.422 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 6071, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-03-21T09:23:03.423 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 9, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 19851337, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 19849722, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 6071 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 13, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 1228, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 1215, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 10, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 21, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 11, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 1219, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 1232, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 13, 2024-03-21T09:23:03.425 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 49148, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215420 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-03-21T09:23:03.426 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.427 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-03-21T09:23:03.428 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-03-21T09:23:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.433 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.434 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.435 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-03-21T09:23:03.436 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.437 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.438 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.439 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 4, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 3866, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:03.440 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:03.441 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:03.442 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 207210, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 66, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:23:03.443 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 9736, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 203, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:03.444 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 909270, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 5140, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 2443, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-03-21T09:23:03.445 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 2443, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2443, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.820964146, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000745380 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2443, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2444 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 2443, 2024-03-21T09:23:03.446 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2443, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1557, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 884, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-03-21T09:23:03.447 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:23:03.448 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-03-21T09:23:03.449 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-03-21T09:23:03.450 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-03-21T09:23:03.450 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-03-21T09:23:03.450 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-03-21T09:23:03.450 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-03-21T09:23:03.450 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 103, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 13, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 7, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 7, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-03-21T09:23:03.451 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-03-21T09:23:03.452 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2717, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 18004301, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2717, 2024-03-21T09:23:03.453 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2717, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 18004301, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-03-21T09:23:03.454 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2443, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 15657038, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2443, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 15657038, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2443, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2443, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2443, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2443, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:23:03.457 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55cd7ca7a6e0": { 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:03.458 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:03.459 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-03-21T09:23:03.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:03 smithi037 ceph-mon[31830]: pgmap v337: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:23:03.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:03 smithi053 ceph-mon[36169]: pgmap v337: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:23:03.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:03 smithi071 ceph-mon[36597]: pgmap v337: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-21T09:23:04.214 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-21T09:23:04.214 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-21T09:23:04.214 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-21T09:23:04.214 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-03-21T09:23:04.215 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-03-21T09:23:04.215 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-21T09:23:04.215 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T09:23:04.215 DEBUG:tasks.check_counter:Getting stats from k 2024-03-21T09:23:04.215 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-21T09:23:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:23:05.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:05 smithi037 ceph-mon[31830]: pgmap v338: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:23:05.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:05 smithi053 ceph-mon[36169]: pgmap v338: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:23:05.856 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:05 smithi071 ceph-mon[36597]: pgmap v338: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 258, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 243, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1869136, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 145536, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.040696751, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.009455003, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.039178493, 2024-03-21T09:23:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.004378761, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 243, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.007883301, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000032441 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1869136, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 145536 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 183, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 320, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 187520, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.008036730, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.004932192, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.001455466, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 180, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.017125524, 2024-03-21T09:23:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000095141 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 187520 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 0, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 0, 2024-03-21T09:23:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.000167104, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000111646, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 148692, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 30, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 256, 2024-03-21T09:23:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1830213, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 256, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 256, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1830213, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-03-21T09:23:06.375 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-03-21T09:23:07.117 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-21T09:23:07.117 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-21T09:23:07.117 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-21T09:23:07.117 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-21T09:23:07.118 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-21T09:23:07.118 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-21T09:23:07.118 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-21T09:23:07.118 DEBUG:tasks.check_counter:Getting stats from c 2024-03-21T09:23:07.118 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-21T09:23:07.393 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:23:07.802 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:07 smithi071 ceph-mon[36597]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T09:23:07.802 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:07 smithi071 ceph-mon[36597]: pgmap v340: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:07.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:07 smithi037 ceph-mon[31830]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T09:23:07.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:07 smithi037 ceph-mon[31830]: pgmap v340: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:07.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:07 smithi053 ceph-mon[36169]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T09:23:07.945 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:07 smithi053 ceph-mon[36169]: pgmap v340: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2008, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1988, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2780031, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 696492, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.138126955, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.038536993, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.064790140, 2024-03-21T09:23:09.305 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.039036312, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1987, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.043192874, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000021737 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1918256, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 145712 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:09.306 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 7, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 187, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2136, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 547386, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 8, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.009170054, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.005178371, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.002435656, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000089225, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 184, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.022840643, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000124133 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.307 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 546048 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 835, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 835, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 207604, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 264713, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-03-21T09:23:09.308 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.056960430, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.018167803, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.013037347, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.021332271, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 834, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.018500799, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000022183 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-21T09:23:09.309 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4259, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.083216717, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000019539 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-03-21T09:23:09.310 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.311 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 21, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 7, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 14, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 12, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-03-21T09:23:09.312 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 2, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-03-21T09:23:09.313 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-03-21T09:23:09.314 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-03-21T09:23:09.315 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:23:09.316 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 165, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-03-21T09:23:09.317 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 2, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 13, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4458921, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 4541653, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 4541593, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.318 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 165 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 16, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 20, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 4, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 11, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 13, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 2, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 21, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 24, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 3, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-03-21T09:23:09.319 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 47436, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 215420 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.320 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.322 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.323 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.326 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.327 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.328 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.329 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:23:09.330 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 7, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 5, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 4, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-03-21T09:23:09.331 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 7380, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:09.332 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:09.333 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 190228, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 55, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-21T09:23:09.334 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9976, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 209, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 74684, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 434, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-21T09:23:09.335 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 2567, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 26, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 4, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 2567, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2567, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.972982647, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000379034 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.336 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2567, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2568 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 2567, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2567, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 1656, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 908, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-03-21T09:23:09.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 2, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:23:09.338 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-03-21T09:23:09.339 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 104, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 14, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 5, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 5, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-03-21T09:23:09.340 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-03-21T09:23:09.341 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 2846, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 2716377, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:09.342 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 2846, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2846, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2716377, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:09.343 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2567, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 347359, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2567, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 347359, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.344 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2567, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2567, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2567, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2567, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.345 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-03-21T09:23:09.346 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55fd178346e0": { 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:09.347 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:09.348 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-21T09:23:09.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:09 smithi071 ceph-mon[36597]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:09.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:09 smithi037 ceph-mon[31830]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:09.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:09 smithi053 ceph-mon[36169]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-21T09:23:10.159 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T09:23:10.159 DEBUG:tasks.check_counter:Getting stats from f 2024-03-21T09:23:10.159 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-21T09:23:10.401 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:23:11.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:11 smithi037 ceph-mon[31830]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:11.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:11 smithi053 ceph-mon[36169]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:11.737 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:11 smithi071 ceph-mon[36597]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2329, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2331, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 663992, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 740629, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 2, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.128873714, 2024-03-21T09:23:12.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.039561455, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.032172284, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.047509125, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2329, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.046406795, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000019925 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:23:12.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 283, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 261, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1922268, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 147946, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 3, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 2, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.040007372, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.009535652, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.039596439, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.004294197, 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:12.149 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 261, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.009781982, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000037478 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1921712, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 147248 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 3, 2024-03-21T09:23:12.150 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 184, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 588, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 470046, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.008153299, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.005102133, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001447459, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000031508, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 181, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.017948898, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000099165 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:12.151 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 469728 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-03-21T09:23:12.152 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3996, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.073237879, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000018327 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-03-21T09:23:12.153 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-03-21T09:23:12.154 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 1, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-03-21T09:23:12.155 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:23:12.156 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-03-21T09:23:12.157 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-03-21T09:23:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-03-21T09:23:12.159 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 8, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 2, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 4196653, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 4196347, 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-03-21T09:23:12.160 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 8 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 3, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 3, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 0, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 1, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 1, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 0, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 0, 2024-03-21T09:23:12.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 48072, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 215420 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-03-21T09:23:12.162 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.164 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-03-21T09:23:12.165 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.166 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-03-21T09:23:12.167 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.167 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.167 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.167 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-03-21T09:23:12.167 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-03-21T09:23:12.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-03-21T09:23:12.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.170 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 4, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 2, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 5734, 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-03-21T09:23:12.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:12.175 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:12.176 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 182037, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 48, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-21T09:23:12.177 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9976, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 209, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 8320, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 13, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-21T09:23:12.178 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 2332, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 21, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 3, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 2332, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2332, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.995982244, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000427093 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2332, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2333 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 2332, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2332, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 1491, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 839, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-03-21T09:23:12.180 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 1, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:23:12.181 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-03-21T09:23:12.182 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 104, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 14, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 4, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 4, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-03-21T09:23:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-03-21T09:23:12.184 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 2611, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 2334589, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:12.185 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 2611, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2611, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2334589, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:12.186 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2332, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2354, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2332, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2354, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.187 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2332, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2332, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2332, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2332, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-03-21T09:23:12.189 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x563f773a06e0": { 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:12.190 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:12.191 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-21T09:23:12.935 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T09:23:12.935 DEBUG:tasks.check_counter:Getting stats from i 2024-03-21T09:23:12.936 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-21T09:23:13.189 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:23:13.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:13 smithi071 ceph-mon[36597]: pgmap v343: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:23:13.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:13 smithi037 ceph-mon[31830]: pgmap v343: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:23:13.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:13 smithi053 ceph-mon[36169]: pgmap v343: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:23:14.933 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 8175, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 13041, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2499605, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 9130393, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 16, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.471290344, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.193054121, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.240277040, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.079847397, 2024-03-21T09:23:14.934 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 13039, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.340190892, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000026090 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5927, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001858936, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000313 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 736 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2559, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2754, 2024-03-21T09:23:14.935 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 638978, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 9035356, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 47, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.215257025, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.083718580, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.054149262, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.061615314, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2751, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.123583318, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000044923 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 84, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000010406, 2024-03-21T09:23:14.936 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000123 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 4064, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 478480 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2792, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 3447, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2682740, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 5659759, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 11, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.228476226, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.083959980, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.112878275, 2024-03-21T09:23:14.937 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.041956024, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3447, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.118979251, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000034516 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2646, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000449777, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000169 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1927968, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 151168 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-21T09:23:14.938 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4190, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.935094197, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000461836 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-03-21T09:23:14.939 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5524, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.236152129, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000042750 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 2430, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 2428, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2428, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.409971804, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000992574 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-03-21T09:23:14.940 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 2, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 11, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 19, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 1, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 7, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 11, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 16, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 6, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 1, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 1, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 3, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 2457, 2024-03-21T09:23:14.941 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 2434, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 1, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 3, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 1, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 2, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 1, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 2, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-03-21T09:23:14.942 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 2, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 2, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 1, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 1, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 1, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 17, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 3627, 2024-03-21T09:23:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 2426, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 1210, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 12, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 1223 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 4, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 11, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 7, 2024-03-21T09:23:14.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 8, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 4, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 1, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 1209, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 1208, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:23:14.945 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 1, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 6071, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 6071, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-03-21T09:23:14.946 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 9, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 9, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 19851337, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2406, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 6.662727040, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002769213 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.947 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 22, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1233, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 1211, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 20, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 27, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 7, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 19, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 1237, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 1218, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 1, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 1214, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 1213, 2024-03-21T09:23:14.948 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 56104, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 223612 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 2682, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 1, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 2430, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 26, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 7, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1204, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.790193823, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000656307 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.949 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001837768, 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001837768 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002029535, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000289933 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000137793, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000137793 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.952 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.953 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-03-21T09:23:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-03-21T09:23:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.004893966, 2024-03-21T09:23:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001631322 2024-03-21T09:23:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-03-21T09:23:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-03-21T09:23:14.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.007726869, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002575623 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.956 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.030641566, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007660391 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.959 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1205, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.572510484, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001304987 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 4, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 2, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 1954, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 977, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 648, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.961 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:14.962 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:14.963 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 316548, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 135, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 704, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 8, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:14.964 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9976, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 209, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 100459, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 610, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:14.965 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 7265, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 15907303, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 28, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 7265, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7265, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 33.917386106, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004668600 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-03-21T09:23:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7265, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 7349 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 7265, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 28, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 7237, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 26, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 2432, 2024-03-21T09:23:14.967 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 25, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 4706, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 2, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-03-21T09:23:14.968 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-03-21T09:23:14.969 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 104, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 14, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 113, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 15 2024-03-21T09:23:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 19, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-03-21T09:23:14.971 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 7, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 4691, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 1208, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 13522, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 4663015, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 13522, 2024-03-21T09:23:14.972 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 13522, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 4663015, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:14.973 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 7265, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 16038959, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 7265, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 16038959, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-03-21T09:23:14.974 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 7265, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 7265, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 7265, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 7265, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1208, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 131656, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1208, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 26, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 131656, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.976 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55e068b9c460": { 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 6071, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 15657033, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 6071, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2406, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 15657033, 2024-03-21T09:23:14.977 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:14.978 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:14.978 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:14.978 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:14.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.978 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:14.978 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-21T09:23:15.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:15 smithi071 ceph-mon[36597]: pgmap v344: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:23:15.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:15 smithi037 ceph-mon[31830]: pgmap v344: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:23:15.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:15 smithi053 ceph-mon[36169]: pgmap v344: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-21T09:23:15.714 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-21T09:23:15.714 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-03-21T09:23:15.714 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=4 2024-03-21T09:23:15.714 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-03-21T09:23:15.714 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-21T09:23:15.715 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-21T09:23:15.715 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T09:23:15.715 DEBUG:tasks.check_counter:Getting stats from l 2024-03-21T09:23:15.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-21T09:23:15.985 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:23:17.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:17 smithi037 ceph-mon[31830]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T09:23:17.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:17 smithi053 ceph-mon[36169]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T09:23:17.462 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:17 smithi071 ceph-mon[36597]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 261, 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 246, 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1869952, 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 147408, 2024-03-21T09:23:17.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.044039097, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.009649025, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.042083957, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.004153063, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 246, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.008789733, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000035730 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:17.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1869952, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 147408 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 187, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 320, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 188224, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.008232835, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.005266297, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001325256, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:23:17.713 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 184, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.019475361, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000105844 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 188224 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.000192075, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000104563, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-21T09:23:17.716 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:23:17.717 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:23:17.718 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148709, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 31, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-21T09:23:17.719 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-21T09:23:17.720 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 259, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1830603, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 259, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-21T09:23:17.721 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 259, 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1830603, 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-21T09:23:17.722 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-21T09:23:18.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:18 smithi071 ceph-mon[36597]: pgmap v346: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-21T09:23:18.419 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-21T09:23:18.419 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-21T09:23:18.429 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-21T09:23:18.429 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:23:18.429 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:23:18.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:18 smithi037 ceph-mon[31830]: pgmap v346: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:18.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:18 smithi053 ceph-mon[36169]: pgmap v346: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:18.493 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-21T09:23:18.493 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:23:18.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:23:18.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 -- 172.21.15.37:0/2645794098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e4810bb20 msgr2=0x7f2e4810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 --2- 172.21.15.37:0/2645794098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e4810bb20 0x7f2e4810dfc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009ca0 tx=0x7f2e4002c180 comp rx=0 tx=0).stop 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 -- 172.21.15.37:0/2645794098 shutdown_connections 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 --2- 172.21.15.37:0/2645794098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e4810bb20 0x7f2e4810dfc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 --2- 172.21.15.37:0/2645794098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e48103020 0x7f2e4810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 --2- 172.21.15.37:0/2645794098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e48102700 0x7f2e48102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 -- 172.21.15.37:0/2645794098 >> 172.21.15.37:0/2645794098 conn(0x7f2e480fbf00 msgr2=0x7f2e480fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 -- 172.21.15.37:0/2645794098 shutdown_connections 2024-03-21T09:23:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.767+0000 7f2e47e00640 1 -- 172.21.15.37:0/2645794098 wait complete. 2024-03-21T09:23:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.768+0000 7f2e47e00640 1 Processor -- start 2024-03-21T09:23:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.768+0000 7f2e47e00640 1 -- start start 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.768+0000 7f2e47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 0x7f2e4819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.768+0000 7f2e47e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e48103020 0x7f2e4819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e47e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e4810bb20 0x7f2e4819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e47e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e481118e0 con 0x7f2e4810bb20 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e47e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e48111760 con 0x7f2e48103020 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e47e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e48111a60 con 0x7f2e48102700 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 0x7f2e4819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e48103020 0x7f2e4819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 0x7f2e4819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42840/0 (socket says 172.21.15.37:42840) 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 -- 172.21.15.37:0/1843799467 learned_addr learned my addr 172.21.15.37:0/1843799467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e47400640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e4810bb20 0x7f2e4819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 -- 172.21.15.37:0/1843799467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e48103020 msgr2=0x7f2e4819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e48103020 0x7f2e4819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 -- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e4810bb20 msgr2=0x7f2e4819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e4810bb20 0x7f2e4819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.769+0000 7f2e46a00640 1 -- 172.21.15.37:0/1843799467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4819eff0 con 0x7f2e48102700 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.770+0000 7f2e46a00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 0x7f2e4819a000 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800e9f0 tx=0x7f2e3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.770+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800cd80 con 0x7f2e48102700 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.770+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e38010910 con 0x7f2e48102700 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.770+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4819f2e0 con 0x7f2e48102700 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.770+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38018990 con 0x7f2e48102700 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.770+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e481a6b70 con 0x7f2e48102700 2024-03-21T09:23:18.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.771+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4804f470 con 0x7f2e48102700 2024-03-21T09:23:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.773+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2e38010ac0 con 0x7f2e48102700 2024-03-21T09:23:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.774+0000 7f2e3f400640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2e2c0a3100 0x7f2e2c0a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.774+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2e380cfed0 con 0x7f2e48102700 2024-03-21T09:23:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.775+0000 7f2e46000640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2e2c0a3100 0x7f2e2c0a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.775+0000 7f2e46000640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2e2c0a3100 0x7f2e2c0a55c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e4810d330 tx=0x7f2e34005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.778+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e3809bf10 con 0x7f2e48102700 2024-03-21T09:23:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.994+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e481a6da0 con 0x7f2e48102700 2024-03-21T09:23:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.996+0000 7f2e3f400640 1 -- 172.21.15.37:0/1843799467 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25290 (secure 0 0 0) 0x7f2e3809ffb0 con 0x7f2e48102700 2024-03-21T09:23:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:23:18.994 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":105,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:23:16.128305+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2204787263":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1173483074":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2805488546":"2024-03-22T09:12:10.987337+0000","172.21.15.37:6800/1300592626":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6801/1300592626":"2024-03-22T09:12:10.987337+0000","172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/1005590151":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/1759352225":"2024-03-22T09:12:10.987337+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:23:18.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2e2c0a3100 msgr2=0x7f2e2c0a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:18.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2e2c0a3100 0x7f2e2c0a55c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e4810d330 tx=0x7f2e34005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:18.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 msgr2=0x7f2e4819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:18.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 0x7f2e4819a000 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800e9f0 tx=0x7f2e3800eeb0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 shutdown_connections 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e4810bb20 0x7f2e4819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f2e2c0a3100 0x7f2e2c0a55c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e48103020 0x7f2e4819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 --2- 172.21.15.37:0/1843799467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48102700 0x7f2e4819a000 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 >> 172.21.15.37:0/1843799467 conn(0x7f2e480fbf00 msgr2=0x7f2e480fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 shutdown_connections 2024-03-21T09:23:18.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:18.999+0000 7f2e47e00640 1 -- 172.21.15.37:0/1843799467 wait complete. 2024-03-21T09:23:19.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T09:03:30.335349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T09:08:44.251938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T09:08:45.778848+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T09:08:49.461084+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T09:23:19.016 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 -- 172.21.15.37:0/128214923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630103660 msgr2=0x7f7630103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 --2- 172.21.15.37:0/128214923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630103660 0x7f7630103ae0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7618009930 tx=0x7f761802bc30 comp rx=0 tx=0).stop 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 -- 172.21.15.37:0/128214923 shutdown_connections 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 --2- 172.21.15.37:0/128214923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7630104020 0x7f7630110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 --2- 172.21.15.37:0/128214923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630103660 0x7f7630103ae0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 --2- 172.21.15.37:0/128214923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 0x7f7630109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 -- 172.21.15.37:0/128214923 >> 172.21.15.37:0/128214923 conn(0x7f76300ff110 msgr2=0x7f7630101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 -- 172.21.15.37:0/128214923 shutdown_connections 2024-03-21T09:23:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.262+0000 7f7635e00640 1 -- 172.21.15.37:0/128214923 wait complete. 2024-03-21T09:23:19.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 Processor -- start 2024-03-21T09:23:19.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 -- start start 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7630103660 0x7f7630199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 0x7f763019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 0x7f763019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630113ad0 con 0x7f7630103660 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630113950 con 0x7f7630109670 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.264+0000 7f7635e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630113c50 con 0x7f7630104020 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 0x7f763019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f7635400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 0x7f763019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f7634a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7630103660 0x7f7630199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f7635400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 0x7f763019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36676/0 (socket says 172.21.15.37:36676) 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f7635400640 1 -- 172.21.15.37:0/1438802123 learned_addr learned my addr 172.21.15.37:0/1438802123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 0x7f763019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42858/0 (socket says 172.21.15.37:42858) 2024-03-21T09:23:19.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 -- 172.21.15.37:0/1438802123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 msgr2=0x7f763019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 0x7f763019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 -- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7630103660 msgr2=0x7f7630199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7630103660 0x7f7630199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 -- 172.21.15.37:0/1438802123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763019efc0 con 0x7f7630104020 2024-03-21T09:23:19.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.265+0000 7f762fe00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 0x7f763019a510 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7618036040 tx=0x7f7618005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:19.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.266+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618043020 con 0x7f7630104020 2024-03-21T09:23:19.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.266+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7618005000 con 0x7f7630104020 2024-03-21T09:23:19.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.266+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618003af0 con 0x7f7630104020 2024-03-21T09:23:19.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.266+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763019f250 con 0x7f7630104020 2024-03-21T09:23:19.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.266+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76301a6b40 con 0x7f7630104020 2024-03-21T09:23:19.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.267+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763004f400 con 0x7f7630104020 2024-03-21T09:23:19.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.270+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7618046020 con 0x7f7630104020 2024-03-21T09:23:19.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.272+0000 7f762d600640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f76140a3110 0x7f76140a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.272+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f76180f19e0 con 0x7f7630104020 2024-03-21T09:23:19.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.272+0000 7f7634a00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f76140a3110 0x7f76140a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.273+0000 7f7634a00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f76140a3110 0x7f76140a55d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7620005fd0 tx=0x7f7620009500 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:19.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.276+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f761804d020 con 0x7f7630104020 2024-03-21T09:23:19.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:19 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1843799467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:23:19.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:19 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1843799467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:23:19.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.478+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f763010bbd0 con 0x7f7630104020 2024-03-21T09:23:19.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.478+0000 7f762d600640 1 -- 172.21.15.37:0/1438802123 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76180be180 con 0x7f7630104020 2024-03-21T09:23:19.476 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-21T09:23:19.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.481+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f76140a3110 msgr2=0x7f76140a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.481+0000 7f7635e00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f76140a3110 0x7f76140a55d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7620005fd0 tx=0x7f7620009500 comp rx=0 tx=0).stop 2024-03-21T09:23:19.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.481+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 msgr2=0x7f763019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.481+0000 7f7635e00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 0x7f763019a510 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7618036040 tx=0x7f7618005e80 comp rx=0 tx=0).stop 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 shutdown_connections 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f76140a3110 0x7f76140a55d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7630109670 0x7f763019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7630104020 0x7f763019a510 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 --2- 172.21.15.37:0/1438802123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7630103660 0x7f7630199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 >> 172.21.15.37:0/1438802123 conn(0x7f76300ff110 msgr2=0x7f76300ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 shutdown_connections 2024-03-21T09:23:19.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.482+0000 7f7635e00640 1 -- 172.21.15.37:0/1438802123 wait complete. 2024-03-21T09:23:19.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:23:19.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:19 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1843799467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 -- 172.21.15.37:0/3665494242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100250 msgr2=0x7f99dc100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 --2- 172.21.15.37:0/3665494242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100250 0x7f99dc100630 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009900 tx=0x7f99cc02bcd0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 -- 172.21.15.37:0/3665494242 shutdown_connections 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 --2- 172.21.15.37:0/3665494242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99dc10de00 0x7f99dc1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 --2- 172.21.15.37:0/3665494242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100b70 0x7f99dc10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 --2- 172.21.15.37:0/3665494242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100250 0x7f99dc100630 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 -- 172.21.15.37:0/3665494242 >> 172.21.15.37:0/3665494242 conn(0x7f99dc077130 msgr2=0x7f99dc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 -- 172.21.15.37:0/3665494242 shutdown_connections 2024-03-21T09:23:19.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.720+0000 7f99db200640 1 -- 172.21.15.37:0/3665494242 wait complete. 2024-03-21T09:23:19.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.721+0000 7f99db200640 1 Processor -- start 2024-03-21T09:23:19.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.721+0000 7f99db200640 1 -- start start 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99db200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 0x7f99dc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99db200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100b70 0x7f99dc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99db200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99dc10de00 0x7f99dc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99db200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc113ab0 con 0x7f99dc10de00 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99db200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc113930 con 0x7f99dc100250 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99db200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc113c30 con 0x7f99dc100b70 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 0x7f99dc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100b70 0x7f99dc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 0x7f99dc199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36694/0 (socket says 172.21.15.37:36694) 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 -- 172.21.15.37:0/1424179444 learned_addr learned my addr 172.21.15.37:0/1424179444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99da800640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99dc10de00 0x7f99dc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 -- 172.21.15.37:0/1424179444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100b70 msgr2=0x7f99dc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100b70 0x7f99dc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 -- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99dc10de00 msgr2=0x7f99dc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99dc10de00 0x7f99dc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.722+0000 7f99d9e00640 1 -- 172.21.15.37:0/1424179444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc19efa0 con 0x7f99dc100250 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.723+0000 7f99d9e00640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 0x7f99dc199fb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f99cc000c00 tx=0x7f99cc0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.723+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc042020 con 0x7f99dc100250 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.723+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99cc0054e0 con 0x7f99dc100250 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.723+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc19f230 con 0x7f99dc100250 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.723+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc003af0 con 0x7f99dc100250 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.723+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc1a6b20 con 0x7f99dc100250 2024-03-21T09:23:19.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.724+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc04f400 con 0x7f99dc100250 2024-03-21T09:23:19.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.728+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f99cc045020 con 0x7f99dc100250 2024-03-21T09:23:19.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.729+0000 7f99d2a00640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f99c00a3090 0x7f99c00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:19.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.729+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f99cc0f1850 con 0x7f99dc100250 2024-03-21T09:23:19.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.729+0000 7f99d9400640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f99c00a3090 0x7f99c00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:19.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.730+0000 7f99d9400640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f99c00a3090 0x7f99c00a5550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99dc10f0b0 tx=0x7f99c8005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:19.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.738+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99cc04c020 con 0x7f99dc100250 2024-03-21T09:23:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.981+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99dc108f90 con 0x7f99dc100250 2024-03-21T09:23:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.981+0000 7f99d2a00640 1 -- 172.21.15.37:0/1424179444 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99cc0bdff0 con 0x7f99dc100250 2024-03-21T09:23:19.979 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:23:19.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f99c00a3090 msgr2=0x7f99c00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f99c00a3090 0x7f99c00a5550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99dc10f0b0 tx=0x7f99c8005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 msgr2=0x7f99dc199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 0x7f99dc199fb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f99cc000c00 tx=0x7f99cc0043c0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 shutdown_connections 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99dc10de00 0x7f99dc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99dc100b70 0x7f99dc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f99c00a3090 0x7f99c00a5550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 --2- 172.21.15.37:0/1424179444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99dc100250 0x7f99dc199fb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 >> 172.21.15.37:0/1424179444 conn(0x7f99dc077130 msgr2=0x7f99dc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 shutdown_connections 2024-03-21T09:23:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:19.984+0000 7f99db200640 1 -- 172.21.15.37:0/1424179444 wait complete. 2024-03-21T09:23:20.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:23:20.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:20 smithi037 ceph-mon[31830]: pgmap v347: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:20.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:20 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1438802123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:23:20.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:20 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1424179444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:23:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2927105270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 msgr2=0x7fdf8010b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2927105270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf8010b430 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009900 tx=0x7fdf6c02bcd0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2927105270 shutdown_connections 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2927105270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf8010ba50 0x7fdf8010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2927105270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf8010b430 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2927105270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80102630 0x7fdf80102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2927105270 >> 172.21.15.37:0/2927105270 conn(0x7fdf80077130 msgr2=0x7fdf800fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2927105270 shutdown_connections 2024-03-21T09:23:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.223+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2927105270 wait complete. 2024-03-21T09:23:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 Processor -- start 2024-03-21T09:23:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 -- start start 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80102630 0x7fdf80197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf80198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf8010ba50 0x7fdf8019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80111850 con 0x7fdf8010ba50 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801116d0 con 0x7fdf80102630 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.225+0000 7fdf7fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801119d0 con 0x7fdf80102f50 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf8010ba50 0x7fdf8019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80102630 0x7fdf80197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf80198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80102630 0x7fdf80197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36698/0 (socket says 172.21.15.37:36698) 2024-03-21T09:23:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7ea00640 1 -- 172.21.15.37:0/2942726398 learned_addr learned my addr 172.21.15.37:0/2942726398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf80198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42888/0 (socket says 172.21.15.37:42888) 2024-03-21T09:23:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 -- 172.21.15.37:0/2942726398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80102630 msgr2=0x7fdf80197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80102630 0x7fdf80197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 -- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf8010ba50 msgr2=0x7fdf8019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf8010ba50 0x7fdf8019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.226+0000 7fdf7e000640 1 -- 172.21.15.37:0/2942726398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8019cdc0 con 0x7fdf80102f50 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.227+0000 7fdf7e000640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf80198310 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c02c210 tx=0x7fdf6c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.227+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c042020 con 0x7fdf80102f50 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.227+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c02ce00 con 0x7fdf80102f50 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.227+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8019d050 con 0x7fdf80102f50 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.227+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c003a50 con 0x7fdf80102f50 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.227+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf801a4940 con 0x7fdf80102f50 2024-03-21T09:23:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.228+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8004f400 con 0x7fdf80102f50 2024-03-21T09:23:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.230+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fdf6c045020 con 0x7fdf80102f50 2024-03-21T09:23:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.232+0000 7fdf77400640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdf640a3110 0x7fdf640a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.232+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdf6c0f1560 con 0x7fdf80102f50 2024-03-21T09:23:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.232+0000 7fdf7ea00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdf640a3110 0x7fdf640a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.233+0000 7fdf7ea00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdf640a3110 0x7fdf640a55d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009c90 tx=0x7fdf70005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.236+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf6c04c020 con 0x7fdf80102f50 2024-03-21T09:23:20.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:20 smithi053 ceph-mon[36169]: pgmap v347: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:20.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:20 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1438802123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:23:20.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:20 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1424179444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:23:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.517+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf80106af0 con 0x7fdf80102f50 2024-03-21T09:23:20.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.518+0000 7fdf77400640 1 -- 172.21.15.37:0/2942726398 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf6c0bdd00 con 0x7fdf80102f50 2024-03-21T09:23:20.515 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-21T09:23:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdf640a3110 msgr2=0x7fdf640a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdf640a3110 0x7fdf640a55d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009c90 tx=0x7fdf70005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 msgr2=0x7fdf80198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf80198310 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c02c210 tx=0x7fdf6c005e80 comp rx=0 tx=0).stop 2024-03-21T09:23:20.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 shutdown_connections 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf8010ba50 0x7fdf8019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf80102f50 0x7fdf80198310 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fdf640a3110 0x7fdf640a55d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 --2- 172.21.15.37:0/2942726398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80102630 0x7fdf80197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 >> 172.21.15.37:0/2942726398 conn(0x7fdf80077130 msgr2=0x7fdf800fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 shutdown_connections 2024-03-21T09:23:20.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.521+0000 7fdf7fe00640 1 -- 172.21.15.37:0/2942726398 wait complete. 2024-03-21T09:23:20.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T09:23:20.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:20 smithi071 ceph-mon[36597]: pgmap v347: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:20.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:20 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1438802123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:23:20.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:20 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1424179444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 -- 172.21.15.37:0/2841083359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941014a0 msgr2=0x7f7894101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 --2- 172.21.15.37:0/2841083359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941014a0 0x7f7894101920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7884009930 tx=0x7f788402bc20 comp rx=0 tx=0).stop 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 -- 172.21.15.37:0/2841083359 shutdown_connections 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 --2- 172.21.15.37:0/2841083359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7894101e60 0x7f789410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 --2- 172.21.15.37:0/2841083359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941014a0 0x7f7894101920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 --2- 172.21.15.37:0/2841083359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894107560 0x7f7894107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 -- 172.21.15.37:0/2841083359 >> 172.21.15.37:0/2841083359 conn(0x7f78940fcfb0 msgr2=0x7f78940ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 -- 172.21.15.37:0/2841083359 shutdown_connections 2024-03-21T09:23:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.724+0000 7f789b000640 1 -- 172.21.15.37:0/2841083359 wait complete. 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.725+0000 7f789b000640 1 Processor -- start 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.725+0000 7f789b000640 1 -- start start 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789b000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78941014a0 0x7f7894199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 0x7f789419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789b000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7894107560 0x7f789419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789b000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894111870 con 0x7f78941014a0 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789b000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941116f0 con 0x7f7894107560 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941119f0 con 0x7f7894101e60 2024-03-21T09:23:20.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 0x7f789419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 0x7f789419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42902/0 (socket says 172.21.15.37:42902) 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 -- 172.21.15.37:0/2752360535 learned_addr learned my addr 172.21.15.37:0/2752360535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f789a600640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7894107560 0x7f789419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899c00640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78941014a0 0x7f7894199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 -- 172.21.15.37:0/2752360535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7894107560 msgr2=0x7f789419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7894107560 0x7f789419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 -- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78941014a0 msgr2=0x7f7894199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78941014a0 0x7f7894199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899200640 1 -- 172.21.15.37:0/2752360535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789419ef60 con 0x7f7894101e60 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.726+0000 7f7899c00640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78941014a0 0x7f7894199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:23:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.727+0000 7f7899200640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 0x7f789419a4b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7884005ab0 tx=0x7f7884005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.727+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884043020 con 0x7f7894101e60 2024-03-21T09:23:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.727+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7884005000 con 0x7f7894101e60 2024-03-21T09:23:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.727+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884003ab0 con 0x7f7894101e60 2024-03-21T09:23:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.727+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789419f1f0 con 0x7f7894101e60 2024-03-21T09:23:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.727+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78941a6ae0 con 0x7f7894101e60 2024-03-21T09:23:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.729+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789404f400 con 0x7f7894101e60 2024-03-21T09:23:20.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.730+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7884046020 con 0x7f7894101e60 2024-03-21T09:23:20.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.732+0000 7f7892000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f787c0a3110 0x7f787c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:20.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.733+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f78840f1aa0 con 0x7f7894101e60 2024-03-21T09:23:20.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.733+0000 7f7899c00640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f787c0a3110 0x7f787c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:20.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.733+0000 7f7899c00640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f787c0a3110 0x7f787c0a55d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7888009c20 tx=0x7f7888005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:20.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.735+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f788404d020 con 0x7f7894101e60 2024-03-21T09:23:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.942+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f78941a6d40 con 0x7f7894101e60 2024-03-21T09:23:20.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.942+0000 7f7892000640 1 -- 172.21.15.37:0/2752360535 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7f78840be240 con 0x7f7894101e60 2024-03-21T09:23:20.940 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f787c0a3110 msgr2=0x7f787c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f787c0a3110 0x7f787c0a55d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7888009c20 tx=0x7f7888005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 msgr2=0x7f789419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 0x7f789419a4b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7884005ab0 tx=0x7f7884005e80 comp rx=0 tx=0).stop 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 shutdown_connections 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f787c0a3110 0x7f787c0a55d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7894107560 0x7f789419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7894101e60 0x7f789419a4b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 --2- 172.21.15.37:0/2752360535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78941014a0 0x7f7894199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.945+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 >> 172.21.15.37:0/2752360535 conn(0x7f78940fcfb0 msgr2=0x7f78940fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.946+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 shutdown_connections 2024-03-21T09:23:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:20.946+0000 7f789b000640 1 -- 172.21.15.37:0/2752360535 wait complete. 2024-03-21T09:23:20.962 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T09:23:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 -- 172.21.15.37:0/2201459813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac0743f0 msgr2=0x7f8fac10d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 --2- 172.21.15.37:0/2201459813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac0743f0 0x7f8fac10d760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009900 tx=0x7f8f9802bcc0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 -- 172.21.15.37:0/2201459813 shutdown_connections 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 --2- 172.21.15.37:0/2201459813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fac10dca0 0x7f8fac1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 --2- 172.21.15.37:0/2201459813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac0743f0 0x7f8fac10d760 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 --2- 172.21.15.37:0/2201459813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fac073ad0 0x7f8fac073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 -- 172.21.15.37:0/2201459813 >> 172.21.15.37:0/2201459813 conn(0x7f8fac0fcff0 msgr2=0x7f8fac0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 -- 172.21.15.37:0/2201459813 shutdown_connections 2024-03-21T09:23:21.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.170+0000 7f8fab200640 1 -- 172.21.15.37:0/2201459813 wait complete. 2024-03-21T09:23:21.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 Processor -- start 2024-03-21T09:23:21.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 -- start start 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 0x7f8fac199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fac0743f0 0x7f8fac19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fac10dca0 0x7f8fac19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac113a40 con 0x7f8fac0743f0 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1138c0 con 0x7f8fac10dca0 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.171+0000 7f8fab200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac113bc0 con 0x7f8fac073ad0 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 0x7f8fac199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8faa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fac10dca0 0x7f8fac19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 0x7f8fac199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42928/0 (socket says 172.21.15.37:42928) 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 -- 172.21.15.37:0/1600367913 learned_addr learned my addr 172.21.15.37:0/1600367913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9400640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fac0743f0 0x7f8fac19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:21.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 -- 172.21.15.37:0/1600367913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fac10dca0 msgr2=0x7f8fac19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:21.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fac10dca0 0x7f8fac19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 -- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fac0743f0 msgr2=0x7f8fac19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:21.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fac0743f0 0x7f8fac19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 -- 172.21.15.37:0/1600367913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac19ef60 con 0x7f8fac073ad0 2024-03-21T09:23:21.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.172+0000 7f8fa9e00640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 0x7f8fac199fc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00e9f0 tx=0x7f8f9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:21.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.173+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00cd80 con 0x7f8fac073ad0 2024-03-21T09:23:21.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.173+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f9c010910 con 0x7f8fac073ad0 2024-03-21T09:23:21.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.173+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac19f250 con 0x7f8fac073ad0 2024-03-21T09:23:21.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.173+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c018990 con 0x7f8fac073ad0 2024-03-21T09:23:21.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.173+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac1a6ae0 con 0x7f8fac073ad0 2024-03-21T09:23:21.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.174+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac04f400 con 0x7f8fac073ad0 2024-03-21T09:23:21.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.176+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8f9c018b30 con 0x7f8fac073ad0 2024-03-21T09:23:21.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.177+0000 7f8fa2a00640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8f900a3090 0x7f8f900a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:21.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.177+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8f9c0cffa0 con 0x7f8fac073ad0 2024-03-21T09:23:21.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.177+0000 7f8fa9400640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8f900a3090 0x7f8f900a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:21.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.178+0000 7f8fa9400640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8f900a3090 0x7f8f900a5550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009230 tx=0x7f8f9802df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:21.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.181+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f9c09bfe0 con 0x7f8fac073ad0 2024-03-21T09:23:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.401+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8fac19b2c0 con 0x7f8fac073ad0 2024-03-21T09:23:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.402+0000 7f8fa2a00640 1 -- 172.21.15.37:0/1600367913 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v105) v1 ==== 81+0+1000 (secure 0 0 0) 0x7f8f9c0a0080 con 0x7f8fac073ad0 2024-03-21T09:23:21.399 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/295866901 2024-03-22T09:00:45.623535+0000 2024-03-21T09:23:21.399 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2468741411 2024-03-22T09:00:45.623535+0000 2024-03-21T09:23:21.399 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/276911571 2024-03-22T09:00:11.515361+0000 2024-03-21T09:23:21.399 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2204787263 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2644652589 2024-03-22T09:00:11.515361+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1173483074 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1907385341 2024-03-22T09:00:11.515361+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2805488546 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1300592626 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3643193292 2024-03-22T09:00:45.623535+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/276911571 2024-03-22T09:00:11.515361+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2055653180 2024-03-22T09:00:11.515361+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/798132403 2024-03-22T09:00:45.623535+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1300592626 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2468741411 2024-03-22T09:00:45.623535+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1005590151 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1759352225 2024-03-22T09:12:10.987337+0000 2024-03-21T09:23:21.400 INFO:teuthology.orchestra.run.smithi037.stderr:listed 17 entries 2024-03-21T09:23:21.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8f900a3090 msgr2=0x7f8f900a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:21.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8f900a3090 0x7f8f900a5550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009230 tx=0x7f8f9802df90 comp rx=0 tx=0).stop 2024-03-21T09:23:21.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 msgr2=0x7f8fac199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 0x7f8fac199fc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00e9f0 tx=0x7f8f9c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 shutdown_connections 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f8f900a3090 0x7f8f900a5550 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fac10dca0 0x7f8fac19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fac0743f0 0x7f8fac19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 --2- 172.21.15.37:0/1600367913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fac073ad0 0x7f8fac199fc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 >> 172.21.15.37:0/1600367913 conn(0x7f8fac0fcff0 msgr2=0x7f8fac0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 shutdown_connections 2024-03-21T09:23:21.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:21.405+0000 7f8fab200640 1 -- 172.21.15.37:0/1600367913 wait complete. 2024-03-21T09:23:21.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:21 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2942726398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:23:21.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:21 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2752360535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:23:21.420 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-21T09:23:21.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:23:21.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-21T09:23:21.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:21 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2942726398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:23:21.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:21 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2752360535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:23:21.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:23:21.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:23:21.543 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-21T09:23:21.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:21 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2942726398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:23:21.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:21 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2752360535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T09:23:21.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:23:21.619 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-21T09:23:21.655 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-21T09:23:21.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:23:21.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T09:23:21.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:23:21.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-03-21T09:23:21.877 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T09:23:21.888 INFO:tasks.cephadm:Teardown begin 2024-03-21T09:23:21.888 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-21T09:23:21.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:23:22.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 -- 172.21.15.37:0/872701020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 msgr2=0x7ff49c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 --2- 172.21.15.37:0/872701020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c1019c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff488009930 tx=0x7ff48802bc30 comp rx=0 tx=0).stop 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 -- 172.21.15.37:0/872701020 shutdown_connections 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 --2- 172.21.15.37:0/872701020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 0x7ff49c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 --2- 172.21.15.37:0/872701020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c1019c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 --2- 172.21.15.37:0/872701020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 0x7ff49c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.084+0000 7ff4a1600640 1 -- 172.21.15.37:0/872701020 >> 172.21.15.37:0/872701020 conn(0x7ff49c0fd050 msgr2=0x7ff49c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.085+0000 7ff4a1600640 1 -- 172.21.15.37:0/872701020 shutdown_connections 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.085+0000 7ff4a1600640 1 -- 172.21.15.37:0/872701020 wait complete. 2024-03-21T09:23:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.085+0000 7ff4a1600640 1 Processor -- start 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 -- start start 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 0x7ff49c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 0x7ff49c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c111910 con 0x7ff49c107600 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c111790 con 0x7ff49c101f00 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a1600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c111a90 con 0x7ff49c101540 2024-03-21T09:23:22.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff49be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 0x7ff49c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff49b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 0x7ff49c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 0x7ff49c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35738/0 (socket says 172.21.15.37:35738) 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.086+0000 7ff4a0c00640 1 -- 172.21.15.37:0/1805992766 learned_addr learned my addr 172.21.15.37:0/1805992766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49be00640 1 -- 172.21.15.37:0/1805992766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 msgr2=0x7ff49c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49be00640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 0x7ff49c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49be00640 1 -- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 msgr2=0x7ff49c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49be00640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 0x7ff49c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49be00640 1 -- 172.21.15.37:0/1805992766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c19efc0 con 0x7ff49c101540 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49b400640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 0x7ff49c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff49be00640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c199fd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e9f0 tx=0x7ff48c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00ccc0 con 0x7ff49c101540 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff48c010910 con 0x7ff49c101540 2024-03-21T09:23:22.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c19f2b0 con 0x7ff49c101540 2024-03-21T09:23:22.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c018960 con 0x7ff49c101540 2024-03-21T09:23:22.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.087+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c1a6b40 con 0x7ff49c101540 2024-03-21T09:23:22.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.088+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c04f470 con 0x7ff49c101540 2024-03-21T09:23:22.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.090+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff48c010ac0 con 0x7ff49c101540 2024-03-21T09:23:22.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.092+0000 7ff498c00640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4800a3080 0x7ff4800a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.092+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff48c0d04f0 con 0x7ff49c101540 2024-03-21T09:23:22.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.092+0000 7ff49b400640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4800a3080 0x7ff4800a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.093+0000 7ff49b400640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4800a3080 0x7ff4800a5540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff488009900 tx=0x7ff48802df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:22.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.098+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff48c09c530 con 0x7ff49c101540 2024-03-21T09:23:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.397+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff49c1a6d10 con 0x7ff49c101540 2024-03-21T09:23:22.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.398+0000 7ff498c00640 1 -- 172.21.15.37:0/1805992766 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7ff48c0a05d0 con 0x7ff49c101540 2024-03-21T09:23:22.396 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:23:22.396 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:23:22.397 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:23:22.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4800a3080 msgr2=0x7ff4800a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4800a3080 0x7ff4800a5540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff488009900 tx=0x7ff48802df90 comp rx=0 tx=0).stop 2024-03-21T09:23:22.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 msgr2=0x7ff49c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c199fd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e9f0 tx=0x7ff48c00eeb0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 shutdown_connections 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff49c107600 0x7ff49c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ff4800a3080 0x7ff4800a5540 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c101f00 0x7ff49c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 --2- 172.21.15.37:0/1805992766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49c101540 0x7ff49c199fd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 >> 172.21.15.37:0/1805992766 conn(0x7ff49c0fd050 msgr2=0x7ff49c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 shutdown_connections 2024-03-21T09:23:22.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.401+0000 7ff4a1600640 1 -- 172.21.15.37:0/1805992766 wait complete. 2024-03-21T09:23:22.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:23:22.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:22 smithi037 ceph-mon[31830]: pgmap v348: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:22.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:22 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1600367913' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:23:22.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:22 smithi053 ceph-mon[36169]: pgmap v348: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:22.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:22 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1600367913' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:23:22.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:22 smithi071 ceph-mon[36597]: pgmap v348: 129 pgs: 129 active+clean; 17 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-21T09:23:22.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:22 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1600367913' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 -- 172.21.15.37:0/1834484562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 msgr2=0x7ffaac10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 --2- 172.21.15.37:0/1834484562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac10c3f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009cd0 tx=0x7ffa9402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 -- 172.21.15.37:0/1834484562 shutdown_connections 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 --2- 172.21.15.37:0/1834484562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaac10c930 0x7ffaac118e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 --2- 172.21.15.37:0/1834484562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac10c3f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 --2- 172.21.15.37:0/1834484562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac111fd0 0x7ffaac1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 -- 172.21.15.37:0/1834484562 >> 172.21.15.37:0/1834484562 conn(0x7ffaac06c4f0 msgr2=0x7ffaac10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.632+0000 7ffab1200640 1 -- 172.21.15.37:0/1834484562 shutdown_connections 2024-03-21T09:23:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.633+0000 7ffab1200640 1 -- 172.21.15.37:0/1834484562 wait complete. 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.633+0000 7ffab1200640 1 Processor -- start 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 -- start start 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac1a8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaac10c930 0x7ffaac1a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac111fd0 0x7ffaac1a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac11c2f0 con 0x7ffaac111fd0 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac11c170 con 0x7ffaac10c930 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffab1200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac11c470 con 0x7ffaac10bfb0 2024-03-21T09:23:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac1a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac1a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42964/0 (socket says 172.21.15.37:42964) 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffaab400640 1 -- 172.21.15.37:0/4291085671 learned_addr learned my addr 172.21.15.37:0/4291085671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffaa2a00640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaac10c930 0x7ffaac1a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.634+0000 7ffaabe00640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac111fd0 0x7ffaac1a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaab400640 1 -- 172.21.15.37:0/4291085671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaac10c930 msgr2=0x7ffaac1a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaab400640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaac10c930 0x7ffaac1a8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaab400640 1 -- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac111fd0 msgr2=0x7ffaac1a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaab400640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac111fd0 0x7ffaac1a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaab400640 1 -- 172.21.15.37:0/4291085671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac1a2f10 con 0x7ffaac10bfb0 2024-03-21T09:23:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaab400640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac1a8590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00d950 tx=0x7ffa9c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c017030 con 0x7ffaac10bfb0 2024-03-21T09:23:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa9c004020 con 0x7ffaac10bfb0 2024-03-21T09:23:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac1a3200 con 0x7ffaac10bfb0 2024-03-21T09:23:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0047a0 con 0x7ffaac10bfb0 2024-03-21T09:23:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.635+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac1af510 con 0x7ffaac10bfb0 2024-03-21T09:23:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.637+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac10d670 con 0x7ffaac10bfb0 2024-03-21T09:23:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.639+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ffa9c0041d0 con 0x7ffaac10bfb0 2024-03-21T09:23:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.640+0000 7ffaa8c00640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffa900a3100 0x7ffa900a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.640+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ffa9c044e00 con 0x7ffaac10bfb0 2024-03-21T09:23:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.640+0000 7ffaa2a00640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffa900a3100 0x7ffa900a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.641+0000 7ffaa2a00640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffa900a3100 0x7ffa900a55c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa94004af0 tx=0x7ffa94005c50 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.646+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa9c09bd80 con 0x7ffaac10bfb0 2024-03-21T09:23:22.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.927+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaac04f470 con 0x7ffaac10bfb0 2024-03-21T09:23:22.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.928+0000 7ffaa8c00640 1 -- 172.21.15.37:0/4291085671 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11372 (secure 0 0 0) 0x7ffa9c09b680 con 0x7ffaac10bfb0 2024-03-21T09:23:22.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:23:22.926 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3418620449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3418620449},{"type":"v1","addr":"172.21.15.37:6835","nonce":3418620449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/495294644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":495294644},{"type":"v1","addr":"172.21.15.37:6837","nonce":495294644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/508705459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":508705459},{"type":"v1","addr":"172.21.15.37:6839","nonce":508705459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1353662837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1353662837},{"type":"v1","addr":"172.21.15.53:6833","nonce":1353662837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3927519167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3927519167},{"type":"v1","addr":"172.21.15.71:6833","nonce":3927519167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1616524011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1616524011},{"type":"v1","addr":"172.21.15.53:6835","nonce":1616524011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T09:08:46.306108+0000","modified":"2024-03-21T09:16:53.222130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24589,"mds_1":24575,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3665426355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3665426355},{"type":"v1","addr":"172.21.15.53:6837","nonce":3665426355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/2027751164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2027751164},{"type":"v1","addr":"172.21.15.71:6835","nonce":2027751164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6837/2290408711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2290408711},{"type":"v1","addr":"172.21.15.71:6837","nonce":2290408711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.71:6839/543433986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":543433986},{"type":"v1","addr":"172.21.15.71:6839","nonce":543433986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.53:6839/380096256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":380096256},{"type":"v1","addr":"172.21.15.53:6839","nonce":380096256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6841/600888190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":600888190},{"type":"v1","addr":"172.21.15.37:6841","nonce":600888190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24589,"qdb_cluster":[24553,24589,24575]},"id":1}]} 2024-03-21T09:23:22.927 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffa900a3100 msgr2=0x7ffa900a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffa900a3100 0x7ffa900a55c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa94004af0 tx=0x7ffa94005c50 comp rx=0 tx=0).stop 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 msgr2=0x7ffaac1a8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac1a8590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00d950 tx=0x7ffa9c00de10 comp rx=0 tx=0).stop 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 shutdown_connections 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac111fd0 0x7ffaac1a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7ffa900a3100 0x7ffa900a55c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaac10c930 0x7ffaac1a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 --2- 172.21.15.37:0/4291085671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaac10bfb0 0x7ffaac1a8590 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 >> 172.21.15.37:0/4291085671 conn(0x7ffaac06c4f0 msgr2=0x7ffaac10ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 shutdown_connections 2024-03-21T09:23:22.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:22.931+0000 7ffab1200640 1 -- 172.21.15.37:0/4291085671 wait complete. 2024-03-21T09:23:22.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:23:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.206+0000 7fb209e00640 1 -- 172.21.15.37:0/2912776736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204109770 msgr2=0x7fb204109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.206+0000 7fb209e00640 1 --2- 172.21.15.37:0/2912776736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204109770 0x7fb204109b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009900 tx=0x7fb1f802bc40 comp rx=0 tx=0).stop 2024-03-21T09:23:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 -- 172.21.15.37:0/2912776736 shutdown_connections 2024-03-21T09:23:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 --2- 172.21.15.37:0/2912776736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb204104120 0x7fb204110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 --2- 172.21.15.37:0/2912776736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb204103760 0x7fb204103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 --2- 172.21.15.37:0/2912776736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204109770 0x7fb204109b50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 -- 172.21.15.37:0/2912776736 >> 172.21.15.37:0/2912776736 conn(0x7fb2040ff1d0 msgr2=0x7fb2041015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 -- 172.21.15.37:0/2912776736 shutdown_connections 2024-03-21T09:23:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.207+0000 7fb209e00640 1 -- 172.21.15.37:0/2912776736 wait complete. 2024-03-21T09:23:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 Processor -- start 2024-03-21T09:23:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 -- start start 2024-03-21T09:23:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb204103760 0x7fb20419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 0x7fb20419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb204109770 0x7fb20419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204113bd0 con 0x7fb204103760 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204113a50 con 0x7fb204104120 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.208+0000 7fb209e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204113d50 con 0x7fb204109770 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 0x7fb20419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb208a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb204103760 0x7fb20419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 0x7fb20419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36798/0 (socket says 172.21.15.37:36798) 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb209400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb204109770 0x7fb20419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 -- 172.21.15.37:0/1714207318 learned_addr learned my addr 172.21.15.37:0/1714207318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 -- 172.21.15.37:0/1714207318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb204109770 msgr2=0x7fb20419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb204109770 0x7fb20419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 -- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb204103760 msgr2=0x7fb20419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb204103760 0x7fb20419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 -- 172.21.15.37:0/1714207318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20419f0a0 con 0x7fb204104120 2024-03-21T09:23:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.209+0000 7fb203e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 0x7fb20419a5f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000e9f0 tx=0x7fb1f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.210+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000cd80 con 0x7fb204104120 2024-03-21T09:23:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.210+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1f0010910 con 0x7fb204104120 2024-03-21T09:23:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.210+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00189d0 con 0x7fb204104120 2024-03-21T09:23:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.210+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20419f330 con 0x7fb204104120 2024-03-21T09:23:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.210+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2041a6c20 con 0x7fb204104120 2024-03-21T09:23:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.211+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20404f470 con 0x7fb204104120 2024-03-21T09:23:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.213+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb1f0010ac0 con 0x7fb204104120 2024-03-21T09:23:23.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.215+0000 7fb201600640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1ec0a3100 0x7fb1ec0a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.215+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fb1f00cff60 con 0x7fb204104120 2024-03-21T09:23:23.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.215+0000 7fb208a00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1ec0a3100 0x7fb1ec0a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.216+0000 7fb208a00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1ec0a3100 0x7fb1ec0a55c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8006010 tx=0x7fb1f8000bc0 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:23.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.218+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb1f009bfa0 con 0x7fb204104120 2024-03-21T09:23:23.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:23 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1805992766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:23:23.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:23 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/4291085671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:23:23.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:23 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1805992766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:23:23.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:23 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/4291085671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:23:23.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.459+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb20410bcd0 con 0x7fb204104120 2024-03-21T09:23:23.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.461+0000 7fb201600640 1 -- 172.21.15.37:0/1714207318 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25290 (secure 0 0 0) 0x7fb1f00bc0e0 con 0x7fb204104120 2024-03-21T09:23:23.460 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-21T09:23:23.460 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":105,"fsid":"2dc01c4e-e761-11ee-95cd-87774f69a715","created":"2024-03-21T08:59:29.362739+0000","modified":"2024-03-21T09:23:16.128305+0000","last_up_change":"2024-03-21T09:07:47.512128+0000","last_in_change":"2024-03-21T09:07:27.218401+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T09:03:30.335349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T09:08:44.251938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T09:08:45.778848+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T09:08:49.461084+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f8ca1c1e-a53f-43bb-b2d7-0df467be85b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6803","nonce":2693651580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6805","nonce":2693651580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6809","nonce":2693651580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2693651580},{"type":"v1","addr":"172.21.15.37:6807","nonce":2693651580}]},"public_addr":"172.21.15.37:6803/2693651580","cluster_addr":"172.21.15.37:6805/2693651580","heartbeat_back_addr":"172.21.15.37:6809/2693651580","heartbeat_front_addr":"172.21.15.37:6807/2693651580","state":["exists","up"]},{"osd":1,"uuid":"97aea20a-8dd3-4a5d-a5a5-341cd7508568","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6801","nonce":916967711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6803","nonce":916967711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6807","nonce":916967711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":916967711},{"type":"v1","addr":"172.21.15.53:6805","nonce":916967711}]},"public_addr":"172.21.15.53:6801/916967711","cluster_addr":"172.21.15.53:6803/916967711","heartbeat_back_addr":"172.21.15.53:6807/916967711","heartbeat_front_addr":"172.21.15.53:6805/916967711","state":["exists","up"]},{"osd":2,"uuid":"3d5b09e0-acc1-488c-8e3d-34c05699a51e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6801","nonce":1968250372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6803","nonce":1968250372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6807","nonce":1968250372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1968250372},{"type":"v1","addr":"172.21.15.71:6805","nonce":1968250372}]},"public_addr":"172.21.15.71:6801/1968250372","cluster_addr":"172.21.15.71:6803/1968250372","heartbeat_back_addr":"172.21.15.71:6807/1968250372","heartbeat_front_addr":"172.21.15.71:6805/1968250372","state":["exists","up"]},{"osd":3,"uuid":"879c9049-bfc3-41fa-be8e-354edcc68a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6811","nonce":3211293526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6813","nonce":3211293526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6817","nonce":3211293526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3211293526},{"type":"v1","addr":"172.21.15.37:6815","nonce":3211293526}]},"public_addr":"172.21.15.37:6811/3211293526","cluster_addr":"172.21.15.37:6813/3211293526","heartbeat_back_addr":"172.21.15.37:6817/3211293526","heartbeat_front_addr":"172.21.15.37:6815/3211293526","state":["exists","up"]},{"osd":4,"uuid":"721729f1-98cf-4ce4-b0e6-3a0d9736d2d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6809","nonce":1561544362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6811","nonce":1561544362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6815","nonce":1561544362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1561544362},{"type":"v1","addr":"172.21.15.53:6813","nonce":1561544362}]},"public_addr":"172.21.15.53:6809/1561544362","cluster_addr":"172.21.15.53:6811/1561544362","heartbeat_back_addr":"172.21.15.53:6815/1561544362","heartbeat_front_addr":"172.21.15.53:6813/1561544362","state":["exists","up"]},{"osd":5,"uuid":"ee6f4f1f-25a9-4e57-a211-178500d9cf88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6809","nonce":2412287007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6811","nonce":2412287007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6815","nonce":2412287007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2412287007},{"type":"v1","addr":"172.21.15.71:6813","nonce":2412287007}]},"public_addr":"172.21.15.71:6809/2412287007","cluster_addr":"172.21.15.71:6811/2412287007","heartbeat_back_addr":"172.21.15.71:6815/2412287007","heartbeat_front_addr":"172.21.15.71:6813/2412287007","state":["exists","up"]},{"osd":6,"uuid":"6de23659-fb28-4c49-988a-84c26500a853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6819","nonce":1203908785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6821","nonce":1203908785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6825","nonce":1203908785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1203908785},{"type":"v1","addr":"172.21.15.37:6823","nonce":1203908785}]},"public_addr":"172.21.15.37:6819/1203908785","cluster_addr":"172.21.15.37:6821/1203908785","heartbeat_back_addr":"172.21.15.37:6825/1203908785","heartbeat_front_addr":"172.21.15.37:6823/1203908785","state":["exists","up"]},{"osd":7,"uuid":"344d4a58-75bc-4741-b385-e590feb5fcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6817","nonce":884972742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6819","nonce":884972742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6823","nonce":884972742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":884972742},{"type":"v1","addr":"172.21.15.53:6821","nonce":884972742}]},"public_addr":"172.21.15.53:6817/884972742","cluster_addr":"172.21.15.53:6819/884972742","heartbeat_back_addr":"172.21.15.53:6823/884972742","heartbeat_front_addr":"172.21.15.53:6821/884972742","state":["exists","up"]},{"osd":8,"uuid":"e2d7e521-1571-45c7-98d2-3a7b3e876488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6817","nonce":4216652236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6819","nonce":4216652236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6823","nonce":4216652236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4216652236},{"type":"v1","addr":"172.21.15.71:6821","nonce":4216652236}]},"public_addr":"172.21.15.71:6817/4216652236","cluster_addr":"172.21.15.71:6819/4216652236","heartbeat_back_addr":"172.21.15.71:6823/4216652236","heartbeat_front_addr":"172.21.15.71:6821/4216652236","state":["exists","up"]},{"osd":9,"uuid":"c7ee448c-645a-44ee-aa9d-f8e9b5898ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6827","nonce":3963720627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6829","nonce":3963720627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6833","nonce":3963720627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3963720627},{"type":"v1","addr":"172.21.15.37:6831","nonce":3963720627}]},"public_addr":"172.21.15.37:6827/3963720627","cluster_addr":"172.21.15.37:6829/3963720627","heartbeat_back_addr":"172.21.15.37:6833/3963720627","heartbeat_front_addr":"172.21.15.37:6831/3963720627","state":["exists","up"]},{"osd":10,"uuid":"a4c10007-9a20-4448-981e-e97b3650beca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6825","nonce":3560446684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6827","nonce":3560446684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6831","nonce":3560446684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3560446684},{"type":"v1","addr":"172.21.15.53:6829","nonce":3560446684}]},"public_addr":"172.21.15.53:6825/3560446684","cluster_addr":"172.21.15.53:6827/3560446684","heartbeat_back_addr":"172.21.15.53:6831/3560446684","heartbeat_front_addr":"172.21.15.53:6829/3560446684","state":["exists","up"]},{"osd":11,"uuid":"ae4df7bc-7bbd-421b-8031-0599545220d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6825","nonce":3952075855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6827","nonce":3952075855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6831","nonce":3952075855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3952075855},{"type":"v1","addr":"172.21.15.71:6829","nonce":3952075855}]},"public_addr":"172.21.15.71:6825/3952075855","cluster_addr":"172.21.15.71:6827/3952075855","heartbeat_back_addr":"172.21.15.71:6831/3952075855","heartbeat_front_addr":"172.21.15.71:6829/3952075855","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:02:37.930344+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:03.803738+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:27.102846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:03:59.844089+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:26.675811+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:04:51.036084+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:25.324964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:05:52.799279+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:17.018111+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:06:52.274696+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:19.765547+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T09:07:45.384515+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/295866901":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6800/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2204787263":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/2644652589":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/1173483074":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/1907385341":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2805488546":"2024-03-22T09:12:10.987337+0000","172.21.15.37:6800/1300592626":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/3643193292":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6801/276911571":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/2055653180":"2024-03-22T09:00:11.515361+0000","172.21.15.37:0/798132403":"2024-03-22T09:00:45.623535+0000","172.21.15.37:6801/1300592626":"2024-03-22T09:12:10.987337+0000","172.21.15.37:6801/2468741411":"2024-03-22T09:00:45.623535+0000","172.21.15.37:0/1005590151":"2024-03-22T09:12:10.987337+0000","172.21.15.37:0/1759352225":"2024-03-22T09:12:10.987337+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1ec0a3100 msgr2=0x7fb1ec0a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1ec0a3100 0x7fb1ec0a55c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8006010 tx=0x7fb1f8000bc0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 msgr2=0x7fb20419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 0x7fb20419a5f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000e9f0 tx=0x7fb1f000eeb0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 shutdown_connections 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb204109770 0x7fb20419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1ec0a3100 0x7fb1ec0a55c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb204104120 0x7fb20419a5f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 --2- 172.21.15.37:0/1714207318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb204103760 0x7fb20419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.464+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 >> 172.21.15.37:0/1714207318 conn(0x7fb2040ff1d0 msgr2=0x7fb2040ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.465+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 shutdown_connections 2024-03-21T09:23:23.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.465+0000 7fb209e00640 1 -- 172.21.15.37:0/1714207318 wait complete. 2024-03-21T09:23:23.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-21T09:23:23.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:23 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1805992766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:23:23.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:23 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/4291085671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:23:23.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.698+0000 7fcb88a00640 1 -- 172.21.15.37:0/2903539007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84073a70 msgr2=0x7fcb84073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.698+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2903539007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84073a70 0x7fcb84073e50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009900 tx=0x7fcb7402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 -- 172.21.15.37:0/2903539007 shutdown_connections 2024-03-21T09:23:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2903539007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 0x7fcb84110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2903539007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84074390 0x7fcb8410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2903539007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84073a70 0x7fcb84073e50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 -- 172.21.15.37:0/2903539007 >> 172.21.15.37:0/2903539007 conn(0x7fcb840fcfb0 msgr2=0x7fcb840ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 -- 172.21.15.37:0/2903539007 shutdown_connections 2024-03-21T09:23:23.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.699+0000 7fcb88a00640 1 -- 172.21.15.37:0/2903539007 wait complete. 2024-03-21T09:23:23.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 Processor -- start 2024-03-21T09:23:23.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 -- start start 2024-03-21T09:23:23.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 0x7fcb8419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84074390 0x7fcb8419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 0x7fcb8419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84113af0 con 0x7fcb8410dd20 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84113970 con 0x7fcb84073a70 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.700+0000 7fcb88a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84113c70 con 0x7fcb84074390 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 0x7fcb8419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 0x7fcb8419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:36818/0 (socket says 172.21.15.37:36818) 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 -- 172.21.15.37:0/2816655299 learned_addr learned my addr 172.21.15.37:0/2816655299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb82a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84074390 0x7fcb8419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83e00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 0x7fcb8419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 -- 172.21.15.37:0/2816655299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84074390 msgr2=0x7fcb8419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84074390 0x7fcb8419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 -- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 msgr2=0x7fcb8419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 0x7fcb8419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 -- 172.21.15.37:0/2816655299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8419efd0 con 0x7fcb84073a70 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83e00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 0x7fcb8419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:23:23.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.701+0000 7fcb83400640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 0x7fcb8419a030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000c00 tx=0x7fcb740043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.702+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74042020 con 0x7fcb84073a70 2024-03-21T09:23:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.702+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb74005000 con 0x7fcb84073a70 2024-03-21T09:23:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.702+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74003b20 con 0x7fcb84073a70 2024-03-21T09:23:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.702+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8419f260 con 0x7fcb84073a70 2024-03-21T09:23:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.702+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb841a6b50 con 0x7fcb84073a70 2024-03-21T09:23:23.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.703+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8404f400 con 0x7fcb84073a70 2024-03-21T09:23:23.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.705+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcb74045020 con 0x7fcb84073a70 2024-03-21T09:23:23.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.706+0000 7fcb7be00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb680a3090 0x7fcb680a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:23.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.706+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcb740f1cd0 con 0x7fcb84073a70 2024-03-21T09:23:23.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.707+0000 7fcb82a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb680a3090 0x7fcb680a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:23.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.707+0000 7fcb82a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb680a3090 0x7fcb680a5550 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb8410f060 tx=0x7fcb70005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:23.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.708+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb7404c020 con 0x7fcb84073a70 2024-03-21T09:23:23.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.970+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcb84108ea0 con 0x7fcb84073a70 2024-03-21T09:23:23.978 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-21T09:23:23.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.981+0000 7fcb7be00640 1 -- 172.21.15.37:0/2816655299 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcb740be470 con 0x7fcb84073a70 2024-03-21T09:23:23.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.984+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb680a3090 msgr2=0x7fcb680a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb680a3090 0x7fcb680a5550 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb8410f060 tx=0x7fcb70005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:23.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 msgr2=0x7fcb8419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:23.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 0x7fcb8419a030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000c00 tx=0x7fcb740043c0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 shutdown_connections 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb8410dd20 0x7fcb8419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb84074390 0x7fcb8419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fcb680a3090 0x7fcb680a5550 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 --2- 172.21.15.37:0/2816655299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb84073a70 0x7fcb8419a030 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000c00 tx=0x7fcb740043c0 comp rx=0 tx=0).stop 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 >> 172.21.15.37:0/2816655299 conn(0x7fcb840fcfb0 msgr2=0x7fcb840fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 shutdown_connections 2024-03-21T09:23:23.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:23.985+0000 7fcb88a00640 1 -- 172.21.15.37:0/2816655299 wait complete. 2024-03-21T09:23:24.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-21T09:23:24.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 -- 172.21.15.37:0/943716523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1828075bd0 msgr2=0x7f1828075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 --2- 172.21.15.37:0/943716523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1828075bd0 0x7f1828075fb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f181c009930 tx=0x7f181c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 -- 172.21.15.37:0/943716523 shutdown_connections 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 --2- 172.21.15.37:0/943716523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18281124e0 0x7f18281148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 --2- 172.21.15.37:0/943716523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18280764f0 0x7f1828111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 --2- 172.21.15.37:0/943716523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1828075bd0 0x7f1828075fb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 -- 172.21.15.37:0/943716523 >> 172.21.15.37:0/943716523 conn(0x7f18280ff310 msgr2=0x7f1828101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 -- 172.21.15.37:0/943716523 shutdown_connections 2024-03-21T09:23:24.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.212+0000 7f1830400640 1 -- 172.21.15.37:0/943716523 wait complete. 2024-03-21T09:23:24.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.213+0000 7f1830400640 1 Processor -- start 2024-03-21T09:23:24.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.213+0000 7f1830400640 1 -- start start 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f1830400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 0x7f182819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f1830400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18280764f0 0x7f182819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f1830400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18281124e0 0x7f18281a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f1830400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828118090 con 0x7f1828075bd0 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f1830400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828117f10 con 0x7f18281124e0 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f1830400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828118210 con 0x7f18280764f0 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f182da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 0x7f182819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f182da00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 0x7f182819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57166/0 (socket says 172.21.15.37:57166) 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f182da00640 1 -- 172.21.15.37:0/1811868412 learned_addr learned my addr 172.21.15.37:0/1811868412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f182e400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18281124e0 0x7f18281a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.214+0000 7f182d000640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18280764f0 0x7f182819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182e400640 1 -- 172.21.15.37:0/1811868412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18280764f0 msgr2=0x7f182819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182e400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18280764f0 0x7f182819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182e400640 1 -- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 msgr2=0x7f182819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182e400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 0x7f182819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182e400640 1 -- 172.21.15.37:0/1811868412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18281a3580 con 0x7f18281124e0 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182d000640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18280764f0 0x7f182819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182da00640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 0x7f182819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f182e400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18281124e0 0x7f18281a2e50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1820005fd0 tx=0x7f182000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182000cdf0 con 0x7f18281124e0 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1820005020 con 0x7f18281124e0 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18200106d0 con 0x7f18281124e0 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18281a3870 con 0x7f18281124e0 2024-03-21T09:23:24.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.215+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18281ab100 con 0x7f18281124e0 2024-03-21T09:23:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.216+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828068520 con 0x7f18281124e0 2024-03-21T09:23:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.218+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f18200054d0 con 0x7f18281124e0 2024-03-21T09:23:24.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.219+0000 7f1826000640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f18100a3110 0x7f18100a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:24.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.220+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9709+0+0 (secure 0 0 0) 0x7f18200d02d0 con 0x7f18281124e0 2024-03-21T09:23:24.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.220+0000 7f182da00640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f18100a3110 0x7f18100a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:24.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.221+0000 7f182da00640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f18100a3110 0x7f18100a55d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f181c0096c0 tx=0x7f181c02df90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:24.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.224+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f182009c8c0 con 0x7f18281124e0 2024-03-21T09:23:24.358 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i[67673]: ignoring --setuser ceph since I am not root 2024-03-21T09:23:24.358 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i[67673]: ignoring --setgroup ceph since I am not root 2024-03-21T09:23:24.358 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mds[67677]: -- 172.21.15.71:0/4131733332 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf49e07a40 con 0x55cf49ddac00 2024-03-21T09:23:24.358 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i[67673]: starting mds.i at 2024-03-21T09:23:24.359 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c[67367]: ignoring --setuser ceph since I am not root 2024-03-21T09:23:24.359 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c[67367]: ignoring --setgroup ceph since I am not root 2024-03-21T09:23:24.359 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mds[67371]: -- 172.21.15.71:0/1543061059 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b54393a40 con 0x555b54366c00 2024-03-21T09:23:24.359 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c[67367]: starting mds.c at 2024-03-21T09:23:24.359 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f[67056]: ignoring --setuser ceph since I am not root 2024-03-21T09:23:24.359 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f[67056]: ignoring --setgroup ceph since I am not root 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mds[67060]: -- 172.21.15.71:0/1137967102 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558eb6e604e0 con 0x558eb6e36800 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mds[67060]: -- 172.21.15.71:0/1137967102 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558eb6e63a40 con 0x558eb6e36800 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f[67056]: starting mds.f at 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: pgmap v349: 129 pgs: 129 active+clean; 17 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1714207318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/2816655299' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T09:23:24.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:24 smithi071 ceph-mon[36597]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: ignoring --setuser ceph since I am not root 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: ignoring --setgroup ceph since I am not root 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mds[70649]: -- 172.21.15.37:0/1658509131 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55666c52fa40 con 0x55666c503000 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: starting mds.a at 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:23 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a[31826]: 2024-03-21T09:23:23.973+0000 7f9d2d800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: pgmap v349: 129 pgs: 129 active+clean; 17 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1714207318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/2816655299' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:23:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T09:23:24.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:23:24.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T09:23:24.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T09:23:24.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:24 smithi037 ceph-mon[31830]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-21T09:23:24.445 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b[66840]: ignoring --setuser ceph since I am not root 2024-03-21T09:23:24.445 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b[66840]: ignoring --setgroup ceph since I am not root 2024-03-21T09:23:24.445 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mds[66845]: -- 172.21.15.53:0/2608981847 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56015fe59a40 con 0x56015fe2c800 2024-03-21T09:23:24.445 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b[66840]: starting mds.b at 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h[66530]: ignoring --setuser ceph since I am not root 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h[66530]: ignoring --setgroup ceph since I am not root 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mds[66534]: -- 172.21.15.53:0/3065549345 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55636e883a40 con 0x55636e857000 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h[66530]: starting mds.h at 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: pgmap v349: 129 pgs: 129 active+clean; 17 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1714207318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/2816655299' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T09:23:24.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:23:24.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T09:23:24.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T09:23:24.447 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:24 smithi053 ceph-mon[36169]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-21T09:23:24.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.473+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f182810d0e0 con 0x7f18281124e0 2024-03-21T09:23:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.984+0000 7f1826000640 1 -- 172.21.15.37:0/1811868412 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f182009c1c0 con 0x7f18281124e0 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f18100a3110 msgr2=0x7f18100a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f18100a3110 0x7f18100a55d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f181c0096c0 tx=0x7f181c02df90 comp rx=0 tx=0).stop 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18281124e0 msgr2=0x7f18281a2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18281124e0 0x7f18281a2e50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1820005fd0 tx=0x7f182000ef10 comp rx=0 tx=0).stop 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 shutdown_connections 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f18100a3110 0x7f18100a55d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18281124e0 0x7f18281a2e50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18280764f0 0x7f182819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 --2- 172.21.15.37:0/1811868412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828075bd0 0x7f182819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 >> 172.21.15.37:0/1811868412 conn(0x7f18280ff310 msgr2=0x7f1828101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.988+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 shutdown_connections 2024-03-21T09:23:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:24.989+0000 7f1830400640 1 -- 172.21.15.37:0/1811868412 wait complete. 2024-03-21T09:23:25.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-21T09:23:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 -- 172.21.15.37:0/3511168289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d8107aa0 msgr2=0x7f82d8107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 --2- 172.21.15.37:0/3511168289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d8107aa0 0x7f82d8107f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402bcc0 comp rx=0 tx=0).stop 2024-03-21T09:23:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 -- 172.21.15.37:0/3511168289 shutdown_connections 2024-03-21T09:23:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 --2- 172.21.15.37:0/3511168289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d8108440 0x7f82d8114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 --2- 172.21.15.37:0/3511168289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d8107aa0 0x7f82d8107f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 --2- 172.21.15.37:0/3511168289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d810c980 0x7f82d810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.237+0000 7f82de800640 1 -- 172.21.15.37:0/3511168289 >> 172.21.15.37:0/3511168289 conn(0x7f82d80772a0 msgr2=0x7f82d80796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.238+0000 7f82de800640 1 -- 172.21.15.37:0/3511168289 shutdown_connections 2024-03-21T09:23:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.238+0000 7f82de800640 1 -- 172.21.15.37:0/3511168289 wait complete. 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 Processor -- start 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 -- start start 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 0x7f82d819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d8108440 0x7f82d819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 0x7f82d81a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8117ec0 con 0x7f82d810c980 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8117d40 con 0x7f82d8108440 2024-03-21T09:23:25.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82de800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8118040 con 0x7f82d8107aa0 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82dde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 0x7f82d81a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.239+0000 7f82dd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 0x7f82d819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 0x7f82d819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38872/0 (socket says 172.21.15.37:38872) 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 -- 172.21.15.37:0/919626442 learned_addr learned my addr 172.21.15.37:0/919626442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 0x7f82d81a2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57178/0 (socket says 172.21.15.37:57178) 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dca00640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d8108440 0x7f82d819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 -- 172.21.15.37:0/919626442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d8108440 msgr2=0x7f82d819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d8108440 0x7f82d819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 -- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 msgr2=0x7f82d81a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 0x7f82d81a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 -- 172.21.15.37:0/919626442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d81a3370 con 0x7f82d8107aa0 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dde00640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 0x7f82d81a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.240+0000 7f82dd400640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 0x7f82d819e380 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00e9f0 tx=0x7f82cc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.241+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc00ccc0 con 0x7f82d8107aa0 2024-03-21T09:23:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.241+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d81a3660 con 0x7f82d8107aa0 2024-03-21T09:23:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.241+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82cc010910 con 0x7f82d8107aa0 2024-03-21T09:23:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.241+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0188a0 con 0x7f82d8107aa0 2024-03-21T09:23:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.241+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d81aaef0 con 0x7f82d8107aa0 2024-03-21T09:23:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.242+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d804f400 con 0x7f82d8107aa0 2024-03-21T09:23:25.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.244+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f82cc010ac0 con 0x7f82d8107aa0 2024-03-21T09:23:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.245+0000 7f82d6000640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f82c00a3090 0x7f82c00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.245+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9709+0+0 (secure 0 0 0) 0x7f82cc0d0090 con 0x7f82d8107aa0 2024-03-21T09:23:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.245+0000 7f82dca00640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f82c00a3090 0x7f82c00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:25.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.246+0000 7f82dca00640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f82c00a3090 0x7f82c00a5550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82c4005ab0 tx=0x7f82c4005a20 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.250+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82cc09c680 con 0x7f82d8107aa0 2024-03-21T09:23:25.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1811868412' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: Cluster is now healthy 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6836/950046127,v1:172.21.15.71:6839/950046127] up:boot 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6836/3245012692,v1:172.21.15.53:6837/3245012692] up:boot 2024-03-21T09:23:25.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6834/683381588,v1:172.21.15.71:6837/683381588] up:boot 2024-03-21T09:23:25.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.71:6835/988128170,v1:172.21.15.71:6838/988128170] up:boot 2024-03-21T09:23:25.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.53:6838/1223627498,v1:172.21.15.53:6839/1223627498] up:boot 2024-03-21T09:23:25.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: mds.? [v2:172.21.15.37:6840/2058534324,v1:172.21.15.37:6841/2058534324] up:boot 2024-03-21T09:23:25.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:25 smithi037 ceph-mon[31830]: fsmap 12 up:standby 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1811868412' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: Cluster is now healthy 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:23:25.445 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6836/950046127,v1:172.21.15.71:6839/950046127] up:boot 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6836/3245012692,v1:172.21.15.53:6837/3245012692] up:boot 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6834/683381588,v1:172.21.15.71:6837/683381588] up:boot 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.71:6835/988128170,v1:172.21.15.71:6838/988128170] up:boot 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.53:6838/1223627498,v1:172.21.15.53:6839/1223627498] up:boot 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: mds.? [v2:172.21.15.37:6840/2058534324,v1:172.21.15.37:6841/2058534324] up:boot 2024-03-21T09:23:25.446 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:25 smithi053 ceph-mon[36169]: fsmap 12 up:standby 2024-03-21T09:23:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:25.459+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f82d810ff50 con 0x7f82d8107aa0 2024-03-21T09:23:25.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1811868412' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: Cluster is now healthy 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: from='mgr.25553 172.21.15.37:0/3540272929' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6836/950046127,v1:172.21.15.71:6839/950046127] up:boot 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6836/3245012692,v1:172.21.15.53:6837/3245012692] up:boot 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6834/683381588,v1:172.21.15.71:6837/683381588] up:boot 2024-03-21T09:23:25.607 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.71:6835/988128170,v1:172.21.15.71:6838/988128170] up:boot 2024-03-21T09:23:25.608 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.53:6838/1223627498,v1:172.21.15.53:6839/1223627498] up:boot 2024-03-21T09:23:25.608 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: mds.? [v2:172.21.15.37:6840/2058534324,v1:172.21.15.37:6841/2058534324] up:boot 2024-03-21T09:23:25.608 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:25 smithi071 ceph-mon[36597]: fsmap 12 up:standby 2024-03-21T09:23:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.205+0000 7f82d6000640 1 -- 172.21.15.37:0/919626442 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7f82cc09bf80 con 0x7f82d8107aa0 2024-03-21T09:23:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f82c00a3090 msgr2=0x7f82c00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f82c00a3090 0x7f82c00a5550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82c4005ab0 tx=0x7f82c4005a20 comp rx=0 tx=0).stop 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 msgr2=0x7f82d819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 0x7f82d819e380 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00e9f0 tx=0x7f82cc00edf0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 shutdown_connections 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d810c980 0x7f82d81a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f82c00a3090 0x7f82c00a5550 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d8108440 0x7f82d819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 --2- 172.21.15.37:0/919626442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d8107aa0 0x7f82d819e380 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.210+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 >> 172.21.15.37:0/919626442 conn(0x7f82d80772a0 msgr2=0x7f82d8077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.211+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 shutdown_connections 2024-03-21T09:23:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.212+0000 7f82de800640 1 -- 172.21.15.37:0/919626442 wait complete. 2024-03-21T09:23:26.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-21T09:23:26.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 -- 172.21.15.37:0/4143708956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 msgr2=0x7f5fd0110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:26.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 --2- 172.21.15.37:0/4143708956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd0110850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009ca0 tx=0x7f5fc802c1d0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 -- 172.21.15.37:0/4143708956 shutdown_connections 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 --2- 172.21.15.37:0/4143708956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd0110850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 --2- 172.21.15.37:0/4143708956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0105380 0x7f5fd01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 --2- 172.21.15.37:0/4143708956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 0x7f5fd0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 -- 172.21.15.37:0/4143708956 >> 172.21.15.37:0/4143708956 conn(0x7f5fd00ff350 msgr2=0x7f5fd0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 -- 172.21.15.37:0/4143708956 shutdown_connections 2024-03-21T09:23:26.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.449+0000 7f5fd7000640 1 -- 172.21.15.37:0/4143708956 wait complete. 2024-03-21T09:23:26.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 Processor -- start 2024-03-21T09:23:26.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 -- start start 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 0x7f5fd019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0105380 0x7f5fd019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0113d90 con 0x7f5fd0105d20 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0113c10 con 0x7f5fd0105380 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.450+0000 7f5fd7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0113f10 con 0x7f5fd0103c40 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd019eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57192/0 (socket says 172.21.15.37:57192) 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 -- 172.21.15.37:0/1484211019 learned_addr learned my addr 172.21.15.37:0/1484211019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fcf400640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0105380 0x7f5fd019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fcfe00640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 0x7f5fd019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 -- 172.21.15.37:0/1484211019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 msgr2=0x7f5fd019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 0x7f5fd019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 -- 172.21.15.37:0/1484211019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0105380 msgr2=0x7f5fd019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0105380 0x7f5fd019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 -- 172.21.15.37:0/1484211019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd019f1d0 con 0x7f5fd0105d20 2024-03-21T09:23:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fcfe00640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 0x7f5fd019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:23:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.451+0000 7f5fd5000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd019eaa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fc802c7e0 tx=0x7f5fc8034740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.452+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8039050 con 0x7f5fd0105d20 2024-03-21T09:23:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.452+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd019f460 con 0x7f5fd0105d20 2024-03-21T09:23:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.452+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8045070 con 0x7f5fd0105d20 2024-03-21T09:23:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.452+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8032ac0 con 0x7f5fd0105d20 2024-03-21T09:23:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.452+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd01a6d50 con 0x7f5fd0105d20 2024-03-21T09:23:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.453+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd0068970 con 0x7f5fd0105d20 2024-03-21T09:23:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.455+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5fc80045b0 con 0x7f5fd0105d20 2024-03-21T09:23:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.457+0000 7f5fccc00640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb40a3090 0x7f5fb40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.457+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9213+0+0 (secure 0 0 0) 0x7f5fc80f2980 con 0x7f5fd0105d20 2024-03-21T09:23:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.457+0000 7f5fcfe00640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb40a3090 0x7f5fb40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.458+0000 7f5fcfe00640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb40a3090 0x7f5fb40a5550 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009b50 tx=0x7f5fc0005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.460+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fc80bf390 con 0x7f5fd0105d20 2024-03-21T09:23:26.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:26 smithi071 ceph-mon[36597]: pgmap v351: 129 pgs: 129 active+clean; 17 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 888 B/s wr, 7 op/s 2024-03-21T09:23:26.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:26 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/919626442' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:26.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:26 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:26.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:26.661+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fd010bea0 con 0x7f5fd0105d20 2024-03-21T09:23:26.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:26 smithi037 ceph-mon[31830]: pgmap v351: 129 pgs: 129 active+clean; 17 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 888 B/s wr, 7 op/s 2024-03-21T09:23:26.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:26 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/919626442' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:26.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:26 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:26.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:26 smithi053 ceph-mon[36169]: pgmap v351: 129 pgs: 129 active+clean; 17 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 888 B/s wr, 7 op/s 2024-03-21T09:23:26.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:26 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/919626442' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:26.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:26 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.213+0000 7f5fccc00640 1 -- 172.21.15.37:0/1484211019 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5fc80bec90 con 0x7f5fd0105d20 2024-03-21T09:23:27.211 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-03-21T09:23:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.220+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb40a3090 msgr2=0x7f5fb40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.220+0000 7f5fd7000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb40a3090 0x7f5fb40a5550 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009b50 tx=0x7f5fc0005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.220+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 msgr2=0x7f5fd019eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.220+0000 7f5fd7000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd019eaa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fc802c7e0 tx=0x7f5fc8034740 comp rx=0 tx=0).stop 2024-03-21T09:23:27.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.221+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 shutdown_connections 2024-03-21T09:23:27.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.221+0000 7f5fd7000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fd0105d20 0x7f5fd019eaa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.221+0000 7f5fd7000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7f5fb40a3090 0x7f5fb40a5550 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.221+0000 7f5fd7000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd0105380 0x7f5fd019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.221+0000 7f5fd7000640 1 --2- 172.21.15.37:0/1484211019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fd0103c40 0x7f5fd019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.221+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 >> 172.21.15.37:0/1484211019 conn(0x7f5fd00ff350 msgr2=0x7f5fd0109760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:27.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.222+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 shutdown_connections 2024-03-21T09:23:27.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.222+0000 7f5fd7000640 1 -- 172.21.15.37:0/1484211019 wait complete. 2024-03-21T09:23:27.243 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-21T09:23:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 -- 172.21.15.37:0/2592011671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c104200 msgr2=0x7fb19c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/2592011671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c104200 0x7fb19c110710 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb194009cd0 tx=0x7fb19402c0f0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 -- 172.21.15.37:0/2592011671 shutdown_connections 2024-03-21T09:23:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/2592011671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c104200 0x7fb19c110710 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/2592011671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c103840 0x7fb19c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/2592011671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c109850 0x7fb19c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 -- 172.21.15.37:0/2592011671 >> 172.21.15.37:0/2592011671 conn(0x7fb19c0ff310 msgr2=0x7fb19c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 -- 172.21.15.37:0/2592011671 shutdown_connections 2024-03-21T09:23:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.475+0000 7fb1a3c00640 1 -- 172.21.15.37:0/2592011671 wait complete. 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.476+0000 7fb1a3c00640 1 Processor -- start 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 -- start start 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c103840 0x7fb19c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c104200 0x7fb19c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 0x7fb19c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c113c20 con 0x7fb19c103840 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c113aa0 con 0x7fb19c104200 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a3c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c113da0 con 0x7fb19c109850 2024-03-21T09:23:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c103840 0x7fb19c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 0x7fb19c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb19be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c104200 0x7fb19c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 0x7fb19c19e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38910/0 (socket says 172.21.15.37:38910) 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb19be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c104200 0x7fb19c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.37:37148/0 (socket says 172.21.15.37:37148) 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.477+0000 7fb1a1c00640 1 -- 172.21.15.37:0/474093434 learned_addr learned my addr 172.21.15.37:0/474093434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a1c00640 1 -- 172.21.15.37:0/474093434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c104200 msgr2=0x7fb19c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a1c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c104200 0x7fb19c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a1c00640 1 -- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c103840 msgr2=0x7fb19c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a1c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c103840 0x7fb19c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a1c00640 1 -- 172.21.15.37:0/474093434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c19f0f0 con 0x7fb19c109850 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a1c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 0x7fb19c19e9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb194005e30 tx=0x7fb194032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194041020 con 0x7fb19c109850 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb194034cd0 con 0x7fb19c109850 2024-03-21T09:23:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.478+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c19f380 con 0x7fb19c109850 2024-03-21T09:23:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.479+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194039040 con 0x7fb19c109850 2024-03-21T09:23:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.479+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1a6c70 con 0x7fb19c109850 2024-03-21T09:23:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.480+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c068520 con 0x7fb19c109850 2024-03-21T09:23:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.482+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb194044020 con 0x7fb19c109850 2024-03-21T09:23:27.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.483+0000 7fb199600640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1840a3090 0x7fb1840a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:23:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.484+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8625+0+0 (secure 0 0 0) 0x7fb1940f13e0 con 0x7fb19c109850 2024-03-21T09:23:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.484+0000 7fb1a1200640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1840a3090 0x7fb1840a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:23:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.484+0000 7fb1a1200640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1840a3090 0x7fb1840a5550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009b50 tx=0x7fb18c005f90 comp rx=0 tx=0).ready entity=mgr.25553 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:23:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.488+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb19404b020 con 0x7fb19c109850 2024-03-21T09:23:27.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:27 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:27.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:27 smithi071 ceph-mon[36597]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T09:23:27.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:27 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1484211019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:27.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:27 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:27.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:27 smithi037 ceph-mon[31830]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T09:23:27.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:27 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1484211019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:27.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:27 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:27.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:27 smithi053 ceph-mon[36169]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T09:23:27.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:27 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1484211019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:27.715+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb19c10be00 con 0x7fb19c109850 2024-03-21T09:23:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.220+0000 7fb199600640 1 -- 172.21.15.37:0/474093434 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb1940bdf10 con 0x7fb19c109850 2024-03-21T09:23:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' removed 2024-03-21T09:23:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1840a3090 msgr2=0x7fb1840a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1840a3090 0x7fb1840a5550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009b50 tx=0x7fb18c005f90 comp rx=0 tx=0).stop 2024-03-21T09:23:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 msgr2=0x7fb19c19e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:23:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 0x7fb19c19e9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb194005e30 tx=0x7fb194032ea0 comp rx=0 tx=0).stop 2024-03-21T09:23:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 shutdown_connections 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb19c109850 0x7fb19c19e9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:6800/1212038665,v1:172.21.15.37:6801/1212038665] conn(0x7fb1840a3090 0x7fb1840a5550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.226+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb19c104200 0x7fb19c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.227+0000 7fb1a3c00640 1 --2- 172.21.15.37:0/474093434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c103840 0x7fb19c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.227+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 >> 172.21.15.37:0/474093434 conn(0x7fb19c0ff310 msgr2=0x7fb19c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.227+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 shutdown_connections 2024-03-21T09:23:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-21T09:23:28.227+0000 7fb1a3c00640 1 -- 172.21.15.37:0/474093434 wait complete. 2024-03-21T09:23:28.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:23:28.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:23:28.380 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:23:28.414 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T09:23:28.415 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:23:28.475 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:23:28.640 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:28 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/1484211019' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:28.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:28 smithi071 ceph-mon[36597]: osdmap e108: 12 total, 12 up, 12 in 2024-03-21T09:23:28.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:28 smithi071 ceph-mon[36597]: pgmap v354: 33 pgs: 33 active+clean; 577 KiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:23:28.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:28 smithi071 ceph-mon[36597]: from='client.? 172.21.15.37:0/474093434' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:28.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:28 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:28.659 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:23:28.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:28 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/1484211019' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:28.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:28 smithi037 ceph-mon[31830]: osdmap e108: 12 total, 12 up, 12 in 2024-03-21T09:23:28.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:28 smithi037 ceph-mon[31830]: pgmap v354: 33 pgs: 33 active+clean; 577 KiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:23:28.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:28 smithi037 ceph-mon[31830]: from='client.? 172.21.15.37:0/474093434' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:28.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:28 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:28.677 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T09:23:28.678 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T09:23:28.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a 2024-03-21T09:23:28.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:28 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/1484211019' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:28.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:28 smithi053 ceph-mon[36169]: osdmap e108: 12 total, 12 up, 12 in 2024-03-21T09:23:28.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:28 smithi053 ceph-mon[36169]: pgmap v354: 33 pgs: 33 active+clean; 577 KiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T09:23:28.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:28 smithi053 ceph-mon[36169]: from='client.? 172.21.15.37:0/474093434' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:28.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:28 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:23:29.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:28 smithi037 systemd[1]: Stopping Ceph mon.a for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:29.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:29 smithi071 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:29.606 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:29 smithi071 ceph-mon[36597]: osdmap e109: 12 total, 12 up, 12 in 2024-03-21T09:23:29.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 ceph-mon[31830]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:29.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 ceph-mon[31830]: osdmap e109: 12 total, 12 up, 12 in 2024-03-21T09:23:29.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a[31826]: 2024-03-21T09:23:29.305+0000 7f9d34600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:29.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a[31826]: 2024-03-21T09:23:29.305+0000 7f9d34600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T09:23:29.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 podman[99084]: 2024-03-21 09:23:29.420166062 +0000 UTC m=+0.272734606 container died 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T09:23:29.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:29 smithi053 ceph-mon[36169]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:23:29.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:29 smithi053 ceph-mon[36169]: osdmap e109: 12 total, 12 up, 12 in 2024-03-21T09:23:30.177 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 podman[99084]: 2024-03-21 09:23:29.914612377 +0000 UTC m=+0.767180881 container cleanup 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-21T09:23:30.177 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:29 smithi037 bash[99084]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a 2024-03-21T09:23:30.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 21 09:23:30 smithi037 podman[99098]: 2024-03-21 09:23:30.232409352 +0000 UTC m=+0.798938441 container remove 70bc41de4721926d5ba3d765ea11782c5e18d910d26323f53f4a9076f1472f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T09:23:30.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T09:23:30.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:30.919 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T09:23:30.919 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-21T09:23:30.919 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.b 2024-03-21T09:23:31.280 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:30 smithi053 systemd[1]: Stopping Ceph mon.b for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:31.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b[36165]: 2024-03-21T09:23:31.280+0000 7f684ab30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:31.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b[36165]: 2024-03-21T09:23:31.280+0000 7f684ab30700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T09:23:31.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:31 smithi053 podman[72730]: 2024-03-21 09:23:31.466319625 +0000 UTC m=+0.334626473 container died fe569e9ff56510e7daebd8bf8fef507e06d6cbeb45ac1fe4c036bc522f93e05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T09:23:32.130 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:31 smithi053 podman[72730]: 2024-03-21 09:23:31.848627832 +0000 UTC m=+0.716934674 container cleanup fe569e9ff56510e7daebd8bf8fef507e06d6cbeb45ac1fe4c036bc522f93e05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T09:23:32.131 INFO:journalctl@ceph.mon.b.smithi053.stdout:Mar 21 09:23:31 smithi053 bash[72730]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-b 2024-03-21T09:23:32.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T09:23:32.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:32.328 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-21T09:23:32.329 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-21T09:23:32.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.c 2024-03-21T09:23:32.642 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:32 smithi071 systemd[1]: Stopping Ceph mon.c for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:33.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c[36592]: 2024-03-21T09:23:32.644+0000 7fe96eca7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:33.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c[36592]: 2024-03-21T09:23:32.644+0000 7fe96eca7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T09:23:33.106 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:32 smithi071 podman[73668]: 2024-03-21 09:23:32.859576465 +0000 UTC m=+0.334147447 container died afdc6583bd51cf60542b22daf8a40cc10b6d054d5952d518167765fcfebd4b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T09:23:33.496 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:33 smithi071 podman[73668]: 2024-03-21 09:23:33.205943789 +0000 UTC m=+0.680514769 container cleanup afdc6583bd51cf60542b22daf8a40cc10b6d054d5952d518167765fcfebd4b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:23:33.496 INFO:journalctl@ceph.mon.c.smithi071.stdout:Mar 21 09:23:33 smithi071 bash[73668]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mon-c 2024-03-21T09:23:33.621 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T09:23:33.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:33.668 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-21T09:23:33.668 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T09:23:33.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x 2024-03-21T09:23:34.177 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:33 smithi037 systemd[1]: Stopping Ceph mgr.x for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:34.927 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:23:34.485+0000 7f0351000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:34.927 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x[32072]: 2024-03-21T09:23:34.485+0000 7f0351000700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-21T09:23:34.927 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:34 smithi037 podman[99206]: 2024-03-21 09:23:34.590171512 +0000 UTC m=+0.491700929 container died 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T09:23:35.410 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:35 smithi037 podman[99206]: 2024-03-21 09:23:35.135815856 +0000 UTC m=+1.037345307 container cleanup 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T09:23:35.410 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:35 smithi037 bash[99206]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x 2024-03-21T09:23:35.677 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 21 09:23:35 smithi037 podman[99220]: 2024-03-21 09:23:35.412426552 +0000 UTC m=+0.805546115 container remove 2d661ad69bc331e4541b5d59ef46b2d16b3227c6584fae990a1dd562cf216639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-21T09:23:36.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T09:23:36.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:36.130 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T09:23:36.130 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T09:23:36.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.y 2024-03-21T09:23:36.445 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:23:36 smithi053 systemd[1]: Stopping Ceph mgr.y for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:36.761 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:23:36 smithi053 podman[72851]: 2024-03-21 09:23:36.482746927 +0000 UTC m=+0.150282274 container died 27f6ceb034e87afc776032c9d0e3ad61e6b95ac0bb4c405674e7c8a5f0d76757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:23:37.119 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:23:36 smithi053 podman[72851]: 2024-03-21 09:23:36.870804644 +0000 UTC m=+0.538339987 container cleanup 27f6ceb034e87afc776032c9d0e3ad61e6b95ac0bb4c405674e7c8a5f0d76757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T09:23:37.119 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Mar 21 09:23:36 smithi053 bash[72851]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-y 2024-03-21T09:23:37.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T09:23:37.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:37.325 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T09:23:37.325 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-21T09:23:37.326 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.z 2024-03-21T09:23:37.606 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:23:37 smithi071 systemd[1]: Stopping Ceph mgr.z for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:38.026 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:23:37 smithi071 podman[73804]: 2024-03-21 09:23:37.672483656 +0000 UTC m=+0.152967885 container died be3dcb7258ecb9af11332d7c4e6f81bdd66e21022b4f7bfa388035ea2d267ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T09:23:38.356 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:23:38 smithi071 podman[73804]: 2024-03-21 09:23:38.128048871 +0000 UTC m=+0.608533088 container cleanup be3dcb7258ecb9af11332d7c4e6f81bdd66e21022b4f7bfa388035ea2d267ddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T09:23:38.356 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Mar 21 09:23:38 smithi071 bash[73804]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mgr-z 2024-03-21T09:23:38.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-21T09:23:38.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:38.569 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-21T09:23:38.569 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T09:23:38.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.0 2024-03-21T09:23:38.926 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:38 smithi037 systemd[1]: Stopping Ceph osd.0 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:39.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0[44154]: 2024-03-21T09:23:39.148+0000 7fc7cb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:39.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0[44154]: 2024-03-21T09:23:39.148+0000 7fc7cb000700 -1 osd.0 109 *** Got signal Terminated *** 2024-03-21T09:23:39.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0[44154]: 2024-03-21T09:23:39.148+0000 7fc7cb000700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:23:45.177 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:44 smithi037 podman[99326]: 2024-03-21 09:23:44.756212175 +0000 UTC m=+5.760338856 container died 50b219b1586c70275700dfb000d7dacb8f063a761ffa059acb422c2fb3ce71d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T09:23:45.642 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:45 smithi037 podman[99326]: 2024-03-21 09:23:45.364446427 +0000 UTC m=+6.368573111 container cleanup 50b219b1586c70275700dfb000d7dacb8f063a761ffa059acb422c2fb3ce71d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T09:23:45.642 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:45 smithi037 bash[99326]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0 2024-03-21T09:23:45.926 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:45 smithi037 podman[99346]: 2024-03-21 09:23:45.643120514 +0000 UTC m=+0.875721252 container remove 50b219b1586c70275700dfb000d7dacb8f063a761ffa059acb422c2fb3ce71d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:23:46.852 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:46 smithi037 podman[99435]: 2024-03-21 09:23:46.368670172 +0000 UTC m=+0.040491219 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:23:46.852 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:46 smithi037 podman[99435]: 2024-03-21 09:23:46.492110931 +0000 UTC m=+0.163932008 container create 7026121da2b072fe1ec25015abbbd2f8567967e9272a0989e545c8039c186922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:23:47.177 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:46 smithi037 podman[99435]: 2024-03-21 09:23:46.854119269 +0000 UTC m=+0.525940348 container init 7026121da2b072fe1ec25015abbbd2f8567967e9272a0989e545c8039c186922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-21T09:23:47.177 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:46 smithi037 podman[99435]: 2024-03-21 09:23:46.866224025 +0000 UTC m=+0.538045081 container start 7026121da2b072fe1ec25015abbbd2f8567967e9272a0989e545c8039c186922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-03-21T09:23:47.177 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:46 smithi037 podman[99435]: 2024-03-21 09:23:46.969736774 +0000 UTC m=+0.641557903 container attach 7026121da2b072fe1ec25015abbbd2f8567967e9272a0989e545c8039c186922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-21T09:23:47.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 21 09:23:47 smithi037 podman[99435]: 2024-03-21 09:23:47.313466352 +0000 UTC m=+0.985287432 container died 7026121da2b072fe1ec25015abbbd2f8567967e9272a0989e545c8039c186922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:23:48.181 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T09:23:48.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:48.307 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T09:23:48.307 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T09:23:48.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.1 2024-03-21T09:23:48.640 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:48 smithi053 systemd[1]: Stopping Ceph osd.1 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:48.945 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1[40766]: 2024-03-21T09:23:48.642+0000 7f5b14df0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:48.945 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1[40766]: 2024-03-21T09:23:48.642+0000 7f5b14df0700 -1 osd.1 109 *** Got signal Terminated *** 2024-03-21T09:23:48.945 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1[40766]: 2024-03-21T09:23:48.642+0000 7f5b14df0700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:23:54.265 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:53 smithi053 podman[72988]: 2024-03-21 09:23:53.976303514 +0000 UTC m=+5.457840786 container died f7648173707c996b4abba60f342e60a382ed57d7989f4bb84d8ac65ec6ec5dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T09:23:54.633 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:54 smithi053 podman[72988]: 2024-03-21 09:23:54.358920829 +0000 UTC m=+5.840458094 container cleanup f7648173707c996b4abba60f342e60a382ed57d7989f4bb84d8ac65ec6ec5dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T09:23:54.633 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:54 smithi053 bash[72988]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1 2024-03-21T09:23:54.928 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:54 smithi053 podman[73006]: 2024-03-21 09:23:54.635673987 +0000 UTC m=+0.657453032 container remove f7648173707c996b4abba60f342e60a382ed57d7989f4bb84d8ac65ec6ec5dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T09:23:55.179 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:54 smithi053 podman[73092]: 2024-03-21 09:23:54.833678444 +0000 UTC m=+0.018000609 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:23:55.179 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:54 smithi053 podman[73092]: 2024-03-21 09:23:54.944952629 +0000 UTC m=+0.129274801 container create ff39da6b2a557b34588bacf7f38be2e6a1d1a805ad87b38c51c804c22d3231d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T09:23:55.179 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:55 smithi053 podman[73092]: 2024-03-21 09:23:55.181826295 +0000 UTC m=+0.366148463 container init ff39da6b2a557b34588bacf7f38be2e6a1d1a805ad87b38c51c804c22d3231d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-21T09:23:55.445 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:55 smithi053 podman[73092]: 2024-03-21 09:23:55.185543738 +0000 UTC m=+0.369865882 container start ff39da6b2a557b34588bacf7f38be2e6a1d1a805ad87b38c51c804c22d3231d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:23:55.445 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:55 smithi053 podman[73092]: 2024-03-21 09:23:55.27286388 +0000 UTC m=+0.457186042 container attach ff39da6b2a557b34588bacf7f38be2e6a1d1a805ad87b38c51c804c22d3231d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T09:23:55.790 INFO:journalctl@ceph.osd.1.smithi053.stdout:Mar 21 09:23:55 smithi053 podman[73092]: 2024-03-21 09:23:55.491323772 +0000 UTC m=+0.675645921 container died ff39da6b2a557b34588bacf7f38be2e6a1d1a805ad87b38c51c804c22d3231d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T09:23:56.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T09:23:56.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:23:56.152 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T09:23:56.152 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T09:23:56.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.2 2024-03-21T09:23:56.470 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:23:56 smithi071 systemd[1]: Stopping Ceph osd.2 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:23:56.857 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:23:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2[41771]: 2024-03-21T09:23:56.471+0000 7f5bcdc39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:23:56.857 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:23:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2[41771]: 2024-03-21T09:23:56.471+0000 7f5bcdc39700 -1 osd.2 109 *** Got signal Terminated *** 2024-03-21T09:23:56.857 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:23:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2[41771]: 2024-03-21T09:23:56.471+0000 7f5bcdc39700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:02.288 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:01 smithi071 podman[73927]: 2024-03-21 09:24:01.965529842 +0000 UTC m=+5.614703130 container died 69ef66ea9b005c7b17c81781a2f411c3adcadbb1ba67e6ddfba8d57bd31614f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-21T09:24:02.605 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:02 smithi071 podman[73927]: 2024-03-21 09:24:02.381654092 +0000 UTC m=+6.030827326 container cleanup 69ef66ea9b005c7b17c81781a2f411c3adcadbb1ba67e6ddfba8d57bd31614f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T09:24:02.605 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:02 smithi071 bash[73927]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2 2024-03-21T09:24:02.856 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:02 smithi071 podman[73944]: 2024-03-21 09:24:02.608258924 +0000 UTC m=+0.645230929 container remove 69ef66ea9b005c7b17c81781a2f411c3adcadbb1ba67e6ddfba8d57bd31614f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:24:03.356 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:03 smithi071 podman[74033]: 2024-03-21 09:24:02.937348645 +0000 UTC m=+0.017484329 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:03.357 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:03 smithi071 podman[74033]: 2024-03-21 09:24:03.084116071 +0000 UTC m=+0.164251755 container create 6d49f7029ca31812a2dd65775a5a189d16b6973bbdc0adbfbee421c8f37c7262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T09:24:03.652 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:03 smithi071 podman[74033]: 2024-03-21 09:24:03.362896064 +0000 UTC m=+0.443031750 container init 6d49f7029ca31812a2dd65775a5a189d16b6973bbdc0adbfbee421c8f37c7262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:03.652 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:03 smithi071 podman[74033]: 2024-03-21 09:24:03.366214878 +0000 UTC m=+0.446350554 container start 6d49f7029ca31812a2dd65775a5a189d16b6973bbdc0adbfbee421c8f37c7262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:24:03.652 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:03 smithi071 podman[74033]: 2024-03-21 09:24:03.4703438 +0000 UTC m=+0.550479499 container attach 6d49f7029ca31812a2dd65775a5a189d16b6973bbdc0adbfbee421c8f37c7262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T09:24:03.954 INFO:journalctl@ceph.osd.2.smithi071.stdout:Mar 21 09:24:03 smithi071 podman[74033]: 2024-03-21 09:24:03.655312619 +0000 UTC m=+0.735448308 container died 6d49f7029ca31812a2dd65775a5a189d16b6973bbdc0adbfbee421c8f37c7262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:04.271 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T09:24:04.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:04.312 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T09:24:04.312 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T09:24:04.312 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.3 2024-03-21T09:24:04.673 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:04 smithi037 systemd[1]: Stopping Ceph osd.3 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:04.927 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:04.891+0000 7f678f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:04.927 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:04.891+0000 7f678f200700 -1 osd.3 109 *** Got signal Terminated *** 2024-03-21T09:24:04.927 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:04 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:04.891+0000 7f678f200700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:05.926 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:05 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:05.676+0000 7f6785a00700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:40.266088+0000 front 2024-03-21T09:23:40.266332+0000 (oldest deadline 2024-03-21T09:24:04.966269+0000) 2024-03-21T09:24:05.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:05.514+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:06.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:06.510+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:06.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:06.539+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:07.177 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:06 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:06.688+0000 7f6785a00700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:40.266088+0000 front 2024-03-21T09:23:40.266332+0000 (oldest deadline 2024-03-21T09:24:04.966269+0000) 2024-03-21T09:24:07.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:07.520+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:07.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:07.565+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:07.926 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:07.722+0000 7f6785a00700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:40.266088+0000 front 2024-03-21T09:23:40.266332+0000 (oldest deadline 2024-03-21T09:24:04.966269+0000) 2024-03-21T09:24:07.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:07 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:07.546+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:07.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:07.552+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:08.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:08.562+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:08.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:08.549+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:08.926 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:08.704+0000 7f6785a00700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:40.266088+0000 front 2024-03-21T09:23:40.266332+0000 (oldest deadline 2024-03-21T09:24:04.966269+0000) 2024-03-21T09:24:08.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:08 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:08.519+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:08.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:08.545+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:09.744 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:09 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:09.495+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:09.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:09.557+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:09.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:09.555+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:09.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:09.569+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:10.177 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:09 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3[50845]: 2024-03-21T09:24:09.746+0000 7f6785a00700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:40.266088+0000 front 2024-03-21T09:23:40.266332+0000 (oldest deadline 2024-03-21T09:24:04.966269+0000) 2024-03-21T09:24:10.551 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:10.180+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:10.677 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:10 smithi037 podman[99541]: 2024-03-21 09:24:10.412475202 +0000 UTC m=+5.687402176 container died 248f5c3a4f3dba63262431cafe23e5b8b47a2fdb08bb305977d1412c36a254dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T09:24:10.677 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:10 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:10.515+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:10.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:10.554+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:10.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:10.577+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:10.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:10 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:10.596+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:11.144 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:10 smithi037 podman[99541]: 2024-03-21 09:24:10.901474922 +0000 UTC m=+6.176401940 container cleanup 248f5c3a4f3dba63262431cafe23e5b8b47a2fdb08bb305977d1412c36a254dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T09:24:11.144 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:10 smithi037 bash[99541]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3 2024-03-21T09:24:11.427 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:11 smithi037 podman[99570]: 2024-03-21 09:24:11.14617903 +0000 UTC m=+0.722737489 container remove 248f5c3a4f3dba63262431cafe23e5b8b47a2fdb08bb305977d1412c36a254dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T09:24:11.516 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:11.152+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:11.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:11.518+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:11.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:11.574+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:11.926 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:11 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:11.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:11.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:11 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:11.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:12.340 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:12 smithi037 podman[99654]: 2024-03-21 09:24:11.928577656 +0000 UTC m=+0.034864256 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:12.340 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:12 smithi037 podman[99654]: 2024-03-21 09:24:12.045337299 +0000 UTC m=+0.151623923 container create 2f578f5503fd8b867a4573fa4092e8799cf00d1b0b6cf53072198e9da3185b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:24:12.544 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:12.167+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:12.676 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:12 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:12.548+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:12.677 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:12 smithi037 podman[99654]: 2024-03-21 09:24:12.341732523 +0000 UTC m=+0.448019186 container init 2f578f5503fd8b867a4573fa4092e8799cf00d1b0b6cf53072198e9da3185b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:24:12.677 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:12 smithi037 podman[99654]: 2024-03-21 09:24:12.354573152 +0000 UTC m=+0.460859799 container start 2f578f5503fd8b867a4573fa4092e8799cf00d1b0b6cf53072198e9da3185b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:12.677 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:12 smithi037 podman[99654]: 2024-03-21 09:24:12.464840145 +0000 UTC m=+0.571126845 container attach 2f578f5503fd8b867a4573fa4092e8799cf00d1b0b6cf53072198e9da3185b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T09:24:12.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:12.547+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:12.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:12.592+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:12.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:12 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:12.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:13.162 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 21 09:24:12 smithi037 podman[99680]: 2024-03-21 09:24:12.868759471 +0000 UTC m=+0.036266918 container died 2f578f5503fd8b867a4573fa4092e8799cf00d1b0b6cf53072198e9da3185b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T09:24:13.572 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:13.213+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:13.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T09:24:13.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:13.733 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T09:24:13.734 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T09:24:13.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.4 2024-03-21T09:24:13.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:13.574+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:13.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:13.600+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:13.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:13 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:13.558+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:13.945 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:13 smithi053 systemd[1]: Stopping Ceph osd.4 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:13.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:13 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:13.576+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:14.445 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:14.045+0000 7fc042a8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:14.446 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:14.045+0000 7fc042a8c700 -1 osd.4 109 *** Got signal Terminated *** 2024-03-21T09:24:14.446 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:14.045+0000 7fc042a8c700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:14.560 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:14.233+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:14.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:14.563+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:14.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:14.619+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:14.869 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:14.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:14.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:14 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:14.531+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:15.195 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:14 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:14.871+0000 7fc03b078700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.345344+0000 front 2024-03-21T09:23:51.345315+0000 (oldest deadline 2024-03-21T09:24:14.845499+0000) 2024-03-21T09:24:15.515 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:15.251+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:15.515 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:15.251+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:15.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:15.555+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:15.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:15.633+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:15.919 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:15 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:15.604+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:15.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:15 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:15.562+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:16.195 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:15 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:15.919+0000 7fc03b078700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.345344+0000 front 2024-03-21T09:23:51.345315+0000 (oldest deadline 2024-03-21T09:24:14.845499+0000) 2024-03-21T09:24:16.563 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:16.222+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:16.563 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:16.222+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:16.677 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:16 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:16.595+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:16.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:16.566+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:16.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:16.602+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:16.890 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:16 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:16.569+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:17.176 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:16 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:16.891+0000 7fc03b078700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.345344+0000 front 2024-03-21T09:23:51.345315+0000 (oldest deadline 2024-03-21T09:24:14.845499+0000) 2024-03-21T09:24:17.177 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:16 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:16.927+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:17.554 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:17.179+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:17.554 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:17.179+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:17.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:17.556+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:17.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:17.640+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:17.873 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:17 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:17.535+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:17.873 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:17 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:17.535+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:17.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:17.629+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:17.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:17 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:17.629+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:18.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:17 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:17.904+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:18.195 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:17 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:17.875+0000 7fc03b078700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.345344+0000 front 2024-03-21T09:23:51.345315+0000 (oldest deadline 2024-03-21T09:24:14.845499+0000) 2024-03-21T09:24:18.583 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:18 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:18.210+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:18.583 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:18 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:18.210+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:18.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:18 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:18.509+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:18.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:18 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:18.509+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:18.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:18 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:18.586+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:18.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:18 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:18.643+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:18.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:18 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:18.636+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:18.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:18 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:18.636+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:18.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:18 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:18.697+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:19.195 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:18 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4[47765]: 2024-03-21T09:24:18.851+0000 7fc03b078700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.345344+0000 front 2024-03-21T09:23:51.345315+0000 (oldest deadline 2024-03-21T09:24:14.845499+0000) 2024-03-21T09:24:19.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:18 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:18.927+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:19.557 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:19 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:19.215+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:19.557 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:19 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:19.215+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:19.695 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:19 smithi053 podman[73202]: 2024-03-21 09:24:19.41539663 +0000 UTC m=+5.477631333 container died fade87b91fd050168dc3ee9bc9617c916f650019da1c049a78ab4cb79ba25684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:19.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:19 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:19.538+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:19.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:19 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:19.538+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:19.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:19 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:19.559+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:19.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:19 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:19.686+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:19.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:19 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:19.664+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:19.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:19 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:19.629+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:19.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:19 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:19.629+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:20.070 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:19 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:19.884+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:20.071 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:19 smithi053 podman[73202]: 2024-03-21 09:24:19.788370315 +0000 UTC m=+5.850605008 container cleanup fade87b91fd050168dc3ee9bc9617c916f650019da1c049a78ab4cb79ba25684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T09:24:20.071 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:19 smithi053 bash[73202]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4 2024-03-21T09:24:20.418 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:20 smithi053 podman[73230]: 2024-03-21 09:24:20.073275429 +0000 UTC m=+0.653251253 container remove fade87b91fd050168dc3ee9bc9617c916f650019da1c049a78ab4cb79ba25684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T09:24:20.517 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:20 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:20.244+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:20.517 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:20 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:20.244+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:20.695 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:20 smithi053 podman[73323]: 2024-03-21 09:24:20.324162858 +0000 UTC m=+0.018127101 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:20.696 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:20 smithi053 podman[73323]: 2024-03-21 09:24:20.449218913 +0000 UTC m=+0.143183181 container create 7ac1f94bff46e44ccd70ba344c06c7641067aa920c7c4920c3e73a37da0ccd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T09:24:20.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:20 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:20.490+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:20.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:20 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:20.490+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:20.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:20 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:20.518+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:20.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:20 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:20.692+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:20.926 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:20 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:20.658+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:20.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:20 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:20.598+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:20.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:20 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:20.598+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:21.000 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:20 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:20.895+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:21.000 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:20 smithi053 podman[73323]: 2024-03-21 09:24:20.701824578 +0000 UTC m=+0.395788828 container init 7ac1f94bff46e44ccd70ba344c06c7641067aa920c7c4920c3e73a37da0ccd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T09:24:21.000 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:20 smithi053 podman[73323]: 2024-03-21 09:24:20.705189525 +0000 UTC m=+0.399153755 container start 7ac1f94bff46e44ccd70ba344c06c7641067aa920c7c4920c3e73a37da0ccd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:24:21.000 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:20 smithi053 podman[73323]: 2024-03-21 09:24:20.801816449 +0000 UTC m=+0.495780709 container attach 7ac1f94bff46e44ccd70ba344c06c7641067aa920c7c4920c3e73a37da0ccd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:21.361 INFO:journalctl@ceph.osd.4.smithi053.stdout:Mar 21 09:24:21 smithi053 podman[73323]: 2024-03-21 09:24:21.003385739 +0000 UTC m=+0.697349970 container died 7ac1f94bff46e44ccd70ba344c06c7641067aa920c7c4920c3e73a37da0ccd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T09:24:21.497 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:21 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:21.222+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:21.498 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:21 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:21.222+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:21.661 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:21 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:21.511+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:21.661 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:21 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:21.511+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:21.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T09:24:21.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:21.738 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T09:24:21.738 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T09:24:21.738 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.5 2024-03-21T09:24:21.767 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:21 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:21.499+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:21.767 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:21 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:21.659+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:21.879 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:21 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:21.880+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:21.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:21.635+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:21.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:21.556+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:21.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:21 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:21.556+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:22.057 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:21 smithi071 systemd[1]: Stopping Ceph osd.5 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:22.356 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:22.059+0000 7fd1db3dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:22.357 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:22.059+0000 7fd1db3dd700 -1 osd.5 109 *** Got signal Terminated *** 2024-03-21T09:24:22.357 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:22.059+0000 7fd1db3dd700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:22.357 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:22.242+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:22.357 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:22.242+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:22.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:22.517+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:22.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:22.638+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:22.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:22 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:22.638+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:22.925 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:22 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:22.552+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:22.925 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:22 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:22.552+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:22.926 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:22 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:22.590+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:22.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:22 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:22.508+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:22.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:22 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:22.508+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:23.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:22 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:22.927+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:23.593 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:23 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:23.479+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:23.593 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:23 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:23.283+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:23.593 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:23 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:23.283+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:23.856 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:23 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:23.594+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:23.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:23 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:23.594+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:23.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:23 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:23.565+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:23.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:23 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:23.565+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:23.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:23 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:23.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:23.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:23 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:23.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:23.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:23 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:23.517+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:23.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:23 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:23.517+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:24.445 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:23 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:23.970+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:24.555 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:24.461+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:24.556 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:24.461+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:24.556 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:24.236+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:24.556 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:24.236+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:24.556 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:24.236+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:58.636080+0000 front 2024-03-21T09:23:58.636056+0000 (oldest deadline 2024-03-21T09:24:23.936205+0000) 2024-03-21T09:24:24.856 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:24.557+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:24.856 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:24 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:24.557+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:24.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:24.570+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:24.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:24.570+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:24.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:24.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:24.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:24 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:24.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:24.940 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:24.511+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:24.940 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:24.511+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:25.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:24 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:24.942+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:25.562 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:25.467+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:25.562 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:25.467+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:25.562 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:25.235+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:25.562 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:25.235+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:25.562 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:25.235+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:58.636080+0000 front 2024-03-21T09:23:58.636056+0000 (oldest deadline 2024-03-21T09:24:23.936205+0000) 2024-03-21T09:24:25.856 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:25.564+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:25.856 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:25 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:25.564+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:25.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:25 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:25.520+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:25.902 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:25 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:25.520+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:25.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:25 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:25.556+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:25.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:25 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:25.556+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:25.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:25 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:25.471+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:25.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:25 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:25.471+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:25.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:25 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:25.471+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:26.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:25 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:25.904+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:26.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:25 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:25.904+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:26.479 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:26.207+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.133445+0000 front 2024-03-21T09:23:44.133581+0000 (oldest deadline 2024-03-21T09:24:10.033448+0000) 2024-03-21T09:24:26.479 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:26.207+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.435044+0000 front 2024-03-21T09:23:53.435057+0000 (oldest deadline 2024-03-21T09:24:14.535131+0000) 2024-03-21T09:24:26.479 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5[48447]: 2024-03-21T09:24:26.207+0000 7fd1d79d1700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:58.636080+0000 front 2024-03-21T09:23:58.636056+0000 (oldest deadline 2024-03-21T09:24:23.936205+0000) 2024-03-21T09:24:26.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:26.481+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:26.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:26.481+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:26.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:26.584+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:26.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:26 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:26.584+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:26.880 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:26 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:26.518+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:26.880 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:26 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:26.518+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:26.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:26.559+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:26.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:26.559+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:26.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:26.507+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:26.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:26.507+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:26.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:26.507+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:27.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:26 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:26.882+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:27.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:26 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:26.882+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:27.683 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:27 smithi071 podman[74150]: 2024-03-21 09:24:27.431579277 +0000 UTC m=+5.498269057 container died 29461e290e60515132c044bee66a791cc0d3270a3e8e9dd42f3573f1b1eb6a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-21T09:24:27.683 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:27 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:27.557+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:27.683 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:27 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:27.557+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:27.683 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:27 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:27.520+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:27.684 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:27 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:27.520+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:27.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:27 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:27.533+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:27.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:27 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:27.533+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:27.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:27 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:27.533+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:27.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:27 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:27.546+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:27.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:27 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:27.546+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:27.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:27 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:27.517+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:27.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:27 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:27.517+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:27.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:27 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:27.517+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:27.976 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:27 smithi071 podman[74150]: 2024-03-21 09:24:27.768372692 +0000 UTC m=+5.835062476 container cleanup 29461e290e60515132c044bee66a791cc0d3270a3e8e9dd42f3573f1b1eb6a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:27.976 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:27 smithi071 bash[74150]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5 2024-03-21T09:24:28.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:27 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:27.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:28.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:27 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:27.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:28.356 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:27 smithi071 podman[74167]: 2024-03-21 09:24:27.978182272 +0000 UTC m=+0.545085814 container remove 29461e290e60515132c044bee66a791cc0d3270a3e8e9dd42f3573f1b1eb6a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T09:24:28.637 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:28.568+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:28.637 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:28.568+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:28.638 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:28.478+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:28.638 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:28 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:28.478+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:28.638 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:28 smithi071 podman[74258]: 2024-03-21 09:24:28.279554876 +0000 UTC m=+0.017208670 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:28.638 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:28 smithi071 podman[74258]: 2024-03-21 09:24:28.387212622 +0000 UTC m=+0.124866397 container create 275bf925b91e78807befa9d4b4a1a21c07ee62e72a1ef4d4af7f43bd086961b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T09:24:28.638 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:28 smithi071 podman[74258]: 2024-03-21 09:24:28.640222652 +0000 UTC m=+0.377876447 container init 275bf925b91e78807befa9d4b4a1a21c07ee62e72a1ef4d4af7f43bd086961b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:24:28.833 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:28.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:28.833 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:28.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:28.833 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:28.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:28.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:28 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:28.587+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:28.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:28 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:28.587+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:28.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:28 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:28.559+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:28.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:28 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:28.559+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:28.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:28 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:28.559+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:28.930 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:28 smithi071 podman[74258]: 2024-03-21 09:24:28.643648479 +0000 UTC m=+0.381302259 container start 275bf925b91e78807befa9d4b4a1a21c07ee62e72a1ef4d4af7f43bd086961b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=) 2024-03-21T09:24:28.930 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:28 smithi071 podman[74258]: 2024-03-21 09:24:28.748115675 +0000 UTC m=+0.485769461 container attach 275bf925b91e78807befa9d4b4a1a21c07ee62e72a1ef4d4af7f43bd086961b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:29.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:28.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:29.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:28 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:28.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:29.281 INFO:journalctl@ceph.osd.5.smithi071.stdout:Mar 21 09:24:28 smithi071 podman[74258]: 2024-03-21 09:24:28.933932277 +0000 UTC m=+0.671586065 container died 275bf925b91e78807befa9d4b4a1a21c07ee62e72a1ef4d4af7f43bd086961b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:24:29.543 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:29 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:29.523+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:29.543 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:29 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:29.523+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:29.597 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T09:24:29.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:29.685 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T09:24:29.685 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T09:24:29.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.6 2024-03-21T09:24:29.817 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:29.563+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:29.817 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:29.563+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:29.818 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:29.607+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:29.818 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:29.607+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:29.818 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:29 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:29.607+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:29.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:29 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:29.545+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:29.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:29 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:29.545+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:29.874 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:29 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:29.539+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:29.875 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:29 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:29.539+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:29.875 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:29 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:29.539+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:30.177 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:29 smithi037 systemd[1]: Stopping Ceph osd.6 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:30.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:29 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:29.876+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:30.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:29 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:29.876+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:30.568 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:30.295+0000 7f26e1200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:30.568 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:30.295+0000 7f26e1200700 -1 osd.6 109 *** Got signal Terminated *** 2024-03-21T09:24:30.568 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:30.295+0000 7f26e1200700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:30.568 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:30.569+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:30.568 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:30.569+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:30.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:30 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:30.521+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:30.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:30 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:30.521+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:30.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:30 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:30.534+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:30.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:30 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:30.534+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:30.924 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:30 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:30.495+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:30.924 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:30 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:30.495+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:30.924 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:30 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:30.495+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:30.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:30.602+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:30.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:30.602+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:30.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:30 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:30.602+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:31.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:30 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:30.926+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:31.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:30 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:30.926+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:31.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:30 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:30.926+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:31.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:31 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:31.510+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:31.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:31 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:31.510+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:31.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:31 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:31.505+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:31.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:31 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:31.505+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:31.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:31 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:31.505+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:31.885 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:31.469+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:31.885 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:31.469+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:31.886 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:31.469+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:31.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:31.597+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:31.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:31.597+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:31.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:31.633+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:31.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:31.633+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:31.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:31.633+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:32.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:31.887+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:32.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:31.887+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:32.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:31 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:31.887+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:32.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:32.425+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:32.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:32.425+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:32.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:32.425+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:32.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:32.425+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:32.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:32.528+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:32.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:32.528+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:32.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:32.482+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:32.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:32.482+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:32.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:32 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:32.482+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:32.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:32 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:32.619+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:32.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:32 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:32.619+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:32.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:32 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:32.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:32.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:32 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:32.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:32.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:32 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:32.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:33.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:32.880+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:33.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:32.880+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:33.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:32 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:32.880+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:33.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:33.384+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:33.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:33.384+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:33.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:33.384+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:33.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:33.384+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:33.758 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:33 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:33.519+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:33.758 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:33 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:33.519+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:33.759 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:33 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:33.445+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:33.759 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:33 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:33.445+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:33.759 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:33 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:33.445+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:33.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:33 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:33.627+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:33.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:33 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:33.627+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:33.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:33 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:33.614+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:33.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:33 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:33.614+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:33.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:33 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:33.614+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:34.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:33.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:34.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:33.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:34.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:33 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:33.835+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:34.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:34.426+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:34.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:34.426+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:34.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:34.426+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:34.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:34.426+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:34.773 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:34 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:34.484+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:34.773 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:34 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:34.484+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:34.773 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:34 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:34.484+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:34.773 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:34 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:34.460+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:34.774 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:34 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:34.460+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:34.774 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:34 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:34.460+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:34.829 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:34.666+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:48.065599+0000 front 2024-03-21T09:23:48.065667+0000 (oldest deadline 2024-03-21T09:24:18.067029+0000) 2024-03-21T09:24:34.829 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:34.666+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.769348+0000 front 2024-03-21T09:23:59.769404+0000 (oldest deadline 2024-03-21T09:24:22.669596+0000) 2024-03-21T09:24:34.829 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6[57395]: 2024-03-21T09:24:34.666+0000 7f26d7a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.673657+0000 front 2024-03-21T09:24:09.673491+0000 (oldest deadline 2024-03-21T09:24:34.373673+0000) 2024-03-21T09:24:34.829 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:34.574+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:34.830 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:34.574+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:34.830 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:34 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:34.574+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:35.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:34.845+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:35.247 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:34.845+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:35.248 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:34 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:34.845+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:35.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:35.382+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:35.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:35.382+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:35.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:35.383+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:35.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:35.383+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:35.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:35 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:35.494+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:35.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:35 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:35.494+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:35.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:35 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:35.494+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:35.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:35 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:35.439+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:35.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:35 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:35.439+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:35.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:35 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:35.439+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:35.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:35 smithi037 podman[99758]: 2024-03-21 09:24:35.831686189 +0000 UTC m=+5.701087149 container died 3bbf905b3400d66ed555e83f90de21c0d4b615e33ee593219385d303418c080a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:24:35.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:35 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:35.617+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:35.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:35 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:35.617+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:35.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:35 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:35.617+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:35.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:35 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:35.617+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:36.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:35.823+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:36.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:35.823+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:36.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:35 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:35.823+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:36.426 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:36 smithi037 podman[99758]: 2024-03-21 09:24:36.287108128 +0000 UTC m=+6.156509038 container cleanup 3bbf905b3400d66ed555e83f90de21c0d4b615e33ee593219385d303418c080a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-21T09:24:36.427 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:36 smithi037 bash[99758]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6 2024-03-21T09:24:36.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:36.417+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:36.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:36.417+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:36.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:36.417+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:36.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:36.417+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:36.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:36 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:36.508+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:36.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:36 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:36.508+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:36.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:36 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:36.508+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:36.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:36 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:36.432+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:36.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:36 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:36.432+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:36.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:36 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:36.432+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:36.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:36.573+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:36.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:36.574+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:36.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:36.574+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:36.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:36 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:36.574+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:36.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:36 smithi037 podman[99779]: 2024-03-21 09:24:36.555010606 +0000 UTC m=+0.712290480 container remove 3bbf905b3400d66ed555e83f90de21c0d4b615e33ee593219385d303418c080a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:24:37.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:36.871+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:37.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:36.871+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:37.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:36 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:36.871+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:37.586 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:37 smithi037 podman[99872]: 2024-03-21 09:24:37.237624321 +0000 UTC m=+0.034259515 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:37.587 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:37 smithi037 podman[99872]: 2024-03-21 09:24:37.364030794 +0000 UTC m=+0.160665992 container create 2e74fdf78142d8425d23d898f8cfecb31ae2b05a18f5eb1622be788e1906d795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:24:37.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:37.406+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:37.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:37.406+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:37.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:37.406+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:37.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:37.406+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:37.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:37 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:37.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:37.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:37 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:37.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:37.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:37 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:37.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:37.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:37 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:37.472+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:37.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:37 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:37.472+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:37.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:37 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:37.472+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:37.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:37 smithi037 podman[99872]: 2024-03-21 09:24:37.676237551 +0000 UTC m=+0.472872818 container init 2e74fdf78142d8425d23d898f8cfecb31ae2b05a18f5eb1622be788e1906d795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:24:37.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:37 smithi037 podman[99872]: 2024-03-21 09:24:37.687304008 +0000 UTC m=+0.483939239 container start 2e74fdf78142d8425d23d898f8cfecb31ae2b05a18f5eb1622be788e1906d795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-21T09:24:37.927 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:37 smithi037 podman[99872]: 2024-03-21 09:24:37.791897826 +0000 UTC m=+0.588533077 container attach 2e74fdf78142d8425d23d898f8cfecb31ae2b05a18f5eb1622be788e1906d795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T09:24:37.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:37.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:37.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:37.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:37.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:37.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:37.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:37.587+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:38.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:37.873+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:38.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:37.873+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:38.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:37 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:37.873+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:38.427 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 21 09:24:38 smithi037 podman[99872]: 2024-03-21 09:24:38.137329752 +0000 UTC m=+0.933965023 container died 2e74fdf78142d8425d23d898f8cfecb31ae2b05a18f5eb1622be788e1906d795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T09:24:38.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:38.395+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:38.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:38.395+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:38.695 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:38.395+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:38.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:38.395+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:38.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:38 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:38.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:38.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:38 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:38.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:38.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:38 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:38.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:38.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:38 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:38.428+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:38.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:38 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:38.428+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:38.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:38 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:38.428+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:38.910 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:38.626+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:38.911 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:38.626+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:38.911 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:38.626+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:38.911 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:38 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:38.626+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:38.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T09:24:39.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:39.075 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T09:24:39.075 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T09:24:39.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.7 2024-03-21T09:24:39.108 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:38.856+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:39.108 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:38.856+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:39.108 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:38 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:38.856+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:39.406 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 systemd[1]: Stopping Ceph osd.7 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:39.695 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:39.408+0000 7f942a1cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:39.696 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:39.408+0000 7f942a1cc700 -1 osd.7 109 *** Got signal Terminated *** 2024-03-21T09:24:39.696 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:39.408+0000 7f942a1cc700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:39.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:39.419+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:39.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:39.419+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:39.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:39.419+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:39.696 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:39.419+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:39.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:39 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:39.546+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:39.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:39 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:39.546+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:39.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:39 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:39.546+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:39.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:39 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:39.471+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:39.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:39 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:39.471+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:39.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:39 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:39.471+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:39.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:39.584+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:39.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:39.584+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:39.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:39.584+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:39.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:39 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:39.584+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:40.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:39.876+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:40.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:39.876+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:40.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:39 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:39.876+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:40.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:40.460+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:40.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:40.460+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:40.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:40.460+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:40.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:40.460+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:40.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:40 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:40.538+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:40.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:40 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:40.538+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:40.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:40 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:40.538+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:40.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:40 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:40.422+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:40.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:40 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:40.422+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:40.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:40 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:40.422+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:40.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:40 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:40.612+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:40.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:40 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:40.612+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:40.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:40 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:40.612+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:40.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:40 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:40.612+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:41.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:40.852+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:41.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:40.852+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:41.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:40.852+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:41.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:40 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:40.852+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.225967+0000 front 2024-03-21T09:24:18.226082+0000 (oldest deadline 2024-03-21T09:24:39.926013+0000) 2024-03-21T09:24:41.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:41.490+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:41.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:41.490+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:41.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:41.490+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:41.834 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:41.490+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:41.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:41 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:41.537+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:41.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:41 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:41.537+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:41.856 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:41 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:41.537+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:41.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:41 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:41.375+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:41.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:41 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:41.375+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:41.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:41 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:41.375+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:41.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:41.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:41.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:41.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:41.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:41.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:41.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:41 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:41.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:42.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:41.836+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:42.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:41.836+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:42.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:41.836+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:42.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:41 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:41.836+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.225967+0000 front 2024-03-21T09:24:18.226082+0000 (oldest deadline 2024-03-21T09:24:39.926013+0000) 2024-03-21T09:24:42.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:42 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:42.576+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:42.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:42 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:42.576+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:42.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:42 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:42.576+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:42.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:42 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:42.357+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:42.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:42 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:42.357+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:42.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:42 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:42.357+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:42.859 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:42.531+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:42.859 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:42.531+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:42.859 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:42.531+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:42.859 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:42.531+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:42.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:42.522+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:42.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:42.522+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:42.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:42.522+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:42.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:42.522+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:42.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:42 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:42.522+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:43.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:42.861+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:43.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:42.861+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:43.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:42.861+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:43.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:42.861+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.225967+0000 front 2024-03-21T09:24:18.226082+0000 (oldest deadline 2024-03-21T09:24:39.926013+0000) 2024-03-21T09:24:43.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:43.588+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:43.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:43.588+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:43.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:43.588+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:43.858 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:43.380+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:43.858 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:43.380+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:43.858 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:43.380+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:43.858 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:43 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:43.380+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:43.886 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:43.564+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:43.886 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:43.564+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:43.886 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:43.564+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:43.886 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:43.564+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:43.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:43.555+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:43.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:43.555+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:43.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:43.555+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:43.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:43.555+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:43.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:43 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:43.555+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:44.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:43.888+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.320543+0000 front 2024-03-21T09:23:53.320609+0000 (oldest deadline 2024-03-21T09:24:16.820729+0000) 2024-03-21T09:24:44.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:43.888+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.721960+0000 front 2024-03-21T09:23:59.721812+0000 (oldest deadline 2024-03-21T09:24:25.021894+0000) 2024-03-21T09:24:44.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:43.888+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.523328+0000 front 2024-03-21T09:24:08.523314+0000 (oldest deadline 2024-03-21T09:24:30.723729+0000) 2024-03-21T09:24:44.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:43 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7[54344]: 2024-03-21T09:24:43.888+0000 7f94227b8700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.225967+0000 front 2024-03-21T09:24:18.226082+0000 (oldest deadline 2024-03-21T09:24:39.926013+0000) 2024-03-21T09:24:44.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:44.550+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:44.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:44.550+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:44.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:44.550+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:44.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:44.550+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:44.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:44.341+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:44.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:44.341+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:44.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:44.341+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:44.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:44 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:44.341+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:44.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:44 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:44.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:44.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:44 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:44.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:44.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:44 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:44.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:44.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:44 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:44.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:44.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:44 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:44.566+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:44.945 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:44 smithi053 podman[73429]: 2024-03-21 09:24:44.731326277 +0000 UTC m=+5.445339876 container died 515f4d74c876f427b3df1f481102910a6cc788c455affb58c149d816eda72a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:44.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:44 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:44.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:44.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:44 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:44.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:44.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:44 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:44.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:44.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:44 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:44.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:44.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:44 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:44.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:45.445 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:45 smithi053 podman[73429]: 2024-03-21 09:24:45.116749939 +0000 UTC m=+5.830763530 container cleanup 515f4d74c876f427b3df1f481102910a6cc788c455affb58c149d816eda72a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-21T09:24:45.445 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:45 smithi053 bash[73429]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:45.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:45.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:45.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:45.548+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:45.312+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:45.312+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:45.312+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:45.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:45 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:45.312+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:45.890 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:45 smithi053 podman[73448]: 2024-03-21 09:24:45.493007636 +0000 UTC m=+0.754666978 container remove 515f4d74c876f427b3df1f481102910a6cc788c455affb58c149d816eda72a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T09:24:45.891 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:45.644+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:45.891 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:45.644+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:45.891 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:45.644+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:45.891 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:45.644+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:45.891 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:45.644+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:45.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:45.572+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:45.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:45.572+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:45.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:45.572+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:45.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:45.572+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:45.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:45 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:45.572+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:46.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:45 smithi053 podman[73540]: 2024-03-21 09:24:45.79620979 +0000 UTC m=+0.018351360 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:46.195 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:45 smithi053 podman[73540]: 2024-03-21 09:24:45.912638481 +0000 UTC m=+0.134780050 container create b6e7e77623e8251d7622ba921de5cea7fbb73e43f4947e80ba851f627830c3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T09:24:46.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:46 smithi053 podman[73540]: 2024-03-21 09:24:46.13791572 +0000 UTC m=+0.360057307 container init b6e7e77623e8251d7622ba921de5cea7fbb73e43f4947e80ba851f627830c3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T09:24:46.196 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:46 smithi053 podman[73540]: 2024-03-21 09:24:46.141707597 +0000 UTC m=+0.363849168 container start b6e7e77623e8251d7622ba921de5cea7fbb73e43f4947e80ba851f627830c3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T09:24:46.357 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:46.265+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:46.357 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:46.265+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:46.357 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:46.265+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:46.357 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:46.265+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:46.456 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:46 smithi053 podman[73540]: 2024-03-21 09:24:46.221108987 +0000 UTC m=+0.443250573 container attach b6e7e77623e8251d7622ba921de5cea7fbb73e43f4947e80ba851f627830c3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T09:24:46.456 INFO:journalctl@ceph.osd.7.smithi053.stdout:Mar 21 09:24:46 smithi053 podman[73540]: 2024-03-21 09:24:46.436954471 +0000 UTC m=+0.659096042 container died b6e7e77623e8251d7622ba921de5cea7fbb73e43f4947e80ba851f627830c3f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T09:24:46.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:46.553+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:46.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:46.553+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:46.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:46.553+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:46.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:46 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:46.553+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:46.871 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:46 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:46.610+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:46.872 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:46 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:46.610+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:46.872 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:46 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:46.610+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:46.872 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:46 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:46.610+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:46.872 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:46 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:46.610+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:46.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:46 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:46.561+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:46.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:46 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:46.561+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:46.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:46 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:46.561+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:46.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:46 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:46.561+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:46.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:46 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:46.561+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:47.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T09:24:47.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:47.296 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T09:24:47.296 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-21T09:24:47.297 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.8 2024-03-21T09:24:47.548 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 systemd[1]: Stopping Ceph osd.8 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:47.548 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:47.245+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:47.548 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:47.245+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:47.548 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:47.245+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:47.549 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:47.245+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:47.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:47.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:47.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:47.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.549+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:47.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.627+0000 7fa9c65f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:47.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.627+0000 7fa9c65f3700 -1 osd.8 109 *** Got signal Terminated *** 2024-03-21T09:24:47.858 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:47 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:47.627+0000 7fa9c65f3700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:47.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:47 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:47.545+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:47.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:47 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:47.545+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:47.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:47 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:47.545+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:47.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:47 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:47.545+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:47.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:47 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:47.545+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:47.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:47 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:47.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:47.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:47 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:47.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:47.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:47 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:47.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:47.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:47 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:47.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:47.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:47 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:47.597+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:48.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:48.227+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:48.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:48.227+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:48.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:48.227+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:48.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:48.227+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:48.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:48.559+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:48.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:48.559+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:48.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:48.559+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:48.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:48 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:48.559+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:48.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:48.519+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:48.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:48.519+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:48.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:48.519+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:48.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:48.519+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:48.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:48 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:48.519+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:48.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:48.576+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:48.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:48.576+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:48.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:48.576+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:48.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:48.576+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:48.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:48 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:48.576+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:49.569 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:49.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:49.569 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:49.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:49.570 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:49.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:49.570 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:49.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:49.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:49.571+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:49.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:49.571+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:49.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:49.571+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:49.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:49.571+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:49.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:49 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:49.571+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:24.350167+0000 front 2024-03-21T09:24:24.350190+0000 (oldest deadline 2024-03-21T09:24:49.050244+0000) 2024-03-21T09:24:49.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:49.476+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:49.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:49.476+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:49.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:49.476+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:49.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:49.476+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:49.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:49 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:49.476+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:49.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:49.602+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:49.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:49.602+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:49.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:49.602+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:49.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:49.602+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:49.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:49.602+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:50.522 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:50.247+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:50.522 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:50.247+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:50.522 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:50.247+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:50.522 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:50.247+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:50.523 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:50.247+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:50.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:50.524+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:50.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:50.524+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:50.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:50.524+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:50.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:50.524+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:50.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:50 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:50.524+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:24.350167+0000 front 2024-03-21T09:24:24.350190+0000 (oldest deadline 2024-03-21T09:24:49.050244+0000) 2024-03-21T09:24:50.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:50.463+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:50.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:50.463+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:50.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:50.463+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:50.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:50.463+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:50.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:50.463+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:50.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:50 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:50.463+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:50.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:50 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:50.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:50.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:50 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:50.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:50.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:50 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:50.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:50.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:50 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:50.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:50.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:50 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:50.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:50.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:50 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:50.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:51.292+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:51.292+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:51.292+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:51.292+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:51.292+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:51.489+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:51.489+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:51.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:51.489+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:51.608 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:51.489+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:51.608 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:51 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:51.489+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:24.350167+0000 front 2024-03-21T09:24:24.350190+0000 (oldest deadline 2024-03-21T09:24:49.050244+0000) 2024-03-21T09:24:51.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:51.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:51.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:51.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:51.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:51.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:51.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:51.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:51.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:51.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:51.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:51 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:51.503+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:51.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:51 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:51.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:51.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:51 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:51.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:51.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:51 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:51.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:51.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:51 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:51.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:51.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:51 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:51.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:51.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:51 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:51.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:52.288+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:52.288+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:52.288+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:52.288+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:52.288+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:52.529+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.843266+0000 front 2024-03-21T09:23:41.843336+0000 (oldest deadline 2024-03-21T09:24:05.843648+0000) 2024-03-21T09:24:52.607 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:52.529+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:59.147087+0000 front 2024-03-21T09:23:59.147188+0000 (oldest deadline 2024-03-21T09:24:23.847172+0000) 2024-03-21T09:24:52.608 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:52.529+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.548314+0000 front 2024-03-21T09:24:08.548164+0000 (oldest deadline 2024-03-21T09:24:33.748557+0000) 2024-03-21T09:24:52.608 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:52.529+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:13.748998+0000 front 2024-03-21T09:24:13.749010+0000 (oldest deadline 2024-03-21T09:24:44.349662+0000) 2024-03-21T09:24:52.608 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:52 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8[55006]: 2024-03-21T09:24:52.529+0000 7fa9c2be7700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:24.350167+0000 front 2024-03-21T09:24:24.350190+0000 (oldest deadline 2024-03-21T09:24:49.050244+0000) 2024-03-21T09:24:52.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:52.470+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:52.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:52.470+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:52.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:52.470+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:52.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:52.470+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:52.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:52.470+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:52.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:52 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:52.470+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:52.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:52 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:52.581+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:52.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:52 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:52.581+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:52.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:52 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:52.581+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:52.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:52 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:52.581+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:52.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:52 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:52.581+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:52.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:52 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:52.581+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:53.451 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:53 smithi071 podman[74376]: 2024-03-21 09:24:53.124806574 +0000 UTC m=+5.631290678 container died a16f126fc91baa86ca0d73b5a4752dd2be9a1795c3e415e5614457b5c4e6aef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:24:53.451 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:53 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:53.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:53.451 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:53 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:53.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:53.451 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:53 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:53.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:53.451 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:53 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:53.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:53.451 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:53 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:53.274+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:53.752 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:53 smithi071 podman[74376]: 2024-03-21 09:24:53.553118523 +0000 UTC m=+6.059602615 container cleanup a16f126fc91baa86ca0d73b5a4752dd2be9a1795c3e415e5614457b5c4e6aef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T09:24:53.752 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:53 smithi071 bash[74376]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8 2024-03-21T09:24:53.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:53.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:53.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:53.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:53.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:53.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:53.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:53.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:53.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:53.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:53.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:53 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:53.490+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:53.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:53.571+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:53.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:53.571+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:53.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:53.571+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:53.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:53.571+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:53.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:53.571+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:53.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:53.571+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:54.107 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:53 smithi071 podman[74394]: 2024-03-21 09:24:53.754306059 +0000 UTC m=+0.621914241 container remove a16f126fc91baa86ca0d73b5a4752dd2be9a1795c3e415e5614457b5c4e6aef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:24:54.438 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:54 smithi071 podman[74485]: 2024-03-21 09:24:54.154855626 +0000 UTC m=+0.103572453 container create ff92abfeeb5f946588b239e8c27b5dd23be0ed24fec9389771248b4a97fa0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-21T09:24:54.438 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:54 smithi071 podman[74485]: 2024-03-21 09:24:54.068667794 +0000 UTC m=+0.017384636 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:24:54.438 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:54 smithi071 podman[74485]: 2024-03-21 09:24:54.365950174 +0000 UTC m=+0.314667010 container init ff92abfeeb5f946588b239e8c27b5dd23be0ed24fec9389771248b4a97fa0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T09:24:54.438 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:54 smithi071 podman[74485]: 2024-03-21 09:24:54.369312064 +0000 UTC m=+0.318028885 container start ff92abfeeb5f946588b239e8c27b5dd23be0ed24fec9389771248b4a97fa0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:24:54.439 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:54 smithi071 podman[74485]: 2024-03-21 09:24:54.440333008 +0000 UTC m=+0.389049855 container attach ff92abfeeb5f946588b239e8c27b5dd23be0ed24fec9389771248b4a97fa0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-21T09:24:54.439 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:54 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:54.251+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:54.439 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:54 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:54.251+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:54.439 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:54 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:54.251+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:54.439 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:54 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:54.251+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:54.439 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:54 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:54.251+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:54.857 INFO:journalctl@ceph.osd.8.smithi071.stdout:Mar 21 09:24:54 smithi071 podman[74485]: 2024-03-21 09:24:54.666191701 +0000 UTC m=+0.614908562 container died ff92abfeeb5f946588b239e8c27b5dd23be0ed24fec9389771248b4a97fa0db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T09:24:54.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:54.506+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:54.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:54.506+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:54.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:54.506+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:54.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:54.506+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:54.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:54.506+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:54.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:54 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:54.506+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:54.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:54 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:54.534+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:54.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:54 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:54.534+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:54.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:54 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:54.534+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:54.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:54 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:54.534+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:54.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:54 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:54.534+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:54.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:54 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:54.534+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:55.250 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:55 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:55.202+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:55.250 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:55 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:55.202+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:55.250 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:55 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:55.202+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:55.250 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:55 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:55.202+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:55.250 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:55 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:55.202+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:55.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.8.service' 2024-03-21T09:24:55.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:24:55.316 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-21T09:24:55.317 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-21T09:24:55.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.9 2024-03-21T09:24:55.677 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 systemd[1]: Stopping Ceph osd.9 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:24:55.678 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:55.678 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:55.678 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:55.678 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:55.678 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:55.678 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.487+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:55.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:55 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:55.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:55.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:55 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:55.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:55.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:55 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:55.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:55.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:55 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:55.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:55.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:55 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:55.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:55.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:55 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:55.556+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:56.427 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.929+0000 7f8ec0c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:24:56.427 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.929+0000 7f8ec0c00700 -1 osd.9 109 *** Got signal Terminated *** 2024-03-21T09:24:56.427 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:55 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:55.929+0000 7f8ec0c00700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:24:56.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:56.220+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:56.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:56.220+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:56.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:56.220+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:56.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:56.220+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:56.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:56 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:56.220+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:56.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:56.441+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:56.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:56.441+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:56.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:56.441+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:56.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:56.441+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:56.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:56.441+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:56.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:56 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:56.441+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:56.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:56.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:56.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:56.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:56.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:56.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:56.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:56 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:56.582+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:24:57.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:57 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:57.236+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:57.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:57 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:57.236+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:57.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:57 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:57.236+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:57.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:57 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:57.236+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:57.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:57 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:57.236+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:57.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:57 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:57.488+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:57.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:57 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:57.488+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:57.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:57 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:57.488+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:57.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:57 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:57.488+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:57.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:57 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:57.488+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:57.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:57 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:57.488+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:57.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:57.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:57.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:57.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:57.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:57.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:57.947 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:57 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:57.621+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:24:58.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:58.212+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:58.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:58.212+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:58.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:58.212+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:58.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:58.212+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:58.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:58.212+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:58.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:58.212+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:24:58.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:58.526+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:58.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:58.526+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:58.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:58.526+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:58.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:58.526+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:58.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:58.526+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:58.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:58 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:58.526+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:58.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:58.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:58.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:58.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:58.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:58.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:58.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:58 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:58.592+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:24:59.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:59 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:59.250+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:24:59.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:59 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:59.250+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:24:59.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:59 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:59.250+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:24:59.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:59 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:59.250+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:24:59.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:59 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:59.250+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:24:59.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:24:59 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:24:59.250+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:24:59.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:59 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:59.550+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:24:59.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:59 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:59.550+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:24:59.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:59 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:59.550+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:24:59.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:59 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:59.550+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:24:59.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:59 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:59.550+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:24:59.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:24:59 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:24:59.550+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:24:59.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:24:59.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:24:59.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:24:59.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:24:59.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:24:59.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:24:59.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:24:59 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:24:59.585+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:00.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:00 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:00.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:00.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:00 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:00.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:00.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:00 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:00.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:00.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:00 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:00.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:00.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:00 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:00.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:00.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:00 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:00.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:00.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:25:00.544+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:41.570682+0000 front 2024-03-21T09:23:41.571261+0000 (oldest deadline 2024-03-21T09:24:07.470629+0000) 2024-03-21T09:25:00.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:25:00.544+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:53.372405+0000 front 2024-03-21T09:23:53.372335+0000 (oldest deadline 2024-03-21T09:24:17.373719+0000) 2024-03-21T09:25:00.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:25:00.544+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:24:01.376729+0000 front 2024-03-21T09:24:01.376856+0000 (oldest deadline 2024-03-21T09:24:24.876977+0000) 2024-03-21T09:25:00.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:25:00.544+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:09.579550+0000 front 2024-03-21T09:24:09.579727+0000 (oldest deadline 2024-03-21T09:24:35.379960+0000) 2024-03-21T09:25:00.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:25:00.544+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.881969+0000 front 2024-03-21T09:24:18.882027+0000 (oldest deadline 2024-03-21T09:24:41.683439+0000) 2024-03-21T09:25:00.928 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:00 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9[64111]: 2024-03-21T09:25:00.544+0000 7f8ebc400700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:26.385301+0000 front 2024-03-21T09:24:26.385365+0000 (oldest deadline 2024-03-21T09:24:49.885399+0000) 2024-03-21T09:25:00.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:00.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:00.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:00.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:00.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:00.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:00.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:00 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:00.607+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:01.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:01.258+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:01.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:01.258+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:01.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:01.258+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:01.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:01.258+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:01.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:01.258+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:01.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:01.258+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:01.677 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:01 smithi037 podman[99978]: 2024-03-21 09:25:01.422452536 +0000 UTC m=+5.629950348 container died c85d11914a520743ef679ba04a5cf760fa01ec78a5c948577bcf63c3012b7501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T09:25:01.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:01.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:01.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:01.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:01.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:01.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:01.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:01 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:01.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:02.021 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:01 smithi037 podman[99978]: 2024-03-21 09:25:01.838962412 +0000 UTC m=+6.046460255 container cleanup c85d11914a520743ef679ba04a5cf760fa01ec78a5c948577bcf63c3012b7501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:25:02.021 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:01 smithi037 bash[99978]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9 2024-03-21T09:25:02.427 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:02 smithi037 podman[99997]: 2024-03-21 09:25:02.023193309 +0000 UTC m=+0.591171711 container remove c85d11914a520743ef679ba04a5cf760fa01ec78a5c948577bcf63c3012b7501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:25:02.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:02.241+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:02.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:02.241+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:02.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:02.241+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:02.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:02.241+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:02.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:02.241+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:02.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:02 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:02.241+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:02.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:02.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:02.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:02.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:02.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:02.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:02.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:02 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:02.586+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:03.175 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:02 smithi037 podman[100083]: 2024-03-21 09:25:02.764203598 +0000 UTC m=+0.037328967 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:25:03.176 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:02 smithi037 podman[100083]: 2024-03-21 09:25:02.924068666 +0000 UTC m=+0.197194010 container create cdab1bfae08779c041ecd126a0f97f6f1380686b4bbd45e1999818e4cd3ce2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2) 2024-03-21T09:25:03.176 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:03 smithi037 podman[100083]: 2024-03-21 09:25:03.177582293 +0000 UTC m=+0.450707680 container init cdab1bfae08779c041ecd126a0f97f6f1380686b4bbd45e1999818e4cd3ce2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:25:03.427 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:03 smithi037 podman[100083]: 2024-03-21 09:25:03.188582208 +0000 UTC m=+0.461707582 container start cdab1bfae08779c041ecd126a0f97f6f1380686b4bbd45e1999818e4cd3ce2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:25:03.427 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:03 smithi037 podman[100083]: 2024-03-21 09:25:03.27640947 +0000 UTC m=+0.549534894 container attach cdab1bfae08779c041ecd126a0f97f6f1380686b4bbd45e1999818e4cd3ce2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:25:03.508 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:03.199+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:03.508 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:03.199+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:03.508 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:03.199+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:03.508 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:03.199+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:03.508 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:03.199+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:03.509 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:03 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:03.199+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:03.927 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 21 09:25:03 smithi037 podman[100083]: 2024-03-21 09:25:03.650573538 +0000 UTC m=+0.923698879 container died cdab1bfae08779c041ecd126a0f97f6f1380686b4bbd45e1999818e4cd3ce2a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T09:25:03.945 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:03.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:03.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:03.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:03.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:03.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:03.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:03 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:03.605+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:04.403 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.9.service' 2024-03-21T09:25:04.520 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:04.249+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:04.521 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:04.249+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:04.521 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:04.249+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:04.521 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:04.249+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:04.521 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:04.249+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:04.521 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:04 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:04.249+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:04.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:04.522 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-21T09:25:04.522 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-21T09:25:04.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.10 2024-03-21T09:25:04.850 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 systemd[1]: Stopping Ceph osd.10 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.611+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.852+0000 7f77eef2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.852+0000 7f77eef2b700 -1 osd.10 109 *** Got signal Terminated *** 2024-03-21T09:25:04.851 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:04 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:04.852+0000 7f77eef2b700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:05.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:05.261+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:05.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:05 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:05.658+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:06.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:06.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:06.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:06.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:06.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:06.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:06.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:06 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:06.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:06.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:06 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:06.623+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:07.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:07 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:07.213+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:07.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:07 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:07.590+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:08.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:08 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:08.224+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:08.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:08.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:08.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:08.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:08.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:08.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:08.947 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:08 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:08.620+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:09.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:09.223+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:42.362647+0000 front 2024-03-21T09:23:42.362943+0000 (oldest deadline 2024-03-21T09:24:05.262410+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-03-21T09:23:51.063955+0000 front 2024-03-21T09:23:51.063990+0000 (oldest deadline 2024-03-21T09:24:16.964180+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:56.964720+0000 front 2024-03-21T09:23:56.964905+0000 (oldest deadline 2024-03-21T09:24:26.965095+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:07.466453+0000 front 2024-03-21T09:24:07.466286+0000 (oldest deadline 2024-03-21T09:24:32.166112+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.067232+0000 front 2024-03-21T09:24:18.067195+0000 (oldest deadline 2024-03-21T09:24:43.967372+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:23.967923+0000 front 2024-03-21T09:24:23.967827+0000 (oldest deadline 2024-03-21T09:24:49.867774+0000) 2024-03-21T09:25:09.946 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:09 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10[61414]: 2024-03-21T09:25:09.631+0000 7f77e7517700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:34.569166+0000 front 2024-03-21T09:24:34.569208+0000 (oldest deadline 2024-03-21T09:24:56.268745+0000) 2024-03-21T09:25:10.446 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:10 smithi053 podman[73656]: 2024-03-21 09:25:10.173381981 +0000 UTC m=+5.448821309 container died 6da9ca9edd1ee2d418b98fe92801cc7d835a3ae695704a725a89a538a0dcb5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T09:25:10.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:10.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:10.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:10.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:10.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:10.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:10.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:10 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:10.185+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:10.784 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:10 smithi053 podman[73656]: 2024-03-21 09:25:10.594098875 +0000 UTC m=+5.869538195 container cleanup 6da9ca9edd1ee2d418b98fe92801cc7d835a3ae695704a725a89a538a0dcb5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:25:10.784 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:10 smithi053 bash[73656]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10 2024-03-21T09:25:11.196 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:10 smithi053 podman[73673]: 2024-03-21 09:25:10.786900895 +0000 UTC m=+0.614930338 container remove 6da9ca9edd1ee2d418b98fe92801cc7d835a3ae695704a725a89a538a0dcb5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T09:25:11.588 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:11 smithi053 podman[73766]: 2024-03-21 09:25:11.118509116 +0000 UTC m=+0.018426197 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:25:11.589 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:11 smithi053 podman[73766]: 2024-03-21 09:25:11.229733512 +0000 UTC m=+0.129650632 container create 0908913521eccdec0aa702c0cad569d23b9430531ee266047946547dce66702b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T09:25:11.589 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:11 smithi053 podman[73766]: 2024-03-21 09:25:11.591161791 +0000 UTC m=+0.491078869 container init 0908913521eccdec0aa702c0cad569d23b9430531ee266047946547dce66702b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:11.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:11 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:11.208+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:11.887 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:11 smithi053 podman[73766]: 2024-03-21 09:25:11.594394962 +0000 UTC m=+0.494312040 container start 0908913521eccdec0aa702c0cad569d23b9430531ee266047946547dce66702b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:25:11.887 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:11 smithi053 podman[73766]: 2024-03-21 09:25:11.665933897 +0000 UTC m=+0.565850984 container attach 0908913521eccdec0aa702c0cad569d23b9430531ee266047946547dce66702b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T09:25:12.166 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:11 smithi053 podman[73766]: 2024-03-21 09:25:11.889810083 +0000 UTC m=+0.789727174 container died 0908913521eccdec0aa702c0cad569d23b9430531ee266047946547dce66702b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T09:25:12.429 INFO:journalctl@ceph.osd.10.smithi053.stdout:Mar 21 09:25:12 smithi053 podman[73792]: 2024-03-21 09:25:12.413194574 +0000 UTC m=+0.517917389 container remove 0908913521eccdec0aa702c0cad569d23b9430531ee266047946547dce66702b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T09:25:12.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.10.service' 2024-03-21T09:25:12.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:12.480 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-21T09:25:12.481 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-21T09:25:12.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.11 2024-03-21T09:25:12.511 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:12.511 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:12.512 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:12.512 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:12.513 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:12.513 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:12.513 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.186+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:12.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 systemd[1]: Stopping Ceph osd.11 for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:12.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.762+0000 7efe7d007700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:12.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.762+0000 7efe7d007700 -1 osd.11 109 *** Got signal Terminated *** 2024-03-21T09:25:12.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:12 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:12.762+0000 7efe7d007700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.151+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.151+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.151+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.151+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.151+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.152+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:13.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:13 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:13.152+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:14.563 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:14.564 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:14.564 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:14.564 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:14.564 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:14.564 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:14.564 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:14 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:14.159+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:15.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:15 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:15.198+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6820 osd.8 since back 2024-03-21T09:24:49.816390+0000 front 2024-03-21T09:24:49.816430+0000 (oldest deadline 2024-03-21T09:25:15.116386+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:16.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:16 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:16.226+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6820 osd.8 since back 2024-03-21T09:24:49.816390+0000 front 2024-03-21T09:24:49.816430+0000 (oldest deadline 2024-03-21T09:25:15.116386+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-21T09:23:44.005964+0000 front 2024-03-21T09:23:44.005921+0000 (oldest deadline 2024-03-21T09:24:07.406265+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-03-21T09:23:57.407964+0000 front 2024-03-21T09:23:57.408055+0000 (oldest deadline 2024-03-21T09:24:22.108226+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-21T09:24:08.510230+0000 front 2024-03-21T09:24:08.509990+0000 (oldest deadline 2024-03-21T09:24:30.809858+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-03-21T09:24:18.912351+0000 front 2024-03-21T09:24:18.912266+0000 (oldest deadline 2024-03-21T09:24:42.412199+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-03-21T09:24:22.412695+0000 front 2024-03-21T09:24:22.412759+0000 (oldest deadline 2024-03-21T09:24:49.413311+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-21T09:24:29.414330+0000 front 2024-03-21T09:24:29.414126+0000 (oldest deadline 2024-03-21T09:24:57.614212+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-03-21T09:24:41.115465+0000 front 2024-03-21T09:24:41.115402+0000 (oldest deadline 2024-03-21T09:25:05.215113+0000) 2024-03-21T09:25:17.607 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:17 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11[61902]: 2024-03-21T09:25:17.253+0000 7efe755f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.71:6820 osd.8 since back 2024-03-21T09:24:49.816390+0000 front 2024-03-21T09:24:49.816430+0000 (oldest deadline 2024-03-21T09:25:15.116386+0000) 2024-03-21T09:25:18.517 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:18 smithi071 podman[74602]: 2024-03-21 09:25:18.255693951 +0000 UTC m=+5.579380234 container died da17d48af0a8cb635e4c9e55ce2335f87fb90b3eb17f5ed8e220ebf8995063bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:25:18.775 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:18 smithi071 podman[74602]: 2024-03-21 09:25:18.593942515 +0000 UTC m=+5.917628780 container cleanup da17d48af0a8cb635e4c9e55ce2335f87fb90b3eb17f5ed8e220ebf8995063bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T09:25:18.775 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:18 smithi071 bash[74602]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11 2024-03-21T09:25:19.107 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:18 smithi071 podman[74618]: 2024-03-21 09:25:18.778206254 +0000 UTC m=+0.523797992 container remove da17d48af0a8cb635e4c9e55ce2335f87fb90b3eb17f5ed8e220ebf8995063bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T09:25:19.479 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:19 smithi071 podman[74711]: 2024-03-21 09:25:19.100434312 +0000 UTC m=+0.017618044 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:25:19.479 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:19 smithi071 podman[74711]: 2024-03-21 09:25:19.203930248 +0000 UTC m=+0.121113975 container create e8f66751ee1ee339458dc5e5d725efc082afb9a6f643cc5ed694ae20918a3e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T09:25:19.479 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:19 smithi071 podman[74711]: 2024-03-21 09:25:19.406552101 +0000 UTC m=+0.323735835 container init e8f66751ee1ee339458dc5e5d725efc082afb9a6f643cc5ed694ae20918a3e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-03-21T09:25:19.479 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:19 smithi071 podman[74711]: 2024-03-21 09:25:19.409848707 +0000 UTC m=+0.327032436 container start e8f66751ee1ee339458dc5e5d725efc082afb9a6f643cc5ed694ae20918a3e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T09:25:19.479 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:19 smithi071 podman[74711]: 2024-03-21 09:25:19.481282496 +0000 UTC m=+0.398466253 container attach e8f66751ee1ee339458dc5e5d725efc082afb9a6f643cc5ed694ae20918a3e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T09:25:19.857 INFO:journalctl@ceph.osd.11.smithi071.stdout:Mar 21 09:25:19 smithi071 podman[74711]: 2024-03-21 09:25:19.700551504 +0000 UTC m=+0.617735234 container died e8f66751ee1ee339458dc5e5d725efc082afb9a6f643cc5ed694ae20918a3e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-osd-11-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T09:25:20.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@osd.11.service' 2024-03-21T09:25:20.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:20.394 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-21T09:25:20.395 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-21T09:25:20.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.a 2024-03-21T09:25:20.927 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:20 smithi037 systemd[1]: Stopping Ceph mds.a for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:21.427 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:20 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: 2024-03-21T09:25:20.980+0000 7f8a86600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:21.427 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:20 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a[70641]: 2024-03-21T09:25:20.980+0000 7f8a86600700 -1 mds.a *** got signal Terminated *** 2024-03-21T09:25:24.409 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:24 smithi037 podman[100200]: 2024-03-21 09:25:24.112532852 +0000 UTC m=+3.263881287 container died 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T09:25:24.677 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:24 smithi037 podman[100200]: 2024-03-21 09:25:24.522956837 +0000 UTC m=+3.674305266 container cleanup 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:25:24.677 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:24 smithi037 bash[100200]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a 2024-03-21T09:25:25.177 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 21 09:25:24 smithi037 podman[100214]: 2024-03-21 09:25:24.821350641 +0000 UTC m=+0.691412954 container remove 69efc5c45f13c1d8ba1f36a485fb3abfa587a4805f8705d3e73c967ac7249eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-a, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T09:25:25.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.a.service' 2024-03-21T09:25:25.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:25.473 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-21T09:25:25.473 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-21T09:25:25.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.d 2024-03-21T09:25:25.927 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:25 smithi037 systemd[1]: Stopping Ceph mds.d for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:26.427 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d[69644]: 2024-03-21T09:25:26.073+0000 7f7be6400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:26.427 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:26 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d[69644]: 2024-03-21T09:25:26.073+0000 7f7be6400700 -1 mds.d *** got signal Terminated *** 2024-03-21T09:25:29.920 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:29 smithi037 podman[100319]: 2024-03-21 09:25:29.661611912 +0000 UTC m=+3.720302754 container died 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:25:30.177 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:30 smithi037 podman[100319]: 2024-03-21 09:25:30.05114052 +0000 UTC m=+4.109831355 container cleanup 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T09:25:30.177 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:30 smithi037 bash[100319]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d 2024-03-21T09:25:30.677 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 21 09:25:30 smithi037 podman[100332]: 2024-03-21 09:25:30.294031167 +0000 UTC m=+0.612114937 container remove 6c7cf65ee58e34c20e6b4b0220407c53d3ee85128cdbb7d93694861d331dc654 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-21T09:25:30.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.d.service' 2024-03-21T09:25:30.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:30.944 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-21T09:25:30.944 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-21T09:25:30.944 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.g 2024-03-21T09:25:31.427 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:31 smithi037 systemd[1]: Stopping Ceph mds.g for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:31.927 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g[70277]: 2024-03-21T09:25:31.552+0000 7f1948200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:31.927 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:31 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g[70277]: 2024-03-21T09:25:31.552+0000 7f1948200700 -1 mds.g *** got signal Terminated *** 2024-03-21T09:25:35.865 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:35 smithi037 podman[100437]: 2024-03-21 09:25:35.576045542 +0000 UTC m=+4.135017739 container died 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T09:25:36.177 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:35 smithi037 podman[100437]: 2024-03-21 09:25:35.971312923 +0000 UTC m=+4.530285140 container cleanup 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T09:25:36.177 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:35 smithi037 bash[100437]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g 2024-03-21T09:25:36.586 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 21 09:25:36 smithi037 podman[100451]: 2024-03-21 09:25:36.329299254 +0000 UTC m=+0.742792623 container remove 842b11d99e5344246b80016c319edb41d4588d57902710d2ca0be814a6cafa96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T09:25:36.617 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.g.service' 2024-03-21T09:25:36.740 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:36.740 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-21T09:25:36.740 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-21T09:25:36.740 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.j 2024-03-21T09:25:37.177 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:36 smithi037 systemd[1]: Stopping Ceph mds.j for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:37.677 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j[69282]: 2024-03-21T09:25:37.365+0000 7fe6af000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:37.677 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:37 smithi037 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j[69282]: 2024-03-21T09:25:37.365+0000 7fe6af000700 -1 mds.j *** got signal Terminated *** 2024-03-21T09:25:41.662 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:41 smithi037 podman[100556]: 2024-03-21 09:25:41.381977681 +0000 UTC m=+4.154530528 container died 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T09:25:41.927 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:41 smithi037 podman[100556]: 2024-03-21 09:25:41.766716404 +0000 UTC m=+4.539269251 container cleanup 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-21T09:25:41.927 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:41 smithi037 bash[100556]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j 2024-03-21T09:25:42.427 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 21 09:25:41 smithi037 podman[100582]: 2024-03-21 09:25:41.969206007 +0000 UTC m=+0.567708115 container remove 09024db02b671880f2a407b1431b9d99b4dec4dd7381dc6b1a5b38ffe146f77f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:25:42.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.j.service' 2024-03-21T09:25:42.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:42.607 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-21T09:25:42.607 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-21T09:25:42.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.b 2024-03-21T09:25:42.920 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:42 smithi053 systemd[1]: Stopping Ceph mds.b for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:43.195 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b[66840]: 2024-03-21T09:25:42.922+0000 7f8631b15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:43.196 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:42 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b[66840]: 2024-03-21T09:25:42.922+0000 7f8631b15700 -1 mds.b *** got signal Terminated *** 2024-03-21T09:25:44.327 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:44 smithi053 podman[73872]: 2024-03-21 09:25:44.051313609 +0000 UTC m=+1.245967633 container died 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:25:44.327 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:44 smithi053 podman[73872]: 2024-03-21 09:25:44.330013277 +0000 UTC m=+1.524667293 container cleanup 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-21T09:25:44.690 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:44 smithi053 bash[73872]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b 2024-03-21T09:25:44.690 INFO:journalctl@ceph.mds.b.smithi053.stdout:Mar 21 09:25:44 smithi053 podman[73897]: 2024-03-21 09:25:44.564514905 +0000 UTC m=+0.512517164 container remove 1aa4f9723303c8391e1e3d478f03bd225783a234260f5860df2246ba64fce86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:25:44.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.b.service' 2024-03-21T09:25:44.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:44.751 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-21T09:25:44.751 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-21T09:25:44.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.e 2024-03-21T09:25:45.082 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:25:44 smithi053 systemd[1]: Stopping Ceph mds.e for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:45.446 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:25:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e[66215]: 2024-03-21T09:25:45.083+0000 7f5685745700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:45.446 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:25:45 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e[66215]: 2024-03-21T09:25:45.083+0000 7f5685745700 -1 mds.e *** got signal Terminated *** 2024-03-21T09:25:48.946 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:25:48 smithi053 podman[74005]: 2024-03-21 09:25:48.688228527 +0000 UTC m=+3.714012920 container died 3408ef0175f3f1547279c258c924712cb3b4b5e263944707d9f6415ec1c8aa9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T09:25:49.324 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:25:49 smithi053 podman[74005]: 2024-03-21 09:25:49.04200455 +0000 UTC m=+4.067788945 container cleanup 3408ef0175f3f1547279c258c924712cb3b4b5e263944707d9f6415ec1c8aa9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:25:49.325 INFO:journalctl@ceph.mds.e.smithi053.stdout:Mar 21 09:25:49 smithi053 bash[74005]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-e 2024-03-21T09:25:49.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.e.service' 2024-03-21T09:25:49.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:49.509 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-21T09:25:49.509 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-21T09:25:49.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.h 2024-03-21T09:25:49.843 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:49 smithi053 systemd[1]: Stopping Ceph mds.h for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:50.196 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h[66530]: 2024-03-21T09:25:49.845+0000 7fee9d59e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:50.196 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:49 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h[66530]: 2024-03-21T09:25:49.845+0000 7fee9d59e700 -1 mds.h *** got signal Terminated *** 2024-03-21T09:25:52.329 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:52 smithi053 podman[74129]: 2024-03-21 09:25:52.037545902 +0000 UTC m=+2.296215684 container died ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:25:52.329 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:52 smithi053 podman[74129]: 2024-03-21 09:25:52.329664527 +0000 UTC m=+2.588334300 container cleanup ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-21T09:25:52.695 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:52 smithi053 bash[74129]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h 2024-03-21T09:25:52.695 INFO:journalctl@ceph.mds.h.smithi053.stdout:Mar 21 09:25:52 smithi053 podman[74144]: 2024-03-21 09:25:52.564552281 +0000 UTC m=+0.517485049 container remove ca54b8f9e05ab62d7ecfebe00221ea8f6902460289ead0b8cfb9b1f1a214b7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T09:25:52.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.h.service' 2024-03-21T09:25:52.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:52.743 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-21T09:25:52.743 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-21T09:25:52.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.k 2024-03-21T09:25:53.065 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:25:52 smithi053 systemd[1]: Stopping Ceph mds.k for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:53.446 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:25:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k[65896]: 2024-03-21T09:25:53.067+0000 7f0a2b2ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:53.446 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:25:53 smithi053 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k[65896]: 2024-03-21T09:25:53.067+0000 7f0a2b2ee700 -1 mds.k *** got signal Terminated *** 2024-03-21T09:25:57.446 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:25:57 smithi053 podman[74252]: 2024-03-21 09:25:57.07957305 +0000 UTC m=+4.114551471 container died f4cb4bb953a1920be476b22f56d7dbe6b968f9ae815cc16fd4ee426931ee891c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-21T09:25:57.783 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:25:57 smithi053 podman[74252]: 2024-03-21 09:25:57.576148315 +0000 UTC m=+4.611126738 container cleanup f4cb4bb953a1920be476b22f56d7dbe6b968f9ae815cc16fd4ee426931ee891c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T09:25:57.783 INFO:journalctl@ceph.mds.k.smithi053.stdout:Mar 21 09:25:57 smithi053 bash[74252]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-k 2024-03-21T09:25:57.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.k.service' 2024-03-21T09:25:57.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:25:57.929 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-21T09:25:57.929 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-21T09:25:57.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.c 2024-03-21T09:25:58.238 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:25:57 smithi071 systemd[1]: Stopping Ceph mds.c for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:25:58.607 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:25:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c[67367]: 2024-03-21T09:25:58.239+0000 7fc6f01d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:25:58.607 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:25:58 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c[67367]: 2024-03-21T09:25:58.239+0000 7fc6f01d5700 -1 mds.c *** got signal Terminated *** 2024-03-21T09:26:00.313 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:26:00 smithi071 podman[74817]: 2024-03-21 09:26:00.046179589 +0000 UTC m=+1.921697508 container died 7250c6777bcb59dbce6aa43271a41a0824992c4a86f881638ea3b9d5d1a3f051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-21T09:26:00.607 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:26:00 smithi071 podman[74817]: 2024-03-21 09:26:00.397310958 +0000 UTC m=+2.272828875 container cleanup 7250c6777bcb59dbce6aa43271a41a0824992c4a86f881638ea3b9d5d1a3f051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T09:26:00.607 INFO:journalctl@ceph.mds.c.smithi071.stdout:Mar 21 09:26:00 smithi071 bash[74817]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-c 2024-03-21T09:26:00.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.c.service' 2024-03-21T09:26:00.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:26:00.805 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-21T09:26:00.805 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-21T09:26:00.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.f 2024-03-21T09:26:01.107 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:26:00 smithi071 systemd[1]: Stopping Ceph mds.f for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:26:01.607 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:26:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f[67056]: 2024-03-21T09:26:01.117+0000 7fda5a48c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:26:01.607 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:26:01 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f[67056]: 2024-03-21T09:26:01.117+0000 7fda5a48c700 -1 mds.f *** got signal Terminated *** 2024-03-21T09:26:04.307 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:26:04 smithi071 podman[74951]: 2024-03-21 09:26:04.04907606 +0000 UTC m=+3.026756311 container died b4766b14ca938b02813a8ad6d5f232a6579e3579578214efc0cbf849890905d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T09:26:04.607 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:26:04 smithi071 podman[74951]: 2024-03-21 09:26:04.40035737 +0000 UTC m=+3.378037621 container cleanup b4766b14ca938b02813a8ad6d5f232a6579e3579578214efc0cbf849890905d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T09:26:04.607 INFO:journalctl@ceph.mds.f.smithi071.stdout:Mar 21 09:26:04 smithi071 bash[74951]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-f 2024-03-21T09:26:04.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.f.service' 2024-03-21T09:26:04.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:26:04.906 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-21T09:26:04.907 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-21T09:26:04.907 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.i 2024-03-21T09:26:05.227 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:26:04 smithi071 systemd[1]: Stopping Ceph mds.i for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:26:05.607 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:26:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i[67673]: 2024-03-21T09:26:05.229+0000 7f47cbaf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:26:05.607 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:26:05 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i[67673]: 2024-03-21T09:26:05.229+0000 7f47cbaf7700 -1 mds.i *** got signal Terminated *** 2024-03-21T09:26:08.357 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:26:08 smithi071 podman[75073]: 2024-03-21 09:26:08.047139422 +0000 UTC m=+2.924554184 container died d9b59ea58aaf572c0e8ab697f9ce4d5a15a2b28f14378d0c7d17fe3f38acd552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:26:08.677 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:26:08 smithi071 podman[75073]: 2024-03-21 09:26:08.453791083 +0000 UTC m=+3.331205846 container cleanup d9b59ea58aaf572c0e8ab697f9ce4d5a15a2b28f14378d0c7d17fe3f38acd552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:26:08.677 INFO:journalctl@ceph.mds.i.smithi071.stdout:Mar 21 09:26:08 smithi071 bash[75073]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-i 2024-03-21T09:26:08.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.i.service' 2024-03-21T09:26:08.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:26:08.840 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-21T09:26:08.840 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-21T09:26:08.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.l 2024-03-21T09:26:09.357 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:08 smithi071 systemd[1]: Stopping Ceph mds.l for 2dc01c4e-e761-11ee-95cd-87774f69a715... 2024-03-21T09:26:09.358 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-l[66747]: 2024-03-21T09:26:09.157+0000 7f4f46f12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:26:09.358 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:09 smithi071 ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-l[66747]: 2024-03-21T09:26:09.157+0000 7f4f46f12700 -1 mds.l *** got signal Terminated *** 2024-03-21T09:26:10.805 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:10 smithi071 podman[75194]: 2024-03-21 09:26:10.559974071 +0000 UTC m=+1.497815397 container died 329a41c2a8979c0d862f3356bbe5fe66c95413df0a43f7534970a6374b08876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-03-21T09:26:11.072 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:10 smithi071 podman[75194]: 2024-03-21 09:26:10.890451244 +0000 UTC m=+1.828292558 container cleanup 329a41c2a8979c0d862f3356bbe5fe66c95413df0a43f7534970a6374b08876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-l, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T09:26:11.072 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:10 smithi071 bash[75194]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-l 2024-03-21T09:26:11.319 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.l.service' 2024-03-21T09:26:11.343 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:11 smithi071 podman[75208]: 2024-03-21 09:26:11.07451928 +0000 UTC m=+0.512222162 container remove 329a41c2a8979c0d862f3356bbe5fe66c95413df0a43f7534970a6374b08876c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-2dc01c4e-e761-11ee-95cd-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T09:26:11.343 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:11 smithi071 systemd[1]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.l.service: Deactivated successfully. 2024-03-21T09:26:11.343 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:11 smithi071 systemd[1]: Stopped Ceph mds.l for 2dc01c4e-e761-11ee-95cd-87774f69a715. 2024-03-21T09:26:11.343 INFO:journalctl@ceph.mds.l.smithi071.stdout:Mar 21 09:26:11 smithi071 systemd[1]: ceph-2dc01c4e-e761-11ee-95cd-87774f69a715@mds.l.service: Consumed 1.008s CPU time. 2024-03-21T09:26:11.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:26:11.353 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-21T09:26:11.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T09:26:11.688 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:26:16.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T09:26:16.881 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:26:19.515 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T09:26:19.704 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:26:22.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:26:22.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:26:22.315 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:26:22.343 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T09:26:22.345 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi037/crash 2024-03-21T09:26:22.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash -- . 2024-03-21T09:26:22.466 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T09:26:22.466 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-03-21T09:26:22.473 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi053/crash 2024-03-21T09:26:22.474 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash -- . 2024-03-21T09:26:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T09:26:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-03-21T09:26:22.523 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi071/crash 2024-03-21T09:26:22.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash -- . 2024-03-21T09:26:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T09:26:22.572 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-03-21T09:26:22.573 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T09:26:22.573 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T09:26:22.688 INFO:teuthology.orchestra.run.smithi037.stdout:2024-03-21T09:01:54.658535+0000 mon.a (mon.0) 186 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:26:22.690 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T09:26:22.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[SEC\]' /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T09:26:22.786 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]' /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T09:26:22.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[WRN\]' /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T09:26:22.970 INFO:teuthology.orchestra.run.smithi037.stdout:2024-03-21T09:01:54.658535+0000 mon.a (mon.0) 186 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:26:22.971 INFO:tasks.cephadm:Compressing logs... 2024-03-21T09:26:22.972 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T09:26:22.976 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T09:26:22.978 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T09:26:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T09:26:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-03-21T09:26:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-21T09:26:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T09:26:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-21T09:26:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-21T09:26:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-21T09:26:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.i.log: /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-21T09:26:23.153 INFO:teuthology.orchestra.run.smithi071.stderr: 92.4%/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.z.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T09:26:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T09:26:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-21T09:26:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.3.log: 90.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.x.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T09:26:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr: 94.4% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr: 83.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log: /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.9.log: /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.log: 90.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr: 94.4% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 92.3% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-21T09:26:23.169 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr: 94.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.2.log: 87.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-21T09:26:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T09:26:23.171 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T09:26:23.171 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.c.log: 94.6% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-21T09:26:23.171 INFO:teuthology.orchestra.run.smithi071.stderr: 93.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-21T09:26:23.173 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T09:26:23.173 INFO:teuthology.orchestra.run.smithi053.stderr:: No such file or directory 2024-03-21T09:26:23.173 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-21T09:26:23.173 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T09:26:23.173 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.10.log: -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-21T09:26:23.173 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log: gzip 92.1% -5 -- replaced with /var/log/ceph/cephadm.log.gz --verbose 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr: -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.b.log: /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.y.log: 94.4% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T09:26:23.174 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T09:26:23.175 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log: 87.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T09:26:23.175 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T09:26:23.175 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T09:26:23.175 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T09:26:23.175 INFO:teuthology.orchestra.run.smithi053.stderr: 93.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-21T09:26:23.431 INFO:teuthology.orchestra.run.smithi071.stderr: 91.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T09:26:23.439 INFO:teuthology.orchestra.run.smithi037.stderr: 89.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T09:26:23.458 INFO:teuthology.orchestra.run.smithi037.stderr: 92.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-21T09:26:23.485 INFO:teuthology.orchestra.run.smithi053.stderr: 95.8% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-21T09:26:23.679 INFO:teuthology.orchestra.run.smithi053.stderr: 91.6% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T09:26:24.240 INFO:teuthology.orchestra.run.smithi037.stderr: 90.4% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T09:26:24.800 INFO:teuthology.orchestra.run.smithi071.stderr: 91.9% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-21T09:26:24.828 INFO:teuthology.orchestra.run.smithi071.stderr: 92.7% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-21T09:26:24.940 INFO:teuthology.orchestra.run.smithi053.stderr: 92.1% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T09:26:25.211 INFO:teuthology.orchestra.run.smithi053.stderr: 92.6% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-21T09:26:25.285 INFO:teuthology.orchestra.run.smithi037.stderr: 92.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T09:26:25.617 INFO:teuthology.orchestra.run.smithi053.stderr: 92.3% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T09:26:25.773 INFO:teuthology.orchestra.run.smithi037.stderr: 92.0% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-21T09:26:25.886 INFO:teuthology.orchestra.run.smithi037.stderr: 92.3% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T09:26:25.907 INFO:teuthology.orchestra.run.smithi071.stderr: 93.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T09:26:26.125 INFO:teuthology.orchestra.run.smithi037.stderr: 92.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T09:26:26.128 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-03-21T09:26:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m3.122s 2024-03-21T09:26:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m12.024s 2024-03-21T09:26:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.965s 2024-03-21T09:26:26.368 INFO:teuthology.orchestra.run.smithi053.stderr: 92.6% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T09:26:26.370 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-03-21T09:26:26.370 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m3.376s 2024-03-21T09:26:26.370 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m11.001s 2024-03-21T09:26:26.370 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.355s 2024-03-21T09:26:26.623 INFO:teuthology.orchestra.run.smithi071.stderr: 93.2% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-21T09:26:28.077 INFO:teuthology.orchestra.run.smithi071.stderr: 94.4% -- replaced with /var/log/ceph/2dc01c4e-e761-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T09:26:28.079 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-03-21T09:26:28.079 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m5.086s 2024-03-21T09:26:28.079 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m15.281s 2024-03-21T09:26:28.079 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.508s 2024-03-21T09:26:28.079 INFO:tasks.cephadm:Archiving logs... 2024-03-21T09:26:28.079 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi037/log 2024-03-21T09:26:28.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T09:26:29.975 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi053/log 2024-03-21T09:26:29.976 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T09:26:31.623 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi071/log 2024-03-21T09:26:31.624 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T09:26:33.764 INFO:tasks.cephadm:Removing cluster... 2024-03-21T09:26:33.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --force 2024-03-21T09:26:34.111 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:26:34.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --force 2024-03-21T09:26:34.988 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:26:35.272 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2dc01c4e-e761-11ee-95cd-87774f69a715 --force 2024-03-21T09:26:35.462 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 2dc01c4e-e761-11ee-95cd-87774f69a715 2024-03-21T09:26:35.735 INFO:tasks.cephadm:Removing cephadm ... 2024-03-21T09:26:35.735 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T09:26:35.797 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T09:26:35.813 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T09:26:35.829 INFO:tasks.cephadm:Teardown complete 2024-03-21T09:26:35.829 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T09:26:35.840 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T09:26:35.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T09:26:35.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T09:26:35.856 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T09:26:36.034 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T09:26:36.045 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T09:26:36.046 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:26:36.054 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:26:36.056 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:26:36.070 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:26:36.070 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:26:36.074 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:26:36.074 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-03-21T09:26:36.074 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 -209us[ -172us] +/- 97ms 2024-03-21T09:26:36.074 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -74us[ -35us] +/- 75ms 2024-03-21T09:26:36.074 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +206us[ +228us] +/- 67ms 2024-03-21T09:26:36.074 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:26:36.075 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:26:36.075 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-03-21T09:26:36.075 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 65 -207us[ -182us] +/- 97ms 2024-03-21T09:26:36.075 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 6 377 65 -59us[ -34us] +/- 75ms 2024-03-21T09:26:36.076 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +270us[ +270us] +/- 68ms 2024-03-21T09:26:36.076 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:26:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:26:36.114 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:26:36.114 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-03-21T09:26:36.114 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 -223us[ -181us] +/- 97ms 2024-03-21T09:26:36.114 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -91us[ -49us] +/- 75ms 2024-03-21T09:26:36.114 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +243us[ +243us] +/- 68ms 2024-03-21T09:26:36.114 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:26:36.116 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T09:26:36.127 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T09:26:36.128 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T09:26:36.164 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T09:26:36.421 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T09:26:36.507 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T09:26:36.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:26:36.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:26:36.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:26:36.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:26:36.729 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:26:36.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:26:36.759 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T09:26:36.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T09:26:36.814 INFO:teuthology.task.internal:Duration was 2191.559771 seconds 2024-03-21T09:26:36.815 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T09:26:36.847 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T09:26:36.848 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T09:26:36.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T09:26:36.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T09:26:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:26:36.911 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:26:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:26:37.511 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T09:26:37.511 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-03-21T09:26:37.511 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T09:26:37.575 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-03-21T09:26:37.575 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T09:26:37.605 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-03-21T09:26:37.605 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T09:26:37.627 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T09:26:37.628 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T09:26:37.637 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T09:26:37.650 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T09:26:37.738 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T09:26:37.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T09:26:37.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T09:26:37.751 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T09:26:38.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T09:26:38.390 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T09:26:38.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T09:26:38.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T09:26:38.532 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T09:26:38.563 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T09:26:38.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:26:38.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:26:38.591 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:26:38.621 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-03-21T09:26:38.631 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-03-21T09:26:38.700 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-03-21T09:26:38.796 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:26:38.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:26:38.853 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:26:38.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:26:38.873 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:26:38.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:26:38.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T09:26:38.955 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T09:26:38.956 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi037 2024-03-21T09:26:38.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T09:26:39.100 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi053 2024-03-21T09:26:39.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T09:26:39.159 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163/remote/smithi071 2024-03-21T09:26:39.159 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T09:26:39.208 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T09:26:39.208 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T09:26:39.217 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T09:26:39.221 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T09:26:39.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T09:26:39.292 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T09:26:39.292 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T09:26:39.333 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T09:26:39.333 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T09:26:39.343 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T09:26:39.348 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T09:26:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 09:26 /home/ubuntu/cephtest 2024-03-21T09:26:39.365 INFO:teuthology.orchestra.run.smithi071.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 09:26 /home/ubuntu/cephtest 2024-03-21T09:26:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 09:26 /home/ubuntu/cephtest 2024-03-21T09:26:39.399 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T09:26:39.413 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T09:26:39.466 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T09:26:39.494 DEBUG:teuthology.nuke:shortname: smithi037 2024-03-21T09:26:39.494 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:26:39.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:39:53.941984', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFpGf8LP1jXp81CNatiBzOo4U2gTBprAuefXx8fImVU2Ss5b1rl6FZm+sbhRNHRZNI+cuqXHrIZFYaS04bfarqc='} 2024-03-21T09:26:39.520 DEBUG:teuthology.nuke:shortname: smithi053 2024-03-21T09:26:39.520 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:26:39.543 DEBUG:teuthology.nuke:shortname: smithi071 2024-03-21T09:26:39.543 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:26:39.548 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:39:53.940548', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhfYZAonPqb6ERwQS87seGftQ9BR7x+++8t5AMUqfN2+ubU4aiGl43nNY4ldEzJ1S7Lrdu+9ChnabsDyuNyICk='} 2024-03-21T09:26:39.573 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:39:53.943229', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN4EKKGF8alrFf6eH5MlbjtZcPbUdjEVRgKmGpBTA6OCZy78yvnrFhy0RMSAuDjEOZEI+JFgCLYwN26fC/6FSOU='} 2024-03-21T09:26:39.580 INFO:teuthology.orchestra.console.smithi037:Power off 2024-03-21T09:26:39.580 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T09:26:39.608 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-03-21T09:26:39.608 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:39.627 INFO:teuthology.orchestra.console.smithi053:Power off 2024-03-21T09:26:39.628 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T09:26:39.640 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-21T09:26:39.653 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-03-21T09:26:39.653 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:39.669 INFO:teuthology.orchestra.console.smithi071:Power off 2024-03-21T09:26:39.669 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T09:26:39.679 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-03-21T09:26:39.692 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-03-21T09:26:39.692 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:39.718 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-03-21T09:26:43.641 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:43.680 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:43.720 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:43.752 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-21T09:26:43.791 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-03-21T09:26:43.829 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-03-21T09:26:47.754 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:47.792 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:47.831 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:47.866 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-21T09:26:47.902 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-03-21T09:26:47.942 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-03-21T09:26:48.004 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-03-21T09:26:51.870 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:51.944 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:26:51.981 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-03-21T09:26:52.054 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-03-21T09:26:52.083 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-03-21T09:26:52.155 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-03-21T09:26:52.263 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} duration: 2191.5597705841064 failure_reason: '"2024-03-21T09:01:54.658535+0000 mon.a (mon.0) 186 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-03-21T09:26:52.264 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:26:52.351 INFO:teuthology.run:FAIL